Academic OB/GYN

When Ortho Evra, the contraceptive patch came on the market, physicians were happy that women had a new and novel contraceptive method that significantly expanded options over what was already available.  Since that time, hundreds of thousands of women worldwide have safely used Ortho Evra for birth control.

As the transdermal patch was a new delivery method for birth control, several new pharmacokinetic studies were done postmarketing, in addition to those done prior to FDA approval.  One of these studies demonstrated that patients using Ortho Evra had on average 60% higher estrogen blood levels than patients on oral contraceptives, despite lower peak levels (1).  Given that estrogen somewhat increases the risk of venous thromboembolism(VTE), this data raised the concern that Ortho Evra might confer a greater attributable risk than traditional oral contraceptives.

Due to this concern, several studies were performed and published with mixed results.   In 2006, a case control study was done that demonstrated no additional risk for VTE with Ortho Evra as compared to oral contraceptives (2).  In 2007, a retrospective cohort study showed that the risk of VTE with Ortho Evra was double what it was with oral contraceptives (3).   And this month in Contraception Jick et al reported an updated case control study that demonstrated that Ortho Evra was not associated with an increased risk of venous thromboembolism when compared to oral contraceptives (4).

After reading this new paper, I was intrigued and wanted to review all the data.  And per usual, when I do this I like to get a blog post out of it.  So lets look at the data!

2006 – Jick et al reported case-control data based on data collected by PharMetrics, a company that reports data based on clinical service and pharmaceutical insurance claims.  This study compared 68 new cases of VTE (cases) with 266 women (controls) who did not experience VTE, from a pool of women who had recently started either Ortho Evra or a norgestimate containing oral contraceptive (Ortho Tri Cyclen, Trinessa, Sprintec, and a few others.)  All cases and controls were new starts of their contraceptives during the study period.  Cases were identified based on diagnosis codes indicating that venous thromboembolism had occurred (DVT, Pulmonary Embolism) and were confirmed with evidence of subsequent long term anticoagulation.  Chart abstraction was not performed.   Medication exposure was identified based on prescription histories.

In this case control analysis, Ortho Evra was found not to be associated with an increased risk of VTE in comparison to OCPs, with an odds ratio of 0.9 (95% CI 0.5 – 1.6.)

2007 – Cole et al reported a different study, with a different data set and methodology.  This study was also based on insurance records, but instead of a case-control analysis a retrospective cohort study was done (Cohort: Exposure -> outcome vs Case-Control: Outcome -> Exposure).   49,048 woman-years of Ortho Evra exposure were compared to 02,344 woman-years of norgestimate OCPs were compared in their frequency of VTE, myocardial infarction, and stroke. Events were identified via diagnosis code search, but all positive outcomes were confirmed by chart abstraction and verification.  A nested case-control design was used to confirm similarity of the two groups to investigate demographic difference between affected and unaffected women.

This study found a different result than the 2006 data from Jick.  It demonstrated a more than two-fold increase in the rate of VTE in Ortho Evra users in comparison to OCP users (Relative Risk 2.2 95% CI 1.3-3.8).

2010 – Jick et al, reporting an expanded dataset of the 2006 study, and an additional analysis based on a new insurance dataset.   Study design was quite similar to the 2006 study.  Again, no increased rate of VTE was found with Ortho Evra users in comparison to OCP users, with an odds ratio of 2.0 (95% CI 0.9 – 4.1) for one dataset and 1.3 (0.8 – 2.1) for the other dataset.

So what the hell?   We have three different studies and no clear answer.   And this brings me to my first major point.

There is a correct answer. Ortho Evra is either associated with a greater risk of VTE than oral contraceptives or it isn’t.  They can’t both be true.  Therefore, one or more of the studies has an incorrect answer.

So there are two ways a study can be incorrect: 1) statistical error and 2) bias.

Statistical error would be based on pure mathematics.  Most studies are powered to have a 80% beta, or 80% chance of finding and difference between groups if there is one, and a 95% alpha, or a 95% chance that if there is no difference between groups none will be found.  If this sounds a bit like my descriptions of sensitivity and specificity in the last post, that is because alpha and beta are just different ways of referring to those concepts.

So a study can be wrong just because the numbers don’t work out.  Sometimes, based on statistical power, it gets the wrong answer.  Typically those studies will get the correct answer if they are repeated, as the chance of hitting the wrong one is low.  This is why any study worth doing is worth doing many times, which is what we generally do.

But bias is different.  Bias is not about mathematics.  It’s about a study design that systematically favors one outcome over another.   Sometimes a study is done twenty times and gets the same result, but because of bias and similar study design, all twenty results are wrong.   Bias, unlike stastistical error, cannot be overcome with increased study power or study repetition.

So we have three different studies that have two different answers.   At least one is wrong, and it is wrong either because of bias or statistics.

In this case, its bias.

Both studies by Jick have a fundamental study design error that biases the result.  Jick and the other authors noted that the use of Ortho Evra increased dramatically over the study period, and the use of OCPs decreased.  They felt that this might create bias in their outcome due to differing availability of the two drugs over the study period, so they decided to control this by making sure that every case and control was a new start on their method of choice.   I believe this badly biased their results.

If a patient was exposed to oral contraceptives as a new start, they were likely on nothing before, as there weren’t a lot of options before.   But if patient was exposed to Ortho Evra, they could have been on oral contraceptives before.  This created a bias.  Patients exposed to OCPs were way more likely to never have been exposed to a contraceptive, and therefore never have had the opportunity to have a estrogen related VTE.  Women on Ortho Evra, on the other hand, likely had been on OCPs before, and therefore did have an opportunity to have a prior OCP related VTE.  And if they had had that, they never would have used the Ortho Evra.  As such, the Ortho Evra exposed women in the study were actually a subset of all women that might have used Ortho Evra – they are women who used Ortho Evra who had already demonstrated the ability to not have a DVT on oral pills.  As DVT risk certainly has a large genetic component, this really skews this group, and systematically decreases the likelihood that VTE cases would have been exposed to Ortho Evra.  Ultimately, I think this is why Jick did not find a difference in VTE for Ortho Evra users, and lacking this bias, why Cole did.   Jick was concerned about the differing availability of the two drugs over the study period, but I believe this should have been controlled for in another way that would not have introduced a bias in favor of Ortho Evra.

So of these three studies, I believe the Cole data, and doubt the Jick data.  So I think that Ortho Evra does indeed increase VTE risk relative to OCPs, which makes the most sense, as it does have a higher level of estrogen exposure than OCPs.

So now for the second point –

You can’t trust abstracts alone. All three of these studies, if you just read the abstracts, appear to be properly done.   That’s because the abstract is just a tiny little summary of the entire manuscript, and says very little about study design  The abstract is paragraphs and the manuscript is six to ten single spaced pages.

In order to really understand a study, you have to read the whole thing.   And importantly, the first thing you have to look at is the Materials and Methods – how they did the study.  Only if the study is properly powered and does not have systematic bias should you really trust the conclusion.

I bring this up because of something I have noticed – the lay public is starting to read our papers, and often misunderstanding them.  I think that’s great (I really do), but I have to make this warning.   If you have not really trained in study design and analysis, its really hard to really know what medical papers say and which ones you can trust.   If you are one of my layperson audience and want to read medical papers, I implore you to spend some time learning how to read papers.  There are many references on this, but the single best volume I can recommend is a small book by David Grimes and Ken Shulz entitled “Handbook of Essential Concepts in Clinical Research” It is a fantastic book that is easy to read and will teach one all one needs to know to appropriately assess the quality and results of a paper.  I’m going to try to cover some points in future blog posts as well.

The other problem is that it can be hard to get the full manuscripts.  This is because medical journals have a completely different business model than regular magazines.   Unlike lay press that is supported by advertising, journals are supported by subscription fees.  A subscription to a journal is often several hundred dollars a year.  Medical libraries spend hundreds of thousands and sometimes millions of dollars a year maintaining an appropriately broad collection.  It would be great if journals made their content free to all, but as they have little advertising dollars, they would not make any money and could not exist. As a physician, I have access to a medical library and can get whatever I want online.  But laypeople lack this access.

So what should you do if you are not one of my physician or scientist readers?   Go read journals at your local hospital or library.  Every hospital has a library, and large hospitals have really big libraries.  Teaching hospitals may even have a university library that is just huge.  Usually you can get access to these libraries for free, and read all the journals you like.

It’s a bit of an effort, but if you want to read medical literature, you have to read the whole thing, and learn how to read it.   Abstracts aren’t enough.

1. van den Heuvel MW, van Bragt AJ, Alnabawy AK, Kaptein MC. Comparison of ethinylestradiol pharmacokinetics in three hormonal contraceptive formulations: the vaginal ring, the transdermal patch and an oral contraceptive. Contraception 2005 Sep;72(3):168-74.

2. Jick SS, Kaye JA, Russmann S, Jick H. Risk of nonfatal venous thromboembolism in women using a contraceptive transdermal patch and oral contraceptives containing norgestimate and 35 microg of ethinyl estradiol. Contraception 2006 Mar;73(3):223-8.

3. Cole JA, Norman H, Doherty M, Walker AM. Venous thromboembolism, myocardial infarction, and stroke among transdermal contraceptive system users. Obstet Gynecol 2007 Feb;109(2 Pt 1):339-46.

4. Jick SS, Hagberg KW, Hernandez RK, Kaye JA. Postmarketing study of ORTHO EVRA and levonorgestrel oral contraceptives containing hormonal contraceptives with 30 mcg of ethinyl estradiol in relation to nonfatal venous thromboembolism. Contraception  Jan;81(1):16-21.

PS Ortho Evra is still a great birth control method for many women.  Any estrogen birth control method increases VTE risk, but the effect is small.  Pregnancy also increases VTE risk.