Real-World Evidence: CDER, Covis Flipped The Script For Leveraging Makena Observational Studies
Executive Summary
US FDA’s drugs center defended its use of observational studies and randomized trials in other populations to argue that evidence of Makena’s efficacy in preventing preterm birth is lacking; Covis’ expert said observational studies have little to contribute and trials in non-indicated populations are irrelevant.
When it comes to the role of real-world evidence in drug regulatory decisions, it tends to be the sponsor trying to rely on such data to demonstrate efficacy and the US Food and Drug Administration raising questions about reliability and applicability.
However, in the recent Makena accelerated approval withdrawal hearing, the tables were turned on this usual dynamic. The FDA’s Center for Drug Evaluation and Research highlighted several RWE studies in its bid to establish that effectiveness has not been demonstrated for Makena’s use to prevent preterm birth, while sponsor Covis Pharma challenged the applicability of these studies.
CDER said it looked to real-world observational studies, as well as randomized clinical trials in populations beyond those for which Makena (hydroxyprogesterone caproate, also known as 17-OHPC or 17-P) currently is labeled, for pharmacologic evidence and other potential indicators of effectiveness. It found no such evidence of effect.
“Real-world observational studies have limitations. They can be confounded, and they reflect the limitations of how a drug is actually used in practice,” Office of New Drugs director Peter Stein said. “Yet, we found five studies that did have a reasonable design and found no evidence of HPC’s or Makena’s effectiveness.”
In contrast, Covis asserted the observational data available do not contribute much to the discussion, and the randomized trials in other populations are irrelevant.
CDER is proposing to withdraw Makena’s accelerated approval because the PROLONG study (Trial 003) failed to confirm the drug’s clinical benefit and the evidence does not establish that the drug is effective under its conditions of use. At the close of a two-and-a-half day hearing under the accelerated approval regulations, the Obstetrics, Reproductive and Urologic Drugs Advisory Committee voted 14-1 in favor of withdrawal. (Also see "US FDA Panel Says Covis’ Makena Should Be Withdrawn – And This Time It Wasn’t Close" - Pink Sheet, 19 Oct, 2022.)
Five Observational Studies
In its presentation to the advisory committee CDER said it looked for evidence of Makena’s benefit beyond the PROLONG trial and the earlier, smaller Meis study (Trial 002) that served as the basis for the drug’s accelerated approval in 2011.
“While the quality of evidence from the observational studies is not at the same level as RCTs, observational studies can provide additional evidence and, in particular, consistency across studies supports stronger conclusions,” said Laura Lee Johnson, director of the Division of Biometrics III.
Johnson said CDER conducted a literature search in PubMed and identified five observational studies of varying designs, settings and data sources involving women eligible to receive Makena. “We wanted to know would they support 002 or 003.”
She said three of these were recent cohort studies that controlled for a number of important confounders and relied on a variety of data sources – an academic tertiary care center in St. Louis, Mo., Medicaid enrollees in Pennsylvania, and commercial insurance claims.
“These studies represent the strongest observational studies reviewed for this program, and none demonstrated a significant effect of Makena on preterm birth,” Johnson said. “The replication of negative results, especially in higher quality recent studies, supports the Trial 003 findings regarding Makena’s lack of treatment effect.”
The remaining two observational studies cited by CDER used a historical control.
One study, conducted at a university teaching hospital in Dallas, Texas, found that the overall rate of recurrent preterm births for the entire cohort treated with HPC was comparable to the expected rate observed in the historical untreated obstetric population, Johnson said.
The final study (Bastek, et al.) compared rates of preterm birth at an urban academic medical center in Pennsylvania before and after implementation of Makena as standard of care. This study found no change in institutional preterm birth rate or gestational delivery age, Johnson said.
RCTs In Other Populations
Johnson also reviewed data from randomized, controlled trials in singleton gestations in populations that are not covered by Makena’s label.
One trial was conducted in HIV+ women in Zambia. Another enrolled US women with a shortened cervix and no prior preterm birth. The third study, conducted in France, enrolled women with a shortened cervix plus at least one other risk factor for preterm birth, and it studied a higher dose than Makena (HPC 500 mg, rather than 250 mg as labeled for Makena). All three studies failed to show treatment effect of HPC on preterm birth in populations distinct from Makena’s indicated population.
Data from several multiple gestation trials also do not support an effect of Makena on preterm birth, Johnson said.
“In summary, well-conducted observational studies do not show a response to Makena. RCTs in singleton and multi-gestation pregnancies also did not show response to HPC,” Johnson said. “There are a lot of randomized, double-blind, placebo-controlled trials here. A lot of real-world data and a lot of different populations. Trial 003 is not the outlier. The outlier is trial 002.”
“As part of approving a drug, CDER’s efficacy review would focus on the RCTs in the to-be-indicated population,” Johnson said. “In recommending withdrawal, beyond the legal grounds we have looked more broadly for scientific evidence to support Trial 002. We've looked at subgroups, observational studies, related indications. This is not a tale of two trials. There is a lot of evidence.”
‘The Same Message’
Raghav Chari, Covis’ chief innovation officer, noted that CDER previously has expressed concerns about the ability of observational studies to measure the endpoint of preterm birth, and he asked Johnson if she would agree that findings from such studies are not reliable in this regard.
In an April filing to the docket, CDER asserted that RWE would not be sufficient to verify clinical benefit for Makena given the failure of the PROLONG trial and limitations of the RWE study that Covis was proposing at that time. (Also see "Accelerated Approval: Makena Real-World Evidence Insufficient To Verify Clinical Benefit, CDER Says" - Pink Sheet, 21 Apr, 2022.)
“We cannot walk away from the real-world evidence studies. They have a role.” – CDER’s Laura Lee Johnson
“We cannot walk away from the real-world evidence studies. They have a role. That's why we pointed to them,” Johnson said. Despite the limitations of observational studies, “there’s a lot of consistency in these five studies.”
The main basis for CDER’s recommendation to withdraw Makena is that PROLONG, a large, well-conducted study, did not show any evidence of benefit on gestational age or neonatal outcomes, Stein said.
“But we certainly looked at the fact that the real-world evidence studies gave the same message and other randomized trials in other populations of patients gave the same message,” Stein said. “And these therefore just provide supportive information to the main study that provided the determination that the drug was not shown to be effective.”
In his closing statement, Stein acknowledged that the RCTs in the non-indicated populations do not directly bear on Makena’s efficacy in the labeled population.
“But they can provide information about the pharmacologic action of the drug in related conditions of increased risk of preterm birth,” Stein said. “The absence of response outside the indicated population is not alone strong evidence that Makena is not effective in the indicated population. But certainly with multiple trials, seeing some suggestion of an effect would have been reassuring. Yet none was seen.”
Covis Takes Issue With ‘Outlier’ Argument
In Covis’ affirmative presentation, Michael Greene, professor emeritus of obstetrics, gynecology and reproductive biology at Harvard Medical School, took issue with CDER’s attempt to paint the positive Meis trial as an outlier based upon RCTs in non-indicated populations and observational studies.
Makena is indicated for use only in women with a singleton pregnancy who have a history of singleton spontaneous preterm birth. Studies that evaluate different potential risk factors for spontaneous preterm birth, such as HIV infection, and studies in patients with multiple gestations are not helpful in evaluating whether Makena is effective for its intended use, Greene said.
“CDER has taken the position that there are ‘inherent limitations’ to observational studies or externally controlled trials, whether retrospective or prospective. I agree with CDER’s position and do not think observational studies have much to contribute to the discussion of Makena’s efficacy.” – Harvard’s Michael Greene
Furthermore, “CDER has taken the position that there are ‘inherent limitations’ to observational studies or externally controlled trials, whether retrospective or prospective. I agree with CDER’s position and do not think observational studies have much to contribute to the discussion of Makena’s efficacy.”
“Applying these principles, we can see that CDER’s effort to paint Meis as an ‘outlier’ is undermined by a careful examination of the individual studies,” Greene said.
Since the observational studies are not sufficiently persuasive by design, and the RCTs in the non-indicated populations are not applicable to Makena’s labeled use, all that is left is Meis and PROLONG, Greene said.
“This is important because only randomized, controlled trials conducted in Makena’s target population are truly relevant to our consideration of Makena’s efficacy here today. And because there are only two such trials, it is not appropriate for CDER to characterize the Meis trial as an outlier.”
Debating The Impact Of EPPPIC
Covis’ Chari asserted the EPPPIC meta-analysis supported Makena’s efficacy.
The Evaluating Progestogens for Preventing Preterm birth International Collaborative (EPPPIC) meta-analysis of individual participant data from randomized, controlled trials was funded by the Patient-Centered Outcomes Research Institute. The study was published in the Lancet in March 2021, with the authors reporting that vaginal progesterone and intramuscular 17-OHPC (the administration format for Makena) both reduced birth before 34 weeks’ gestation in high-risk singleton pregnancies, with the absolute risk reduction greater for women with a short cervix.
“EPPPIC is the largest existing individual patient data meta-analysis of progestogens used to prevent preterm birth. These include vaginal progesterone, intramuscular Makena, and oral progesterone,” Chari said.
“The meta-analysis includes participant-level data from 31 trials, more than 11,000 women and 16,000 offspring and five randomized trials for intramuscular Makena. EPPPIC is the first individual patient data meta-analysis of Makena in singleton gestation pregnancies,” he said. “The meta-analysis found that 17-OHPC reduced the relative risk of early preterm birth in high-risk singleton pregnancies before 34 weeks, with a relative risk of 0.83. Favorable reductions are also seen before 28 weeks and 37 weeks, and they indicated potential reductions in serious neonatal complications and incidence of low birth weight endpoints.”
CDER disagreed with Covis’ assertion that EPPPIC supported Makena’s efficacy.
Among EPPPIC’s 31 clinical trials, CDER focused on the five placebo-controlled clinical trials, including Trials 002 and 003, that were relevant to Makena because they evaluated the effect of HPC on preterm birth in singleton pregnancy. These data did not show a beneficial treatment effect on preterm birth < 28 weeks, < 34 weeks, or < 37 weeks gestation in women with a prior sPTB or a short cervix in the current pregnancy, the drugs center said.
Chari also said there is evidence for Makena’s efficacy in the Bastek observational study, which was cited by CDER as not showing benefit.
“The authors concluded that 17-P was associated with a meaningful delay in preterm birth,” Chari said. While the overall rate of preterm birth ≤ 37 weeks did not differ between the two time periods, the authors observed a shift toward later preterm births among patients treated with 17-P.
“These findings are clinically relevant because outcomes in babies born late preterm are generally improved compared to those [at] early gestational ages,” Chari said. “As the authors explained, the data provide evidence that 17-OHPC may have brought us closer towards mitigating the adversity associated with prematurity, which is of great public health significance.”
However, CDER’s Johnson cited significant limitations with this finding of increase in gestational delivery age during the Makena period. The study was unable to capture actual exposure to Makena and other important patient-level data, and there was no accounting for changes over time that could have explained the results, Johnson said, concluding that the study does not support Makena’s effectiveness.
Utility Of New Observational Study In Dispute
CDER and Covis again found themselves with opposite viewpoints – but more akin to the usual dynamic of regulator and sponsor perspectives on RWE – when it came to the prospect of conducting a new observational study of Makena.
Covis has proposed that Makena stay on the market, with a narrower label, while it conducts a new randomized, placebo-controlled trial in a high-risk population, a process it estimates would take four to six years. The company envisions using a continuous endpoint, such as increase in delivery time from randomization focused on <35 weeks. (Also see "Makena Randomized, Controlled Trial Proposal Shows Evolution In Covis’ Thinking" - Pink Sheet, 14 Oct, 2022.)
The company also proposed to conduct an observational study intended to further validate the benefit of prolonging gestational age on neonatal morbidity and mortality with Makena. “The goal of the study would be to establish the relationship between gestational age and neonatal outcomes in treated-versus-untreated patients to validate the benefit of weeks gained on 17-P,” Chari said.
“This approach is based on a review of the available literature on the association of neonatal morbidity and mortality with gestational age. Overall, there is a consistent picture of significant benefits that you see in prolongation of gestation from week 30 to week 36. In our minds, one residual question that remains is does treatment with Makena change that pattern in some fashion?”
Christine Nguyen, deputy director of CDER’s Office of Rare Diseases, Pediatrics, Urologic and Reproductive Medicine, said that although such data may provide some supportive information, an observational study is unlikely to be able to provide clarity on neonatal benefits.
“Given the failure of the randomized trials 002 and 003 to show a drug effect on neonatal outcomes, observational studies would not provide clarity to the important clinical questions,” Nguyen said.