‘Biased Trials Undermine EU Cancer Drug Approvals’
Executive Summary
Nearly half of the pivotal randomized controlled trials supporting European marketing authorizations for new oncology drugs are at high risk of bias, claims a new study.
A new study has called into question the credibility of randomized controlled trials that support European oncology drug approvals, claiming that almost half the trials were judged to be at high risk of bias.
The study also found that journal publications did not acknowledge the key trial limitations identified in regulatory documents, and said that policymakers, investigators and clinicians “should carefully consider risks of bias in pivotal trials that support regulatory decisions.”
Although the researchers are calling for regulatory change, the European Medicines Agency said that the findings were consistent with its own and that regulators could recommend the approval of medicines even where there were weaknesses in clinical trial design.
Findings
The study, published in the British Medical Journal on 18 September, set out to examine the design characteristics, risk of bias and reporting adequacy of pivotal randomized controlled trials (RCTs) of oncology drugs recommended by the European Medicines Agency for EU approval.
Between 2014 and 2016, 32 new cancer drugs supported by 54 pivotal studies were approved in the EU. Of those studies, 41 (76%) were randomized controlled trials. The researchers were able to identify published accounts of 39 of the trials and therefore included those RCTs in the study.
Overall, 19 (49%) of the 39 trials were found to be at a high risk of bias for the primary outcome, while 18 (46%) were thought to be at low risk of bias. There were concerns over the remaining two trials. “Despite the widely accepted strengths of [RCTs], we concluded that almost half of randomised controlled trials were at high risk of bias because of deficits in their design, analysis, and conduct,” according to the authors of the study.
Only 10 of the 39 RCTs used overall survival as a primary or co-primary endpoint, while progression-free survival was the primary end point in 21 of them. The remaining trials evaluated disease response, event-free survival or endpoints.
According to the study, fewer trials powered to evaluate overall survival were at high risk of bias than those geared to measure surrogate endpoints. Two out of 10 trials (20%) focusing on overall survival were at high risk of bias compared with 16 out of the 29 trials (55%) looking at surrogate endpoints.
Of the 13 drugs approved in orphan conditions, four (31%) had at least one RCT at low risk of bias.
Regulators identified additional problems concerning 10 drugs (31%). These issues, which were not disclosed as limitations in scientific literature, included magnitude of clinical benefit, inappropriate comparators, and non-preferred study endpoints.
Oncology drugs are the single biggest category of medicines approved in Europe. In 2019, more than a quarter of EU approvals, or 24 out of 92, were for cancer medicines, according to the study. Close scrutiny of evidence supporting these drugs as they enter the market is important because cancer medicines are responsible for recent rises in spending on medicines across different healthcare systems, it says.
The study’s lead author, Dr Huseyin Naci, assistant professor of health policy at the London School of Economics and Political Science, said that the legitimacy of the evidence supporting new oncology medicines was crucial to decision making in clinical practice. “We recommend devising and testing novel strategies for collating and communicating information about the validity of clinical studies in the future,” he said.
There are already deep concerns surrounding the development of cancer drugs, said the senior author of the study, Christopher Booth, professor of oncology at Queen’s University, Kingston. “This study adds another layer of complexity for patients and clinicians by identifying bias which may influence the analysis and interpretation of RCTs,” he declared.
The study also brings to light “shortcomings within the regulatory system,” according to Jaume Vidal, senior policy advisor at the NGO Health Action International, which helped fund the study. Vidal called on the EMA to act on the findings.
The study “adds to previous evidence showing many newly marketed medicines bring negligible or non-existent improvements to survival rates and quality of life for patients, while becoming ever-more unaffordable to already stretched health systems,” Vidal said. “Regulators must take on the findings to help ensure new medicines on the market are there for the benefit of the patient and society and not pharmaceutical companies and shareholders.”
A BMJ editorial on the study said it“confirms and extends the existing body of research that raises serious concerns about low standards of evidence supporting new cancer drug approvals.” And because nearly half the trials were considered to be at high risk of bias, the study also “indicates that treatment effects might have been exaggerated,” it said.
EMA Response
Despite the concerns raised, the EMA said that the study’s findings were consistent with its own. “Clinical trials included in marketing authorisation applications (MAA) often have methodological flaws in terms of their design, conduct, analysis or presentation of data,” it told the Pink Sheet.
“Ultimately, if the totality of the evidence shows convincingly that a medicine’s benefits outweigh its risks, despite the possible weaknesses in clinical trials design, regulators can take decisions to bring new medicines to patients in a timely fashion,” it observed.
It said that every marketing authorization application was subject to a robust scientific assessment of all available evidence before a decision was taken whether or not to approve a medicine. An “integral part” of that scientific assessment, it said, was to scrutinize trials to find biases, uncertainties and limitations in the data. The agency is then able to detail areas of uncertainty and consider what can be done to address them.
“Where the uncertainties and biases can be addressed by statistical or regulatory tools such as additional data or analyses, extrapolation, restriction of the indication, or other risk minimization activities, the medicine can be recommended for authorization. But if these measures are not successful in managing the limitations identified during the assessment, then authorization is not granted,” the EMA pointed out.
It added that any uncertainties were conveyed in its communications about scientific assessments, including any measures taken to address risks. Examples of such communications are assessment reports and the product information published for each medicine evaluated by the EMA.
The agency said that transparency was “paramount for EMA in delivering its service to patients and society.” It pointed to a number of initiatives aimed at allowing greater public scrutiny of its assessments, including its policy on the proactive publication of the clinical data that underpins medicines authorized for human use.
It also “strongly encourages companies to seek the Agency’s advice on the conduct of trials, an advice that can be enriched with the views of patients and HTAs.”