Tolerating False Positives: Amylyx, FDA, And The Legal Case For Broad Regulatory Flexibility
Executive Summary
Amidst bipartisan congressional pressure to approve ALS drugs, Billy Dunn, head of FDA’s neuroscience division, told advisory committee members that the law ‘demands’ flexibility in review of Amylyx’s AMX0035, and he came prepared with the citations.
If Amylyx Pharmaceuticals, Inc.’s amyotrophic lateral sclerosis drug AMX0035 secures US Food and Drug Administration approval, it looks to be in no small part thanks to Billy Dunn, director of the Office of Neuroscience in the Office of New Drugs and his interpretation of the agency’s legal duty to exercise the broadest possible regulatory flexibility.
A key part of that regulatory flexibility in this case, per Dunn, is being more comfortable approving a drug that later turns out to be ineffective than not approving a drug that later turned out to be effective. In statistical speak, he believes he is obligated to be more comfortable making a Type I error than a Type II error.
“Our underlying legal authority is clear in not only allowing, but also endorsing and encouraging the application of regulatory flexibility in the setting of serious and life-threatening diseases. It is unquestionably relevant to ALS, drug development in general, and to our specific consideration of the data before us,” Dunn said in unusually long introductory remarks at the 7 September meeting of the Peripheral and Central Nervous System Drugs Advisory Committee – an unusual meeting it its own right since it was the second advisory committee review during a single application cycle.
In remarks that lasted about 35 minutes, Dunn went on to outline a potential case for approval regardless of the acknowledged flaws in Amylyx’s data package and lingering questions about the drug’s efficacy, though he stressed multiple times that FDA has not yet made a final decision. (Also see "Amylyx’s ALS Drug Lacks ‘Confirmatory Evidence’ Needed To Support Approval On Single Study, FDA Says" - Pink Sheet, 5 Sep, 2022.)
If Dunn was hoping to influence the meeting’s outcome with his remarks, he clearly achieved his goal as multiple panel members credited his remarks with influencing their positive vote. (Also see "Second Time’s The Charm: Amylyx’s ALS Drug Wins US FDA Panel Nod" - Pink Sheet, 7 Sep, 2022.)
This vote sets the drug up for a much higher chance of FDA approval than the earlier negative committee vote in March. (Also see "Amylyx’s ALS Drug Should Wait For Phase III Results, US FDA Panel Says" - Pink Sheet, 30 Mar, 2022.)
[For a breakdown of who switched their vote, see the table at the end of the story.]
A Pattern For FDA?
AMX0035 would not be the first time FDA has taken on a controversial marketing application with the consequences of making a Type II error seemingly weighing more heavily on FDA’s mind than the consequences of making a Type I error.
“The cost of committing a Type II error,” was a concern of FDA’s in the approval of Biogen, Inc.’s Alzheimer’s drug Aduhelm (aducanumab-avwa), a decision Dunn heavily helped shepherd. (Also see "FDA On What Aduhelm AdCom Got Wrong: Statistics, Investigational Drugs, Patient Perspective" - Pink Sheet, 29 Jun, 2021.)
Janet Woodcock, then the director of the Center for Drug Evaluation and Research, also brought up the risk of committing a Type II error at the advisory committee for Sarepta Therapeutics, Inc.’s Exondys 51. (Also see "Woodcock's Consideration of Sarepta Financial Issues Raises Eyebrows" - Pink Sheet, 19 Sep, 2016.)
The agency’s focus on regulatory flexibility in the AMX0035 approval also comes amidst intense pressure from both Democratic and Republican members of Congress to back up its public statement on flexibility in considering ALS therapies with concrete action. (Also see "More Action, Less Plan? House Keeps Pressure On US FDA Over ALS Drugs" - Pink Sheet, 25 Jul, 2022.)
‘Foundational Construct’
In his advisory committee presentation, Dunn took great care to cite law and regulation in making his case that the agency’s mandate to apply broad regulatory flexibility could result in a situation where a drug is approved despite data gaps, perhaps in part to try and insulate the agency from critics who are likely to pounce on such an approval as lowering the regulatory bar for FDA clearance.
Regulatory flexibility is “a well-recognized concept” that is found throughout FDA’s regulatory framework, Dunn said.
His first citation was 21 CFR 314.105 Part C: “FDA will approve an NDA after it determines the drugs meets the statutory standards for safety and effectiveness. While the standards apply to all drugs, the many kinds of drugs that are subject to the statutory standards and the wide range of uses for those drugs demand flexibility in applying the standards. Thus, FDA is required to exercise its scientific judgment. To determine the kind and quantity of data and information an applicant is required to provide for a particular drug to meet the statutory standards,” Dunn quoted.
“It is apparent from this that regulatory flexibility is a foundational construct,” Dunn said.
‘The Broadest Flexibility’
Dunn then turned to 21 CFR 312.80, which discusses drugs intended to treat life-threatening and severely debilitating illnesses and as he quoted stressed that these drugs are entitled to “the broadest flexibility.”
“The Food and Drug Administration has determined that it is appropriate to exercise the broadest flexibility in applying the statutory standards while preserving appropriate guarantees for safety and effectiveness. These procedures reflect the recognition that physicians and patients are generally willing to accept greater risks or side effects from products that treat life-threatening and severely-debilitating illnesses than they would accept for products that treat less serious illnesses. These procedures also reflect the recognition that the benefits of the drug need to be evaluated in light of the severity of the disease being treated,” Dunn quoted.
“This makes it abundantly clear that for these serious diseases, like ALS, and so many other neurological conditions, the maximum degree of regulatory flexibility, ‘the broadest flexibility in applying the statutory standards’ is operational,” Dunn said.
Dunn also cited 21 CFR 312.84, which discusses the risk-benefit analysis, and a 2019 draft guidance on demonstrating substantial evidence of effectiveness for human drugs which says that while FDA must reach a conclusion that there is substantial evidence of effectiveness to approve a drug “the degree of certainty supporting” the conclusion may differ depending on circumstances like the severity of the disease and unmet need and in some cases a greater risk of a false positive conclusion – approving a drug that turns out to not be effective, “may be acceptable.”
Tolerating Uncertainties and Risk Of False Positive
The neuroscience head talked about the “easy” ways FDA can provide regulatory flexibility such as granting more meetings with sponsors to flesh out complicated topics and contrasted that with the harder application he suggested may apply in the case of Amylyx’s AMX0035 (sodium phenylbutyrate/taurursodiol): the “need to tolerate a greater degree of residual uncertainty when making a subjective decision on approval.”
“In appropriate circumstances, such as serious and life-threatening diseases and settings of substantial unmet need, regulatory flexibility applied to assessments of effectiveness means increased tolerance for concluding that a drug is effective when there is residual uncertainty that the drug may not actually be effective, which would be a conclusion at risk of being a false positive and decreased tolerance for concluding that a drug is ineffective when there is residual uncertainty that the drug may actually be effective, which would be a conclusion at risk of being a false negative,” Dunn said.
He argued this type of flexibility has played a part in the approval of the two other available ALS treatments.
An Unusual Company Pledge
Dunn also went to great lengths to assure the committee that FDA has the authority under 21 CFR 314.50 to withdraw the drug if new data subsequently demonstrates that FDA made such a Type I error. This regulation, he notes recognizes that “a finding of substantial evidence of effectiveness sufficient to support approval is not a static fixed conclusion.”
Furthermore, the statutory standard of substantial evidence is a qualitative one, not a quantitative one that is based on whether a trial meets statistical significance or not, he said.
This is important for the Amylyx drug because an ongoing Phase III trial PHOENIX is expected to better answer the lingering questions around effectiveness and Dunn seemed to suggest that while this trial could change a substantial evidence determination, FDA doesn’t need to wait on its results to make a judgement call. It can make a determination of substantial evidence at this time with more limited data and then go back and revise later if necessary.
He encouraged Amylyx’s CEO to state that if that late-stage study “does not succeed” the company would voluntarily withdraw the product from market. And the company quickly obliged after Dunn finished speaking – a highly notable interaction because of the unique nature of the request at an advisory panel and because a voluntary withdrawal is swift, while a contested push to pull a product could take years and tremendous labor on FDA’s part, a fact that Dunn did not hide from the committee when asked.
He emphasized it would not be simple, easy or quick if the company contested FDA’s decision to pull the drug.
Later in the meeting, Teresa Buracchio, director of the division of neurology I in the FDA’s Office of New Drugs, reemphasized many of the points Dunn made in his opening response in her presentation on the regulatory considerations for the drug’s application.
Advisory Committee Members Internalized The Message
Dunn and Buracchio’s discussion, along with the Amylyx CEO’s commitment to voluntarily withdraw the drug later on if the ongoing Phase III study is negative, was mentioned in the voting justifications of many advisory committee members who favored approval.
Dean Follmann, assistant director for Biostatistics at the National Institute of Allergy and Infectious Diseases at the National Institutes of Health said the agency’s “thoughtful and thorough discussion about the evidentiary basis for approval” helped him reach his yes vote.
“The drug is not harmful. It seems like it has a benefit. There’s not a safety signal here. So in the worst case scenario, if the PHOENIX trial does show lack of benefit it’s not like we harmed patients by licensing it now,” Follmann argued.
Bryan Traynor, a senior investigator at the NIH, said he was “struck” by the CEO’s commitment to withdraw the drug if the Phase III study is negative and by FDA’s “established method” to withdraw if the company doesn’t cooperate.
“So on balance, I think that the danger of delaying treatment by three years kind of overcomes in a pretty robust way, the relatively real risk of a false positive result of the Phase III,” Traynor said.
“To deprive ALS patients of a drug that might work, it’s probably not something I’ll feel terribly comfortable with my conscience,” said Liana G. Apostolova, of Indiana University School of Medicine.
‘No’ Votes See Other Routes To Data, Patient Access
Dunn’s push may have backfired with the no voters, however.
G. Caleb Alexander of Johns Hopkins Bloomberg School of Public Health’s was frustrated by FDA’s reliance on its ability to later withdraw the drug as a justification to potentially approve now on weaker evidence.
“The FDA, with all due respect, significantly understates the complexity and likelihood of their pulling a product from the market,” Alexander said.
“Regardless, as we heard from the FDA, whether or not they can ultimately pull the product from the market, it’s no substitute for the evidentiary thresholds that are required for market access,” a standard Alexander said he didn’t think AMX0035 has met.
Another no vote, Kenneth Fischbeck of the NIH, said that if there is concern about denying ALS patients access to a potentially effective drug while the Phase III study completes there are other ways to provide patients access now besides allowing the company to “sell the drug.”
Fischbeck said the company could broaden their expanded access program or reopen sites in the Phase III trial in the US to enroll more US patients.