Hahn Defends Using Less ‘Robust’ Data During COVID, But Critics Contend It Has Gone Too Far
Executive Summary
US FDA Commissioner Hahn says agency's choice to make fast decisions on less robust data sets during COVID-19, paired with constantly reevaluating these decisions drives best public health outcomes. But critics argue the agency may be pushing the bar too low and needs to be more transparent about the evidence it is using for emergency use authorizations.
The US Food and Drug Administration Commissioner Stephen Hahn defended the agency’s decision making during the COVID-19 pandemic as being rooted in “good data and sound science,” while acknowledging that it has made risk-benefit decisions on medical products using “data sets that might not be as robust as they would under normal circumstances.”
In a speech given over teleconference to the Alliance For A Stronger FDA on Monday, Hahn also said that FDA is “constantly reevaluating” these types of fast decisions and adjusting them as new information become available. He argued that the balance of moving fast but also constantly course-correcting shouldn’t be a sign of agency failure.
“I think that’s an important lesson learned from the COVID-19 outbreak that we will always incorporate new data, new science, new information to revise and make the best absolute decision for the American people,” Hahn said.
“And I don’t think that’s something to be criticized. I think that’s something to embrace. Because we know that in a fast-moving pandemic ... one must make the best decisions possible with the data that you have, but one must also be willing to visit those decisions and change them as possible.”
Hahn previously spoke about the idea of revisiting FDA approvals as more data becomes available at FDA's annual Rare Disease Day event. (Also see "Could Revisiting US Approval Decisions Become Less Rare? " - Pink Sheet, 24 Feb, 2020.)
Hahn’s speech also laid out other ways the commissioner expects COVID-19 will change the agency. (See sidebar.) A transcript of his remarks can be viewed on FDA’s website but it differs from the version he delivers and omits the remarks about the use of less robust datasets.
Anecdotes Are Not Data Sets
The regulatory flexibility touted by Hahn was embraced to some degree by even some of the agency’s most notable critics, who agreed that the nature of the COVID-19 pandemic should allow for a different pace of decision making than under normal circumstances. However, they said they were concerned that FDA is misusing such flexibility, particularly when it came to the emergency use authorization (EUA) for hydroxychloroquine and chloroquine.
“I agree with Dr. Hahn that in emergency situations it makes sense to move quickly and change course quickly if the evidence is different from what was hoped for. But the only justification for making [hydroxychloroquine] immediately widely available was that President Trump had ‘a good feeling’ about it,” said Diana Zuckerman, president of the National Center for Health Research.
The EUA was granted based on laboratory data and anecdotal clinical reports. (Also see "FDA's Emergency Use Authorization for Malaria Drugs Could Hinder Trials of Other COVID-19 Treatments" - Pink Sheet, 30 Mar, 2020.)
Zuckerman described this not as datasets but as anecdotal evidence. “Scientists don’t call anecdotal evidence datasets. As any scientist can tell you, the plural of anecdote is ‘anecdotes’ not ‘data,’” Zuckerman said.
In an emergency situation “making decisions on the fly and using the best data that you have is inevitable,” but the agency should follow certain principles when it does this, said Aaron Kesselheim, the director of the Program on Regulation, Therapeutics and Law at Harvard Medical School and Brigham and Women’s Hospital.
These principles include “being very clear when you are making decisions on data that are suboptimal,” and recognizing “that there are some datasets out there, or some studies out there that are so bad that you wouldn’t want to make decisions on them in any circumstances."
Kesselheim's third principle was that the FDA should revisit or revise these decisions just as quickly as they made them when new information emerges. He said the FDA has done a good job in some circumstances of adhering to these principles and done less well in others.
As Hahn mentioned in his speech, the agency has reversed some of its early EUAs for COVID-19 tests. (Also see "FDA Yanks Potentially Faulty COVID-19 Antibody Tests – And More May Be On The Chopping Block" - Medtech Insight, 22 May, 2020.)
Why Hasn’t Hydroxychloroquine Decision Been Revisited?
For hydroxychloroquine and chloroquine, Kesselheim was concerned that FDA has not yet updated or revoked the hydroxychloroquine and chloroquine EUA as the evidence has mounted against the drug. (Also see "Coronavirus Update: Setback With Hydroxychloroquine, Slowdown In UK ADR Reports, But Novel Drugs Begin Trials" - Pink Sheet, 26 May, 2020.)
Following Hahn’s speech the Pink Sheet asked FDA whether it thinks that the data that has come out on the two anti-malarials in COVID-19 since the EUA warrant a revision of that decision.
“The agency supports well designed controlled trials to generate high quality data for medical product development. We continue to look at the data on hydroxychloroquine and chloroquine and we will make future determinations on these products based on ongoing clinical trials,” spokesperson Stephanie Caccomo said.
Kesselheim took issue with this focus on controlled trials now.
“The standard for them changing the decisions should not be different from the standard that they used to make the decision in the first place. … It looked like they made the decision in the first place based on almost no evidence and now that a whole bunch of evidence has been gathered in the last two months, they are like now we’ll see, let’s wait for the definitive randomized trial.”
Michael Carome, the Director of Public Citizen’s Health Research Group came to a similar conclusion as Kesselheim.
“Given the standard on which it was originally approved, which was incredibly flimsy evidence, they’re setting too high a bar to revoke the EUA, I think.” Carome said this harkens back to his concerns that the initial decision was made to please President Donald Trump.
“In a public health emergency … we understand that there’s a need to more quickly bring potentially beneficial medical products to patients,” Carome said, but the potential consequences of relying on this less robust data, he added, is that we may find out that products don’t work and that they’re harmful.
This “danger” of using less robust data is particular high in the current political climate where this is “an extraordinary amount of pressure coming from the White House to basically ignore science, ignore the data and just go ahead and do things that President Trump on a whim wants to do," Carome said.
‘How Much Weaker Is Still Strong Enough?’
FDA could counter concerns it is making politically-based decisions by more clearly communicating the standard it is using to determine whether a emergency use authorization is warranted, said Steve Joffe, a pediatric oncologist and bioethicist at University of Pennsylvania.
“Dr. Hahn is correct; Congress intended to allow the FDA to authorize drugs and other medical products based on weaker evidence during a state of emergency. But the devil is in the details – just how much weaker is still strong enough?” Joffe said. “The FDA has had to face that question during the current pandemic and if it has achieved internal clarity, that’s not evident to those of us on the outside.”