COVID-19 Vaccines: Advisory Committee Picks Apart US FDA Guidance On Efficacy Endpoints
Executive Summary
FDA's recommendation that COVID-19 vaccine sponsors conduct two months of median follow-up to support an emergency use authorization may not be long enough, some experts say in what can be viewed as validation for agency's position in battle with White House.
The US Food and Drug Administration's first advisory committee meeting on COVID-19 vaccines revealed that external experts are somewhat dissatisfied with various aspects of the agency’s development advice to vaccine sponsors. However, whether their objections will lead to any substantive change in the agency's path going forward seems unlikely.
At the 22 October meeting of the Vaccines and Related Biological Products Advisory Committee, panel members found plenty to dislike about the agency’s recommendations in the two COVID-19 vaccine guidances issued to date. In particular, they raised concerns about recommended endpoints, the threshold level of efficacy set by the agency, and the amount of safety follow-up recommended to support an emergency use authorization (EUA) request.
However, the panel’s advice may well be viewed as too little, too late. In the US, Phase III trials already are underway for four vaccine candidates, two of which – mRNA vaccines from Pfizer Inc./BioNTech and Moderna, Inc. – are nearing completion and may have enough data in hand to support an EUA request as early as November.
That the committee members concluded the recommendation for two months’ median safety follow-up was not unduly long and, in fact, may not go far enough, had to be viewed as vindication for the Center for Biologics Evaluation and Research, which battled with the White House over the issue.
It seems unlikely the panel’s advice will result in any sort of major overhaul of the FDA’s approach on COVID-19 vaccine development to date. Rather, the major takeaway is likely to be the transparency FDA provided by allowing a group of external experts to opine on the agency’s recommendations in a meeting that was broadcast on YouTube to an American public that has become increasingly skeptical and hesitant about the fast pace - and what is widely viewed as the attempted politicization - of the vaccine development process.
The transparency was particularly important as panelists criticized the agency’s recommendation that sponsors conduct a median two months of follow-up after study participants get their final dose of a COVID-19 vaccine before requesting an EUA, a recommendation that had become a political football. That the committee members concluded this request was not unduly long and, in fact, may not go far enough, had to be viewed as vindication for the FDA and its Center for Biologics Evaluation and Research, which have battled with the White House over the issue.
At the end of the meeting, Marion Gruber, director of the Office of Vaccines Research and Review, summed up what she heard the committee say about the agency’s vaccine development recommendations.
“In terms of the guidance documents and the approaches for safety and effectiveness data as we’ve outlined them, I’ve heard that the general principles and the standards we are applying are right on the money and that there is really buy-in for that,” Gruber said.
“I hear there were some concerns and suggestions made for some of the details, the importance of making sure that minorities are included in clinical studies. We had some discussion of endpoints. We can take this forward as we have new vaccines entering clinical studies. It may be a little bit difficult for those who are already in Phase III.”
As for the recommended minimum duration of safety follow-up, some panelists said it may not be long enough, “but if it then cannot be longer, by no means should it be shorter than two months of median follow-up,” Gruber said. “That’s what I heard.”
Efficacy Target And Endpoint
The meeting marked the first time the agency convened external experts to weigh issues around COVID-19 vaccine development, but it won’t be the last. FDA officials have pledged to bring every application for an EUA or a BLA before the panel for vetting of issues specific to that vaccine.
Among those issues will be how a given sponsor can ensure that ongoing, placebo-controlled clinical trials will be able to continue after an EUA is granted. (Also see "US COVID-19 Advisors Vexed By Vaccine Post-EUA Placebo Controls, But Agree On Need" - Pink Sheet, 22 Oct, 2020.)
At the inaugural meeting, the agency presented general issues for discussion. It sought comment on its approach to safety and effectiveness as outlined in two documents: a June guidance on development and licensure of COVID-19 vaccines, and an October guidance on emergency use authorization for vaccines. (Also see "After EUA, COVID Vaccine Sponsors Need Plans For Continuing Trials" - Pink Sheet, 6 Oct, 2020.)
In the June document, the agency made the unusual move of setting a specific efficacy threshold – a primary efficacy endpoint point estimate of at least 50%, with the lower bound of the confidence interval >30%. (Also see "COVID-19 Vaccine Should Demonstrate At Least 50% Effectiveness, US FDA Says" - Pink Sheet, 30 Jun, 2020.)
As for endpoints, the FDA’s June guidance recommends that either the primary endpoint or a secondary endpoint be defined as virologically confirmed and symptomatic SARS-CoV-2 infection; severe COVID-19 should be evaluated as a primary endpoint or secondary endpoint.
“There are multiple examples of vaccines where the data do appear to show that the vaccines are most effective against more severe disease, less so against less severe disease, and even less so against asymptomatic infection. We took that experience into consideration as well.” – FDA’s Doran Fink
The four Phase III trials that have gotten underway in the US were designed with a higher vaccine efficacy target of 60%. Janssen Pharmaceuticals Inc. is the only one of the vaccine sponsors in Phase III using a moderate-to-severe disease primary endpoint; all the others are using confirmed and symptomatic COVID-19 in the primary analysis. (Also see "One Goal, Four Paths: Comparing COVID-19 Vaccine Pivotal Trial Protocols" - Pink Sheet, 29 Sep, 2020.)
Some panelists took issue with the 50% threshold and the recommended primary efficacy endpoints.
Consumer representative Sheldon Toubman, a staff attorney at New Haven Legal Assistance Association, raised concerns that the guidance did not require serious disease as the primary endpoint. He suggested a vaccine could meet the 50% threshold in avoiding mild cases of COVID-19, but said that does little to address the more serious cases, which is “what we really care about.”
Doran Fink, deputy director of the Division of Vaccines and Related Products Applications, said that with the primary efficacy endpoint recommendation in the guidance, the agency was “trying to strike a balance between getting information on the most clinically significant outcomes of COVID-19 [and] how a vaccine might be able to prevent those outcomes versus being able to make an impact on the pandemic in as reasonable amount of time as possible based on good data.”
Vaccine manufacturers are free to choose what they consider to be the most relevant primary endpoints for their clinical trials, Fink said. “We felt that we could not mandate a specific primary endpoint, including a primary endpoint that focused on severe disease.” However, the agency expects to have some data on severe disease when it makes its risk/benefit determination, he said.
In addition, “there are multiple examples of vaccines where the data do appear to show that the vaccines are most effective against more severe disease, less so against less severe disease, and even less so against asymptomatic infection,” Fink said. “We took that experience into consideration as well.”
Panelist Luigi Notarangelo, chief of the laboratory of clinical immunology and microbiology at the National Institute of Allergy and Infectious Diseases, said the efficacy measures included in the guidance are biased toward mild disease, which “may not mean much,” and most are subjective.
The 50% efficacy threshold “shouldn’t be the minimum for everything.” – NCATS’ Michael Kurilla
The 50% efficacy threshold “shouldn’t be the minimum for everything,” said committee member Michael Kurilla, director of the division of clinical innovation at the National Center for Advancing Translation Sciences.
“I have some concerns about the utility of a 50% reduction in symptomatic disease when we don’t really have any evidence that these vaccines are going to induce sterilizing immunity,” Kurilla said. The idea that with the vaccine high-risk individuals could reduce their risk of mild disease to asymptomatic disease, but still be infectious, “doesn’t seem like it’s something worthy of an EUA.”
“Now on the other hand a 50% reduction in the progression in high-risk groups to serious disease, that is actually …quite significant. That is something that, to me, would be EUA-able.”
Trial Demographics Also A Concern
Fink’s comments that the FDA cannot mandate demographic representation in COVID-19 clinical trials also received pushback.
Several panelists said that if there is not sufficient evidence of efficacy and safety in minorities and other populations disproportionately affected by COVID-19, the trust of these subpopulations in the vaccine development and FDA regulatory process will be further eroded.
“Since severe disease and death occur primarily among minorities with this virus, if we put a vaccine out there that does not address that issue, it’s just going to perpetuate the perception that that population or that segment of our population does not matter much,” said James Hildreth, president and CEO of Meharry Medical College. “I would just ask that consideration be given that whatever we do, we have a vaccine that does address severe disease.”
“You said you cannot mandate what the drug companies might set as their primary endpoints,” Hildreth said. “If I’m not mistaken, the taxpayers of the United States of America are paying the tab for this, so maybe you might have more authority to mandate than you might think.”
Two Months’ Follow-Up Not Enough For Some
Panelists also said the agency's recommendation for a median of two months of safety follow-up to support an EUA might not be enough to adequately identify vaccine-related adverse events or a waning of immunity.
The two-month recommendation raised objections from the White House and President Trump because it was viewed as making a vaccine EUA unlikely before the 3 November election. The EUA guidance’s release was delayed as a result, with the document issued only after the agency included a summary of its recommendations on data needed for an EUA as part of the advisory committee briefing document. (Also see "Transparency, And A Mirror: US FDA Advice On COVID-19 Vaccine EUAs Finally Published – Twice" - Pink Sheet, 6 Oct, 2020.)
What President Trump viewed as unnecessarily long in terms of duration, the FDA’s external experts viewed as perhaps too arbitrary and not long enough.
“As you have waning vaccines, you might have more chances to have … vaccine-related problems that wouldn’t be seen at the two-month mark.” – University of Iowa’s Stanley Perlman
Toubman recommended against use of an EUA for a vaccine but said if the emergency route were used, it should come only after three or four months of follow-up. He noted the World Health Organization has recommended three months of follow-up for emergency use.
“We might want to prolong the two months to a few more months for a few reasons,” said panelist Stanley Perlman, a professor in the departments of microbiology, immunology and pediatrics at University of Iowa. From what is known about common coronaviruses and immune responses, there is probably a good immune response at two months, but it wanes between six and 12 months, Perlman said.
“As you have waning vaccines, you might have more chances to have … vaccine-related problems that wouldn’t be seen at the two-month mark,” he said. “In a way, two months would pick up a lot of the early adverse events, but I think it’s a continuum.”
Amanda Cohn, chief medical officer for the Centers for Disease Control and Prevention’s National Center for Immunizations and Respiratory Diseases, said she was more concerned about what the two months of follow-up would reveal about vaccine efficacy as opposed to safety.
“I do worry a little bit that the VE estimate for mild disease may be overestimated when we’re just looking at the first two months after vaccination, and that we may have a lower VE estimate, for example, if we looked at the data after four or six months just because of waning immunity,” Cohn said. “Very rarely do we look at VE so shortly after completing a series.”