US FDA Looks To Standardize Premarketing Safety Assessments
Executive Summary
Ellis Unger tells the Pink Sheet that Premarket Safety Assessment Working Group is overseeing various projects aimed at bringing more standardization to NDA and BLA safety assessments across applications and review divisions, while also bolstering the quality of those reviews; initiatives include development of queries that group similar types of adverse events together and standardizing the safety information requested at pre-NDA meetings.
A US Food and Drug Administration initiative to standardize safety assessments of new drug and biologic applications could mean changes in how sponsors classify clinical trial adverse events and in the types of information companies routinely are asked to submit to the agency.
The Premarket Safety Assessment Working Group is overseeing various projects aimed at bringing more standardization to NDA and BLA safety assessments across applications and review divisions, while also bolstering the quality of those reviews.
Office of Drug Evaluation I Director Ellis Unger, who co-leads the working group with Office of New Drugs Director Peter Stein, talked about the initiative in a recent interview with the Pink Sheet.
Premarket Safety Assessment Working Group Projects
-
FDA queries project
-
Pre-NDA data request list
-
Type C meeting for data request
-
Safety signal tracker
-
Standard tables and figures
-
Data integrity assessment
The working group is composed mostly of staff from OND. The initiative started in January 2017 and is part of the Center for Drug Evaluation and Research’s broader effort to modernize the new drug review process, Unger said. (Also see "CDER Safety Overhaul Planned As Review Process Modernization Continues" - Pink Sheet, 8 Jul, 2018.)
Reducing Variation And Improving Review Quality
Given that there is a wide variation across divisions in review practices, the working group was convened to perform a detailed assessment of the review process, especially from a safety analytics perspective, to see if OND could develop efficient, standardized processes and analyses that can leverage new technologies.
Improving the quality of premarketing safety assessments is another goal.
Although clinical reviewers often are aided by statisticians and other review disciplines in their efficacy assessment of a new drug or biologic application, when it comes to conducting safety analyses, review division medical officers are “pretty much on their own,” Unger said.
“We depend on the medical officer to do a great job on safety analyses, and they do it by themselves. Statisticians up until recently weren’t involved in generating any safety analyses,” he said. “We recognized the need to try to improve our processes to make sure that we apply the best methods possible to every safety review.”
The working group is focused on six projects. (See box.)
Queries For Groups Of AEs
The "queries" project is aimed at ensuring that similar types of adverse events are considered together for purposes of safety analyses.
In their analyses of clinical trial adverse events, applicants will code or translate verbatim terms reported by investigators into standard MedDRA preferred terms. However, this MedDRA classification is highly granular, with more than 23,000 preferred terms. When related preferred terms are not grouped together, it is possible to miss important safety signals, FDA officials said.
For example, reports of suicidal ideation with a drug could fall below the 2% frequency threshold for purposes of inclusion in labeling, but when suicidal ideation is combined with events of completed suicide, suicide attempt and suicidal depression, the event rate surpasses the 2% threshold.
Unger also pointed to the example of abdominal pain, which may be coded as the MedDRA preferred terms abdominal pain, upper abdominal pain or lower abdominal pain.
“When I analyze a new drug, I just want to know how many patients had abdominal pain. I don’t care if it’s upper, lower, middle … I need to combine these,” Unger said. “The way that the industry has dealt with these since the beginning of time is to tabulate them separately. And so they send us pages and pages of mind-numbing tables where all these terms are separated, for better or for worse, really mostly for worse. And so if you want to know … how many patients had abdominal pain, you’ve got to figure it out for yourself.”
“We used natural language processing of 38,000 labels to see what are the most frequently labeled terms that we should design these queries on, and then we established ground rules and then apply medical judgment to develop logical groupings for the queries.” – OND’s Vaishali Popat
This approach of segmenting groups of adverse events into smaller categories is sometimes referred to as “divide and hide,” Unger said.
“When you divide the term, signals go away,” he said. “It’s not really so much that the company is making an effort to divide the signal. It’s that they’re not putting the signal together because it’s not their job to put it together.”
The working group project is aimed at developing FDA standard queries for detecting and summarizing safety signals from clinical trial adverse event datasets.
For example, if the agency constructs a query that encompasses every type of abdominal pain, then it is easier to figure out how many patients on drug or placebo experienced abdominal pain, Unger said.
The agency wants to develop queries that are consistent, standard, and can be applied across all therapeutic areas. Vaishali Popat, associate director for bioinformatics in OND, told a recent Duke-Margolis Center for Health Policy meeting that by standardizing groupings of related preferred terms, reviewers will be better able to detect safety signals, and labeling can be standardized.
The project is being undertaken in three phases, Popat said. The first phase involves looking at the most frequently labeled terms and combining similar preferred terms in a single medical concept.
In addition to evaluating prior efforts in this area, “we used natural language processing of 38,000 labels to see what are the most frequently labeled terms that we should design these queries on, and then we established ground rules and then apply medical judgment to develop logical groupings for the queries,” she said.
The project’s second phase involves asking all OND review divisions if there is a particular adverse event they frequently encounter in their day-to-day work for which a query would be helpful. The final phase will involve algorithmic queries to detect syndromes and complex conditions, such as hypersensitivity and opportunistic infections, Popat said.
Between 50 and 70 standard queries have been generated so far under this initiative, with many more to come, Unger said. “The idea is we would make these publicly available so companies would know when you’re talking about abdominal pain you’re talking about these 12 … preferred terms. They all go in there, and then they can run the same query that we run.”
“Sponsors are going to have to pay more attention to how they translate the verbatim terms to preferred terms. The companies will be keenly interested in these groupings because they don’t want to be blindsided by the FDA.” – ODE 1 Director Ellis Unger
When these queries become publicly available, “sponsors are going to have to pay more attention to how they translate the verbatim terms to preferred terms,” Unger said. “The companies will be keenly interested in these groupings because they don’t want to be blindsided by the FDA. They don’t like it when we say, ‘Look, we combined heart failure and pulmonary edema, because pulmonary edema is heart failure after all, and we found this.’ They don’t want to hear that. They would rather do the analysis themselves. So they’ll be keenly interested in these queries and obviously they’ll be more inclined to run them themselves.”
The agency is creating new positions for clinical data scientists who are experienced in working with databases and can actually run these queries on applications, Unger said.
The clinical data scientist program, which is still being established, is expected to improve the consistency of clinical data analyses by medical reviewers. (Also see "Data Standards: Technical ‘Postmortem’ Could Clarify US FDA Preferences, Industry Says " - Pink Sheet, 19 Jun, 2019.)
CDER Director Janet Woodcock has said that clinical data scientists will be added to assessment teams and conduct exploratory safety analyses, removing some burden from medical and statistical reviewers. (Also see "Vocabulary Change: CDER Eliminating 'Review' For NDAs And BLAs " - Pink Sheet, 15 Apr, 2019.)
Standardizing Pre-NDA Data Requests
Sponsors also are likely to feel an impact from the working group’s pre-NDA data request list project, which aims to standardize the information that is requested from sponsors at the time of a pre-NDA meeting.
Some review divisions have detailed lists of information requests that they send to companies before an NDA filing, such as in the form of preliminary comments for a pre-NDA/BLA meeting, Unger said, adding that some review divisions do not have this practice.
“What we’re trying to do is kind of look all around the Office of New Drugs and figure out which should be a standard and maybe what we … should be asking at all, and come up with a set of standard preliminary comments basically,” Unger said. “This could be altered for particular applications, but that’s the goal.”
Standardization of pre-NDA data requests could be a big change for sponsors.
“I think as that information gets shared, companies will be aware of it and they’ll be attuned to make sure they collect the data,” Unger said. “If we ask companies for newsletters they send out to investigators, for example, well not many divisions ask for that, but it turns out they can be very useful. So when companies find out about that, they’re going to pay more attention to it. That’ll make a rapid impact once it gets out there, I think.”
Type C Meetings On Safety Data
The Type C meeting initiative is aimed at gaining earlier alignment between the agency and sponsor on strategies for the integrated safety summary, related data requirements, and other principal safety analyses.
The FDA is encouraging sponsors to request Type C meetings a year ahead of application filing to gain alignment on a safety analytics strategy.
This meeting is encouraged because waiting to discuss a safety analytics strategy – such as study pooling and customized queries – until the pre-NDA meeting is often too late, the agency said. “This meeting will help identify, discuss and proactively address important topics for safety analytics early in the drug development process.”
This is an optional meeting, but if held should precede the pre-NDA meeting by approximately one year. Sponsors can request this meeting through the established industry meeting request process.
Safety Signal Tracker
Another working group initiative is the safety signal tracker, which is a new informatics tool that allows reviewers to make a list of safety issues during drug development in the IND phase and directly link the list to its data sources, such as an IND safety report, a table in a toxicology report, or a literature report.
Although most reviewers keep notes of their assessment of safety issues for each IND in a Word document, an Excel spreadsheet or even hand-written notes, if that reviewer leaves the agency these individual notes do not consistently get transferred to the new reviewer, the agency said.
“Having this tool supports easy access to all information about the issue, and the system can be viewed by team members,” the agency said. “This provides a consolidated site where all members can view (and assigned team members can enter) the emerging safety profile as needed.”
In addition, there is an ongoing effort to combine this premarket tool with postmarket tracking systems in one interface in CDER’s new workflow management tool, the agency said.
Optimizing Data Presentation
The project on standard tables and figures is aimed at developing uniform approaches to presenting data in review documents and labeling.
“Some of the tables and figures we’ve seen in our reviews are maybe not as good as they might be,” Unger said. Staff from OND, along with biostatisticians and graphic arts experts, are working to develop best practices for formatting tables and figures to be used in reviews and potentially labeling, he said.
Data Integrity Assessment
The data integrity assessment project is examining the ability to look at datasets and identify anomalies in clinical data from particular trial sites, such as if all the patients at a site were reported to have been seen by the investigator on a Sunday.
It’s a potentially more systematic approach to looking for fraudulent data than sending investigators to individual sites, Unger said. “It’s a more useful way of looking at data integrity than going and looking for signatures on consent forms.”
Lessons Learned
Unger said there is no timetable for the premarketing safety assessment working group to wrap up its various projects. “I think we recognize we want to get all of this implemented as quickly as we can, but it takes time.”
The veteran agency staffer said the project has helped him better understand how much variation there is across review divisions.
“I guess I knew that the practices within the Office of New Drugs were variable in terms of how safety reviews are done. They ended up being, I think, maybe even more variable than I expected,” Unger said.
“People basically have their ways of doing safety analyses, and people who’ve been here a long time are quite proud of the work they do and they’re set in their ways,” he said. “And I’m guilty of the same thing, I suppose. The point is there’s a lot of room for improvement, I think, in how we do safety reviews, and we’re going to get there.”