Reality Check For Real World Data: FDA’s Sentinel Network Enters “Production” Stage

FDA’s Sentinel drug safety surveillance network has completed an important transition, moving from “experimental” to “production stage” as a tool to help inform post-market safety decisions. Now the question is whether real-world data can be used in making efficacy decisions.

There’s no formal way to quantify it, but 2015 annual Sentinel Initiative stakeholder meeting was probably the least newsworthy of the seven held since the launch of the “real world” data monitoring system was mandated by the 2007 FDA Amendments Act.

The 2015 update conference did not include any major new functionality announcements, significant new milestones in the scope or capability of the program, or even much in the way of previously undisclosed outputs from the claims data safety analyses done as part of the project. Instead, it conveyed the sense that Sentinel is now a routine and integrated part of the agency’s post-marketing monitoring programs – not inherently more or less exciting than any other safety tool.

Or, as Center for Drug Evaluation & Research Director Janet Woodcock emphasized in her opening remarks to the at the seventh annual Sentinel review held by the Brookings Institute on February 5, Sentinel is no longer under development: it is now a “true production system” for generating a range of standard post-marketing safety reports.

She underscored that message by formally announcing a shift in organizational oversight for Sentinel from the CDER’s Office of Medical Policy to the Office for Surveillance & Epidemiology (the drug safety side of CDER). (Also see "FDA Sentinel Transition Will Involve Expanding Rx Safety Data Sources" - Pink Sheet, 23 Feb, 2015.)

On one level, the shift is a non-event: OSE staff and management have already been closely involved in the project and preparing for the transition for almost a year – especially after the departure of long time OMP Director Rachel Sherman. In particular, OSE’s Marsha Reichman has been closely involved with Mini-Sentinel as the senior adviser and scientific lead on surveillance programs. Woodcock predicts a “seamless” one year transition for Sentinel from OMP to OSE.

However, the positioning of Sentinel firmly within CDER’s drug safety operations means that—for now at least—it is “just” a safety tool, and that the agency is not going to devote resources to explore ways for the evidence in the database to meld into other regulatory functions.

The potential to use the data sources and search capabilities of Sentinel for other regulatory or clinical information purposes has been an issue almost from the start of the program. And the idea is certain to be revisited in the context of proposals in the “21^st Century Cures” legislative process that focus on ways to streamline approval of new indications and/or reduce post-marketing study burdens for manufacturers.

But FDA is not giving up on the idea that the Sentinel network can serve as a “national resource,” to support many real-world evidence efforts beyond safety monitoring. Instead, FDA seems to be making two things clear: (1) there are significant methodological issues that need to be worked out to enable those uses; and (2) someone other than FDA needs to fund the effort.

Commissioner Hamburg Highlights Opportunity In Senate Testimony

A month after the Sentinel conference, FDA Commissioner Margaret Hamburg testified before the Senate Health Committee in its kick off hearing for the parallel process to the House “21^st Century Cures” Initiative.

The importance of her testimony as an agenda setting effort was heightened by the fact that it would also be Hamburg’s last appearance before the committee before stepping down after nearly six years as Commissioner.

Hamburg’s verbal testimony departing significantly from her written statement, with her primary focus on countering the “mis-diagnosis” that FDA is in some way an impediment to innovation. She urged Congress to look elsewhere, and avoid what she sees as the mistaken idea that lowering the drug approval standard would help innovators. (Also see "FDA’s Top Two Reform Priorities Aren’t Really Legislative Material" - Pink Sheet, 16 Mar, 2015.)

But she also offered four ideas (selected from a longer list in her written testimony) for Congress to consider. The first two build on ideas from the 2012 user fee reauthorization that have been broadly popular: enhancing patient input in the drug development and regulatory process, and improving the biomarker/surrogate endpoint qualification process. The fourth is a perennial: enhancing FDA’s ability to attract and retain qualified people.

Sandwiched in between? A call for help to advance the use of “real world” data for purposes beyond drug safety.

She began with a short recap of the success of Sentinel: “Real world data provides a vital tool to monitor medical products in use in the marketplace,” she said. “FDA’s Sentinel initiatives with more than 170 million lives is one of the largest uses of big data in health care and proving vital for monitoring safety and emerging safety concerns.”

Moving beyond that, however, will take more work. “The science of using big data to establish product effectiveness is still in its infancy,” Hamburg declared. “Real progress demands that we develop methodologies needed to harness the promise of real world data.”

Hamburg’s written statement is slightly more expansive on that point:

“Although there is reason to believe that in the appropriate setting these data may be helpful in providing information on the effectiveness of marketed products, such as for new uses of approved products to support label expansion, many experts in the field agree that more work is needed to make these data operational for and directly applicable to regulatory purposes. We should move quickly to further develop methodologies needed to better understand and harness the promise of real-world observational data for regulatory purposes.”

Not FDA’s Job, For Now

In the meantime, however, Woodcock’s decision to nest Sentinel back in OSE and her statements at the Feb. 5 Brookings meeting indicate that she does not intend to allow Sentinel (as an FDA-run project) to spill out to other uses while those ideas are debated on Capitol Hill. Woodcock said plainly “we have built our piece of Sentinel as a safety system.”

FDA’s focus on safety uses “does not mean that other parties could not utilize the same data network for other research purposes or other types of purposes,” Woodcock said, adding that having outside partners using the data would “help us” by sharing the financial load.

She noted ongoing work with the Reagan-Udall Foundation’s IMEDS (Innovation in Medical Evidence Development & Surveillance) project on post-market evidence generation and the Patient-Centered Outcomes Research Institute as potential areas where Sentinel could play a role. As she has during past Sentinel events, Woodcock suggested that the FDA system could make a good base for some of PCORI’s studies. However, she repeated a line from last year’s conference: the potential partners are “not singing form the same book” but “are working on it.”

There is still a lingering sense when Woodcock talks about Sentinel that it is viewed as something of an unfunded mandate: the agency was required to build the system and does not want to be saddled with having to maintain it for a wide variety of uses – going beyond FDA’s pharmacovigilance efforts. Woodcock began the meeting with some gentle ribbing of Brookings Senior Fellow (and former FDA Commissioner) Mark McClellan for saddling FDA with the project. She said she especially wanted to acknowledge McClellan who “got this into the statute” (FDAAA, 2007).

Despite the stress and burden imposed on the agency to set up Sentinel (including funding $10-$15 million annually to support the project), Woodcock sees the system as now providing a cost-effective tool for pharmacovigilance projects.

Sentinel principal investigator, Richard Platt (Harvard), explained that before the development of the system a protocol-based study of a post-marketing drug use issue would cost “millions of dollars.” Through Sentinel, the cost for the same study is likely to be “a few hundred thousand.” Using the preset modular programs for searching the database network, the costs drop to the “tens of thousands.” Platt said he hopes to drive that cost down further.

Sentinel Is Now Delivering ‘Couple of Reports” Per Week To FDA

The scale-up and development of preset analytical tools has allowed Sentinel to begin delivering “a couple of reports a week to FDA on questions of interest,” Platt reported. He said that during Mini-Sentinel “we spent a lot of time and effort” developing reusable tools like the “modular programs” for routine querying of the data sources.

Platt highlighted the progress in reducing the time and cost for doing “quite sophisticated, adjusted, comparisons between products.” Noting that a protocol examination of angioedema associated with various ARB blood pressure therapies conducted on Mini-Sentinel two years ago took five months and cost $250,000, Platt showed how using new standard query forms recently developed can generate the same results in “weeks not months” and for “tens of thousands” of dollars.

At that level of cost and quick turnaround, Platt said, the searches are “useful” to FDA. The original ARB study itself was a substantial improvement at the time it was done two years ago. It showed that a protocol-based comparative study could be done for hundreds of thousands of dollars instead of millions. The fast progress from that point to the more recent modules demonstrates how Sentinel is bringing database safety searches into the standard tool category.

Two presentations from senior surveillance staff within CDER and the Center for Biologics Evaluation & Research (CBER) shed light on some of the internal dynamics in the centers as Sentinel is increasingly adopted into their standard procedures.

OSE’s Reichman reflected the institutional view emerging within CDER that the Sentinel efforts tend to reinforce CDER’s premarket review activities.

Reichman noted that there had only been two safety communications arising from Sentinel review. The first, for dabigatran (BI’s Pradaxa) and severe bleeding was in effect an early test of the Sentinel system. (Also see "Pradaxa Medicare Study: Observing a Slow Attitude Change at FDA on Observational Studies" - Pink Sheet, 29 May, 2014.)

The second, for olmesartan (Daichii Sankyo’s Benicar)and sprue like enteropathy, was a case where FDA modified its first look at the issue and used the Sentinel system ability to study specific periods of drug usage to focus on an effect of the drug after a two-and-a-half year treatment period. The celiac-like symptoms suggested with olmesartan were found to show up after a two-year latency period.

In many cases, Reichman said, the queries of the system confirm “what we expected.” The Mini-Sentinel searches have shown “that the preapproval processes are working as we hoped they do.” That view within FDA is making Sentinel less threatening to the new drug review teams than when the project was forced on the agency following the FDAAA drug safety act.

Reichman, in fact, focused on a new effort — stimulated by groups within the Office of New Drugs — to identify sudden cardiac death and see if some work currently required in the premarket could be looked at more post-approval. The project is also important to FDA as a way to see if it can incorporate death information from outside the hospital setting into Sentinel studies by bringing in data from the National Death Index Plus from the National Center for Health Statistics at the Centers for Disease Control & Prevention.

QT studies are often required as part of NDAs as a marker for sudden cardiac death risk, Reichman noted. She pointed out that they are costly and have not always predictive of risk of pro-arrhythmia. “The question came up: could we look at this issue reliably in the post-approval area?” She said protocols are being reviewed for the study and finalized.

Shadowing and Enhancing Phase IV Vaccine Commitments

CBER Director Division of Epidemiology Michael Nguyen demonstrated the enthusiasm in his center for shadowing and expanding sponsor post-approval activities with broader Sentinel projects.

Nguyen described how CBER is supplementing the recent approval of Merck’s 9-valent form of the Gardasil HPV vaccine with Sentinel based studies. The sponsor’s post-marketing pharmacovigilance plan was also broad-based as typical of many vaccine programs and included a 10,000-patient observational study.

Nguyen said that CBER believes that “we can do better with Sentinel enhancements” that operate to complement the sponsor’s own work. CBER is going to use a publicly-available statistical method that has been developed for use with Sentinel called TreeScan to detect serious and unexpected adverse events on a broader scale.

CBER will also run a PROMPT (Prospective Routine Observational Monitoring Tool) study for “near real-time active surveillance for prespecified outcomes,” an autoimmune surveillance study; and a larger pregnancy outcomes study. These seek to “enlarge the existing safety database and monitor safety in the real-world setting. While this is designed for the vaccine world which is more attuned to large safety trials post-approval, it does create  model for drugs (and therapeutic biologics) going forward.

Industry May Have New Ideas For PDUFA VI

One voice for expanded use of Sentinel evidence-gathering features at the Feb. 5 meeting was AstraZeneca PLC Senior VP Global Regulatory Affairs David Wheadon (formerly with PhRMA).  He said that he wanted challenge the sentiment that he heard at the meeting that “Sentinel is solely for safety.”

Wheadon said that the potential larger roles for Sentinel or Sentinel-like databases include helping to “further flesh out descriptive statistics on diseases or compounds with similar mechanisms of action and natural history of diseases. It could certainly help us to be more refined in how we decide to study innovative therapies in certain patient populations.”

For comparative studies, Wheadon suggested that “certainly this continues to be important: not just product vs. product, but mechanism of action against mechanism of action will be an important area for further research.”

He said that databases of Sentinel’s size can be very helpful in determining “feasibility of trial design.” Because trials for unmet need are getting more innovative, Wheadon said, “many times we are not sure about the type of trial, the shape of the trial, the design of the trial. Such large outcome based databases could be useful in helping sponsors to refine the feasibility of trial designs, the endpoint assessments and the effect size that one might expect.“ He further suggested that the Sentinel database could be useful for studying unapproved indications.

Wheadon’s comments reflect the continuing pressure to use the FDA-developed system as a way to modify FDA’s current data standards. It may also indicate an area for further, direct discussions once FDA and industry begin the formal negotiation process for the Prescription Drug User Fee Act reauthorization.

Robert Reynolds, Pfizer Inc. VP-Global Head, Epidemiology in Worldwide Safety, suggested that the IMEDS project could permit drug sponsors to conduct the supportive post-market work in the large Sentinel network that was described by the CBER plans for Gardasil-9.

“What we heard earlier about risk management and pre and post-approval safety assessment,” Reynolds said, “that would really mirror what we would look to have manufacturers use the IMEDS program in the same way.” Reynolds elaborated:

“If there were a new therapy in the pre-approval stage being reviewed by the FDA and there were identified and potential risks, then I would imagine that it would be agreed that those risks would be monitored in the IMEDS system. It could either be a regulatory commitment or a regulatory requirement.”