Adaptive Trials Could Have A Steep Regulatory Learning Curve

Sponsors should take care that drug development efficiencies gained with the advent of adaptive clinical trials aren't offset by lack of opportunity to reflect on data from one trial to the next and "design a thoughtful, complete program," FDA warns in its draft guidance for adaptive trial design.

The ¹ draft guidance on Adaptive Design Clinical Trials for Drugs and Biologics was posted for comment Feb. 26. Comments are due May 27.

Though industry has been eager to fully embrace the new clinical trial methodologies - which provide a more flexible, streamlined approach to clinical development - FDA's first official document on the subject includes several reasons to be cautious about their use, especially if companies use adaptive designs to try to skip the step of doing smaller, earlier exploratory trials.

One such issue is that unexpected findings are "unlikely to be identified by the limited, rapid, interim data analysis of the adaptive study design," the guidance notes. The "apparent time advantage" may well be "less valuable" in the long run if taking time between exploratory and registrational trials would have resulted in a better study, i.e., one adequate for regulatory review.

Lack of time to ponder and explore the full meaning of data might also mean overlooked safety issues, says FDA. Companies that discover a safety signal late in the game because they made rapid decisions about dosing, for example, could find themselves taking time to do additional studies to satisfy the agency's safety concerns.

For those reasons, adaptive design approaches that could eliminate a separate exploratory study are probably best used only when a great deal is already known about a drug and there is minimal risk that an important but previously unrecognized issue will arise, the draft guidance concludes.

Another potential downside of adaptive design is lack of replication. In an adaptive study that evolves from a more exploratory phase into an adequate and well-controlled study after interim analyses, one stage does not confirm the other, FDA asserts.

Because the goal of a single, adaptive design is to use data from all stages of the study to test one or more primary hypotheses, the study remains a single-study source of evidence. The stages cannot therefore be viewed as independent confirmatory studies, as needed for approval.

Limit Modifications To Limit Risk

FDA defines an "adaptive design clinical study" as a study that includes a prospectively planned opportunity for modification of one or more specified aspects of the study design and hypothesis based on interim analysis of data. The fewer modifications the better, or sponsors risk getting into territory better defined as exploratory research - and not suitable for a marketing application.

In fact, FDA recommends that because exploratory studies have less impact on regulatory approval decisions, sponsors practice adaptive designs on them. Indeed, some in industry have advocated that the best time for adaptive strategies is in early development (² (Also see "Time To Be Creative With Adaptive Trials May Be Non-Pivotal Portion" - Pink Sheet, 17 Mar, 2008.)).

The range of adaptive possibilities is broad, including altering inclusion/exclusion criteria or trial arms and changing endpoints in response to input from interim analyses, but all points where adaptation might be implemented must be described prospectively. And the complexity of identifying, documenting and justifying all the adaptations (and the related decision criteria) a trial might employ calls for more front-end planning by sponsors than traditional protocols.

In what the agency calls the "better understood" models, adaptations in most cases are made in response to pooled blinded data, baseline data or data not related to efficacy outcomes. But in the "less understood" options, where circumstances might dictate that the objectives cannot be achieved by other designs, unblinding is involved, and the caution flags are up.

Less Understood Options: Risky But Useful

Dose selection studies might actually be a good use of adaptive designs, because too often pivotal studies examine only a single dose or two doses spanning a narrow dose range "based on a tenuously understood dose-response relationship developed from very limited data," the draft states.

An exploratory dose study with the number of dose groups adaptively decreased - according to a prespecified plan - could also allow addition of new, more preferable doses, the agency proposes. Caution must be exercised, however, to avoid falling into the "exploratory" trap (where the trial would no longer suffice for registration). An adequate and well-controlled trial calls for "careful statistical adjustment" to control bias and "modest pruning" of the number of dose groups.

Adaptive randomization based on relative treatment group responses is another model that if carefully handled could be an improvement over some current registrational trials. The method allocates fewer subjects to doses that appear to have a low probability of efficacy, a high chance of an adverse event or are unlikely to contribute useful information. Again, this option looks a lot like an exploratory study, and it is important to maintain placebo enrollment to maintain a balance in group characteristics. The danger is that the analysis is not as easy for the agency to interpret as when fixed randomization is used.

Other "less understood" models include adapting sample size based on interim-observed treatment effect, adapting patient population based on interim analysis or adapting endpoints.

Preserve Integrity, Maintain Blinding!

The issue of how best to maintain data integrity in a design that inherently exposes so much data has dogged discussions surrounding adaptive trials since the concept was first discussed at least five years ago. In fact, it is arguably the reason the guidance was so long in development (³ (Also see "FDA Puts Hold On Adaptive Trials, Wants More Clarity On Basics" - Pink Sheet, 27 Nov, 2006.)).

By far the agency's overriding concern for adaptive trial designs is that the adaptive process not somehow introduce bias that can lead to a false conclusion that a treatment is effective or to other positive results that are difficult to interpret because there are so many adjustments in the data.

For that reason, the draft guidance repeats the safe harbor advice that "risk of bias is greatly reduced or entirely absent" when adaptations are based only on blinded analyses.

When unblinded analysis is part of a study design, a "well-trusted firewall" - beyond what would be established for a conventional group sequential trial - should be implemented to ensure that statistical and operational biases are not introduced.

An independent steering committee of some kind might be best suited to review an interim analysis of unblinded data and make adaptive decisions, the agency recommends. An independent data monitoring committee could do the job, or a DMC might be delegated only the more standard roles, such as ongoing safety assessments, and a separate adaptation committee used to examine the interim analysis and make recommendations following the predetermined adaptation plan.

Added to the usual trial documentation, adaptive trials must have written standard operating procedures defining who will implement interim analyses and adaptation plans and all procedures accomplishing the implementation, including identifying who will perform the analyses and who will have access to the results. Importantly, the SOP defines what information, under what circumstances, is permitted to pass from the DMC to the sponsor or investigators.

In any event, careful records should be kept of all committee meetings. And the actual interim analysis results and a "snapshot" of the databases used for the analysis and adaptation decision should be retained "in a secure manner."

CROs Should Be Open To On-Site Auditing

In recent years, industry has shifted research duties performed by direct sponsor employees to contract research organizations. However, many of those CROs might not be qualified to carry out an adaptive design study, the draft notes.

If the CRO can't follow the statistical analysis plan or maintain confidentiality, the project is in jeopardy of not coming up to regulatory standards. "Many CROs do not have long histories of carrying out [the] responsibilities" involved in performing interim analyses and making study decisions based on that data, the draft asserts. FDA must be able to verify their compliance, potentially by on-site auditing.

- Shirley Haley ( ⁴ [email protected] )