US FDA’s Current Technical Standards Not A Good Fit For Real-World Data, Stakeholders Say
Executive Summary
The agency’s existing technical standards are better-suited to clinical trial data, and mapping RWD to these formats will result in the loss of granularity and introduce new opportunities for human error, pharmaceutical industry representatives and data companies say in comments on a draft guidance.
The US Food and Drug Administration is trying to shoehorn real-world data into existing technical standards that are better-suited to clinical trial data, pharmaceutical industry stakeholders and data companies say.
In comments on the FDA’s draft guidance on data standards for drug submissions containing real-world data, stakeholders raise concerns about the loss of detail that can result from having to map RWD to a standard that is currently accepted by the agency, and they request the FDA support the review of RWD in diverse formats through means that require minimal transformations.
The October 2021 guidance “seeks to align modern data sets to data standards (those existing within the FDA) created for a different era in health care research,” the Biotechnology Innovation Organization’s comments state. “BIO recommends that the agency consider providing guidance for a more far-reaching approach to data standards and create a new data standard approach tailored for RWD, given the evolving and emerging nature of RWD assets that can be at the FDA's disposal to facilitate data use for 21st Century research problems.”
The draft guidance discusses considerations for the use of data standards currently supported by the FDA in submissions containing study data derived from RWD sources, such as electronic health records, medical claims data and registries. (Also see "Real-World Evidence: ‘Labor-Intensive’ Data Standardization May Be Required By US FDA" - Pink Sheet, 21 Oct, 2021.)
The guidance was one of four RWD/real-world evidence documents released in the latter half of 2021. The others related to electronic medical records and claims databases, registries, and regulatory considerations. (Also see "Real-World Evidence: US FDA Urged To Leverage Prior Validation Work and Regulatory Experience" - Pink Sheet, 17 Feb, 2022.)
Data Standards In FDA Catalog …
Currently, sponsors submitting clinical and nonclinical study data, including information derived from RWD sources, in certain applications are required to use the supported data standards listed in the agency’s Data Standards Catalog. In the case of clinical study datasets, this means the Clinical Data Interchange Standards Consortium’s (CDISC) Study Data Tabulation Module (SDTM) or Analysis Data Model (ADaM).
The guidance includes two potential options to permit use of RWD in drug submissions through mapping, which is the process of creating data element linkages between two distinct data models:
-
Translating the codes to their mapped structured definitions, with subsequent mapping to appropriate CDISC controlled terminologies; or
-
Mapping all original codes to the least granular analogous codes, and then mapping those to CDISC controlled terminologies.
The guidance acknowledges the first option is more labor intensive, but the second option is likely to result in the loss of more detail.
The FDA said it plans to issue further guidance and/or update its standards catalog for study data derived from RWD sources.
… Are Not Aligned With RWD Formats
In comments on the draft guidance, pharma industry representatives and data companies supported the FDA’s plans to update the standards catalog, asserting the currently accepted standards are not a good fit for RWD.
The data formats in the catalog “are designed to organize clinical trial data and do not consistently align with data organization and formats in RWD sources,” state comments from the RWE Alliance, a coalition of real-world data and analytics companies. (Also see "Data Firms Form Real-World Evidence Alliance, Giving Them A Voice With US FDA, Congress" - Pink Sheet, 19 May, 2021.)
“As the draft guidance acknowledges, this lack of alignment can create challenges in converting RWD to a currently supported data format. For example, granularity in the source data (and the insights that can be gleaned from this information) may be lost when mapping the data to a currently supported format, or certain RWD sources may be excluded from use as a threshold matter due to a lack of alignment,” the RWE Alliance said.
“Granularity in the source data (and the insights that can be gleaned from this information) may be lost when mapping the data to a currently supported format, or certain RWD sources may be excluded from use as a threshold matter due to a lack of alignment.” – RWE Alliance
“These consequences may result in reduced internal validity and/or external validity (i.e., generalizability) for submitted studies,” the group said. In addition, “the linkage of data across multiple RWD sources can raise further considerations because each source may diverge from FDA’s currently supported data standards in different ways.”
The FDA’s currently supported standards were developed “for purposes of submitting randomized clinical trial results to demonstrate substantial evidence of effectiveness,” state comments from the Duke-Margolis Center for Health Policy. “Given this emphasis on supporting clinical trial data, CDISC standards may define key concepts and terms differently compared to RWD sources.”
There is a broad set of vocabularies included in RWD sources that differ from CDISC-supported formats, Duke-Margolis said.
“Often vocabularies like SNOMED and RXNorm are used in RWD sources, whereas CDISC standards are specific to MedRA and WHODrug Global vocabularies,” the health policy center said. “We note such differences in vocabularies may result in mapping discrepancies that cause errors in the submitted dataset. Therefore, we suggest FDA support and encourage transparent data mappings across these vocabularies.”
Data analytics company Aetion said RWD sources typically do not contain enough information to appropriately map to CDISC SDTM or ADaM formats. Furthermore, existing SDTM and ADaM concepts do fully account for the nuances of the data collected in observational studies.
In addition, mapping RWD is a complex task that introduces new opportunities for human error – and need for FDA reviewer verification – that otherwise would not exist if the data were analyzed in near-native format, Aetion said. Even absent human error, mapping can result in loss of critical detail, which “can materially affect the validity of a study or the fitness of the dataset for answering a particular research question.”
Aetion urged the agency to consider alternate approaches to standardizing the review of RWD submitted to support regulatory decision-making.
“For example, technologies such as centralized, platform-based analytics enable the review and analyses of RWD from a variety of sources, without converting to a single supported data standard,” Aetion said. “Such an approach could enable reviewers … access to source data in a transparent manner while maintaining information and detail that might be lost in the process of mapping the data to a particular standard.”
McKesson Corp. requests more specific guidance on the process for mapping RWD to acceptable data standards.
“While the guidance references the need for sponsors to perform a mapping of RWD to study data submission standards, we would like to see this guidance provide more clarity as to ‘how’ the mapping should be performed,” McKesson’s comments state. “A prescribed process by the FDA will be helpful to the agency and sponsors to further minimize the potential for errors when mapping RWD to study data standards.”
Consider Other Standards
Biopharma industry comments requests the agency consider referencing other specific data standards and common data models as potentially feasible for RWD submissions. These include:
-
Fast Healthcare Interoperability Resources (FHIR);
-
Sentinel;
-
Observational Medical Outcomes Partnership (OMOP) CDM; and
-
PCORnet CDM.
Janssen R&D LLC’s comments seek more clarity as to when waivers of data standard requirements for RWD submissions may be considered.
“We request that FDA consider including in the guidance types of submissions which include RWD that would be suitable or not suitable for consideration of waivers for conversion to the SDTM standard,” Janssen said. “Additionally, we also request that FDA reference, under circumstances when a waiver is granted, alternative standards to SDTM that may be acceptable in submissions which include RWD.”
“While the expansion of RWD types and sources poses challenges to data standardization, it also presents an opportunity for the FDA to re-evaluate data standards for existing data sources.” - PhRMA
The Pharmaceutical Research and Manufacturers of America encouraged the FDA to exercise flexibility in granting waivers on data standard requirements for RWD submissions, “particularly for those studies that were initiated prior to issuance of this draft guidance where sponsors did not have the benefit of clarity that FDA has provided here and in other recently published RWD/E draft guidances.”
PhRMA and other commenters urge the agency to seek broad input on data standards that should be included in any update to the catalog.
“While the expansion of RWD types and sources poses challenges to data standardization, it also presents an opportunity for the FDA to re-evaluate data standards for existing data sources and collaborate with clinical researchers, data scientists, standards development organizations, bioinformaticians, technologists, other US government agencies, and other health authorities to better organize data standards for current and future RWD use in a globally harmonized manner,” PhRMA's comments state.
“We strongly encourage FDA to work with these stakeholders as the agency establishes modern data standards best suited to the evolving and emerging RWD assets.”