14 Real-World Evidence and Evidence Integration
Breast cancer in men runs to roughly 2,670 U.S. cases a year. When a drug approved in women with hormone receptor-positive metastatic disease needs a label extension to cover men with the same disease, there are too few of them to power a randomized trial. The randomized controlled trial is the most reliable instrument for establishing that a treatment causes its observed effect. It is not always available, and it is not always enough. A vaccine approved in a clinical trial of healthy volunteers still needs its effectiveness measured across the diverse population receiving it in community pharmacies; a drug studied in a three-year trial needs its long-term safety assessed over a decade; a rare pediatric disease may have only a few hundred patients worldwide, none of whom can ethically be assigned to placebo.
These are not edge cases. They are a large and growing fraction of the evidence questions drug development has to answer, and over fifty years the response has hardened from informal practice into a regulated discipline.
Real-world evidence (RWE) is clinical evidence about the usage and potential benefits or risks of a medical product derived from analysis of real-world data (RWD): data collected outside of traditional randomized clinical trials, including electronic health records, insurance claims, disease registries, patient-reported outcomes, and wearable devices. RWD is the raw material; RWE is what rigorous analysis of it produces. Conversation runs the two together, but the analytic step between them is the whole difference.
14.1 The Regulatory Framework
Regulatory acceptance of real-world evidence was built statute by statute and guidance by guidance, and it still varies by jurisdiction.
EMA and International Harmonization
The EMA’s approach leans infrastructure-centric. Alongside sponsor-facing documents such as its guideline on registry-based studies, EMA established DARWIN EU (Data Analysis and Real World Interrogation Network) as a coordination center for regulatory-led RWD studies, pulling from databases across European healthcare systems: the UK’s CPRD (Clinical Practice Research Datalink, >60 million patient records), Spain’s BIFAP, Italy’s Pedianet, and the Danish national registries (European Medicines Agency 2026). EMA scientific committees commission the studies and feed the results to national competent authorities and HTA bodies. Sponsors cannot commission one directly.
EMA’s April 2024 guide (EMA/152628/2024) describes how regulators can request RWD studies and what study types are available. In parallel, ICH adopted its first harmonization signal on RWE in June 2024: a reflection paper on “Pursuing Opportunities for Harmonisation in Using Real-World Data to Generate Real-World Evidence.”
ICH E6(R3) (2025) (International Council for Harmonisation 2025), the updated Good Clinical Practice guideline, addresses hybrid designs that incorporate EHR and RWD within interventional trials, extending principles of data integrity and traceability to these sources.
Fit-for-Purpose: The Governing Concept
Both FDA and EMA converge on fit for purpose as the evaluative standard, and the weight falls on purpose. A registry dataset that serves as an external control for a natural history comparison in a rare pediatric disease may be unfit as the primary comparator for a Phase III efficacy claim in a common chronic condition.
The assessment adds a third test to relevance and reliability: analytic validity, whether the proposed analysis can extract the target estimand from the available data without biases that exceed what is acceptable for the regulatory purpose. Sponsors who engage with FDA early, through pre-submission meetings under the Advancing RWE Program, are encouraged to settle the data source and analysis plan there rather than presenting RWE for the first time in a marketing application.
HTA and Payer Frameworks
Regulatory approval and market access are separate decisions with separate evidence standards. FDA asks whether a drug is safe and effective for the proposed indication; HTA bodies such as NICE (National Institute for Health and Care Excellence, UK) ask whether it is cost-effective relative to standard of care, for a specific patient population in a specific healthcare system. The two questions call for different evidence, and they make different demands on RWE (Table 14.2).
| Dimension | Regulatory (FDA/EMA) | HTA/Payers (NICE, payers) |
|---|---|---|
| Central question | Does it work under trial conditions or real-world use? | Is it cost-effective in this health system? |
| Comparator | Placebo or trial-specified active comparator | Actual standard of care in that country |
| Time horizon | Trial duration | Long enough to capture all important cost and outcome differences, often a lifetime extrapolated beyond the trial (National Institute for Health and Care Excellence 2022) |
| Population | Label indication | Patients who will receive reimbursement |
| RWE role | Label expansion, post-approval commitments | Fills long-term effectiveness gaps; informs HTA models |
NICE’s RWE framework (ECD9, June 2022) provides structured guidance for RWE studies submitted in technology appraisals. Outcomes-based managed entry agreements (MEAs), reimbursement made conditional on accumulation of real-world evidence confirming effectiveness in routine practice (as distinct from the finance-based MEAs that turn on discounts and price-volume caps), have become common in Europe. Their designs vary. Under coverage with evidence development, the payer covers the drug for a limited period while an agreed study collects data, then maintains, extends, or withdraws coverage, or adjusts the price, according to the results; under payment-by-results schemes, payment or refunds depend on whether each patient reaches a pre-specified response (Wenzl and Chapman 2019). When one study is meant to serve both the regulator and the HTA body, it has to satisfy both sets of evidentiary standards at once.
14.2 Data Sources
A data source is defined as much by what it leaves out as by how many patients it holds.
Electronic Health Records are the richest source of clinical detail: diagnoses, procedures, medications, laboratory values, vital signs, and clinical notes. Their coverage, though, is bounded by the patient’s engagement with a particular healthcare system: a patient who sees a cardiologist at a different hospital does not appear in the EHR. U.S. EHR networks of regulatory-grade scale include Flatiron Health (>5 million oncology patients across ~220 practices, with structured abstraction of tumor response and genomic data), Epic Cosmos (>200 million patients across Epic-using health systems), and PCORnet. In the UK, CPRD supplies >60 million patient records from general practice. EHR data are collected for clinical care, not research: coding completeness and measurement fidelity are driven by billing incentives and workflow conventions, not study protocols.
Administrative claims and insurance data capture billing transactions: ICD diagnosis codes, procedure codes, drug dispensing (NDC codes), and enrollment periods. Coverage is population-level for the insured population, billed services are captured throughout enrollment (care paid out of pocket or by another payer is missed), and follow-up can be long. U.S. sources include Medicare (CMS, covering >65 million beneficiaries), Medicaid, and commercial claims (e.g., Optum Clinformatics, Merative MarketScan, and IQVIA). The tradeoff is clinical depth: claims capture what was billed, not what the patient’s hemoglobin was or how advanced their tumor was.
Disease and condition registries are purpose-built systems, prospective or retrospective, that collect uniform data on a population defined by a disease, exposure, or condition. On data quality they sit between clinical trials and EHR: entry criteria are defined, data elements standardized, outcomes often adjudicated. The Surveillance, Epidemiology, and End Results (SEER) program covers ~48% of the U.S. population across its core registries. The Scandinavian cardiac registries (SCAAR, SWEDEHEART) have been the foundation for large-scale registry-based randomized trials. Natural history registries (the Pompe Registry, the Cystic Fibrosis Foundation Patient Registry, rare disease-specific databases) supply the comparator population for single-arm trials in rare disease.
Linked databases combine two or more data sources using patient identifiers. SEER-Medicare links cancer diagnoses with Medicare claims for fee-for-service enrollees and encounter records for Medicare Advantage enrollees, which supports studies of clinical outcomes and health service utilization together. Flatiron Health linked its EHR with Foundation Medicine’s comprehensive genomic profiling (Flatiron+FMI), enabling studies of genomic markers as predictors of real-world treatment outcomes. Whether a linkage is deterministic (exact identifier match) or probabilistic, and what its error rate is, feeds directly into bias.
Global RWD networks enable federated analysis across many institutions. The FDA’s Sentinel System maintains over 1.3 billion person-years of mostly U.S. administrative claims data (covering more than 371 million unique patient identifiers as of April 2024; a patient who changes health plans can hold more than one) for active post-market safety surveillance, and is now expanding to some effectiveness analyses (Sentinel Operations Center 2025); DARWIN EU serves the analogous function for EMA. The OHDSI/OMOP network is a global open-science consortium of 544 data sources across 54 countries representing >974 million unique patient records, all standardized to the OMOP Common Data Model (Ryan 2024). Standardizing the vocabulary (SNOMED, RxNorm, LOINC) and the table structure is what lets identical analysis code run across distributed sites without patient data leaving any institution. PCORnet, the U.S. patient-centered clinical research network, links 78 U.S. health systems, with more than 13,000 hospitals and other care sites, on its own Common Data Model, which was originally based on the FDA’s Mini-Sentinel Common Data Model (Cohen et al. 2026; PCORnet 2025).
Patient-level data sharing platforms open IPD (individual patient data) from completed clinical trials, the raw material for an external control built from historical trial patients. Vivli hosts >6,000 clinical trials; the YODA Project (Yale Open Data Access) provides third-party review of data sharing requests for J&J and Kenvue datasets; Project Data Sphere focuses on oncology trial control arm data. These data came out of trials, under standardized protocols, unlike EHR or claims.
Outcome Ascertainment: The Critical Quality Issue
A structural weakness of most RWD sources is that outcomes were not designed to be measured. The diagnosis code for “disease progression” in an EHR reflects what a physician documented for billing purposes, not a protocol-specified assessment against RECIST criteria at a pre-specified timepoint. The gap between documented and protocol-adjudicated outcomes is why real-world endpoints carry their own names: rwOS (real-world overall survival), rwPFS (real-world progression-free survival), and rwTR (real-world tumor response). These are not equivalent to their trial-measured counterparts.
Friends of Cancer Research ran multi-partner pilots that standardized the definitions of these endpoints so that they can be measured the same way across RWD sources; its 2023 considerations call for evidence that the dataset is fit for purpose when RWD support a regulatory submission and urge caution in comparing real-world endpoints with trial endpoints, because the populations and measurements differ (Friends of Cancer Research 2023). Flatiron Health has published validation studies comparing rwOS and rwPFS against clinical trial outcomes across multiple tumor types.
Phenotyping algorithms (computable definitions of conditions, exposures, and outcomes applied to RWD) require validation against a reference standard, often medical record review, sometimes an established external source such as national death records (U.S. Food and Drug Administration 2024). The positive predictive value of ICD coding varies widely by condition and again across care settings: in one systematic review, it was 86-98% for hip fracture when hospital and physician claims were combined, but below 60% for osteoporosis diagnoses (Hudson et al. 2013). Non-differential outcome misclassification usually biases effect estimates toward the null, although with perfect specificity the risk ratio can be left unbiased; differential misclassification, where the rate differs between treatment groups, can bias in either direction. A serious RWE study reports the algorithm, its validation statistics against the reference standard (PPV, sensitivity, and specificity), and a sensitivity analysis for outcome misclassification.
14.3 Study Design Taxonomy
The designs below run from no randomization at all to randomization executed inside a national registry. The clinical question fixes the estimand (International Council for Harmonisation 2019); where a study sits on that range determines what must be assumed to estimate it, and those assumptions, together with data quality and study conduct, set how much weight a regulator can give the result (U.S. Food and Drug Administration 2018).
Observational Comparative Effectiveness Research
In observational CER both treatment groups are drawn from RWD and nothing is randomized: patients who received Drug A in routine clinical practice are compared with patients who received Drug B. The central challenge is confounding: patients and their physicians chose their treatments for reasons that may also predict outcomes. Propensity scores, outcome regression, and doubly robust estimators attempt to control for the measured confounders; the unmeasured ones remain the irreducible limitation.
Observational CER is most credible when the outcome is objective and reliably captured in routine care (overall survival is better than symptom scores), the comparison is between two established alternatives rather than between treated and untreated, which concentrates confounding by indication, and the database is large enough to achieve covariate balance after matching or weighting. Beyond that, sensitivity analyses have to demonstrate that the result is robust to plausible unmeasured confounding.
Natural History and Registry Studies
Natural history studies characterize disease progression, event rates, and mortality in untreated or standard-of-care populations. They do not compare treatments. In rare disease drug development a well-designed natural history study is often the indispensable prerequisite for a single-arm trial: without a reliable characterization of the untreated disease course, the single arm’s outcomes cannot be interpreted against anything. FDA’s 2023 guidance on registry assessment sets out the data elements, follow-up duration, and outcome adjudication a registry needs to serve as a regulatory-grade external control comparator.
External Control Arms and Synthetic Control Arms
An external control arm (ECA) replaces the concurrent randomized control arm with a population assembled from outside the trial: everyone in the trial receives the investigational treatment, and their outcomes are compared with a pre-specified external cohort drawn from historical trials, registries, EHR, or claims. The FDA’s 2023 guidance on externally controlled trials (U.S. Food and Drug Administration 2023) treats ECAs as most appropriate where concurrent randomization is infeasible or unethical, the disease has a well-characterized natural history, outcomes can be measured reliably in the external data source, and the comparison population is demonstrably similar to the trial population at baseline.
A synthetic control arm (SCA) is the version in which patient-level data from historical trials, often proprietary datasets, are statistically matched to the current trial’s population by propensity score or covariate-matching algorithms. Medidata’s platform, drawing on >38,000 trials and 12 million patients, is one of the largest commercial SCA systems. In a widely cited example from 2020, Medicenna Therapeutics gained FDA support to use a synthetic control arm within a hybrid Phase 3 registrational design for MDNA55 in recurrent glioblastoma, in which matched external controls supply roughly two-thirds of the pooled control arm, reducing the number of patients randomized to standard of care (Medicenna Therapeutics Corp. 2020; Medidata Solutions 2024). SCAs drawn from historical RCT patient data are generally stronger than ECAs from EHR or claims because the data were collected under controlled conditions.
Hybrid Randomized Designs and Bayesian Augmented Controls
Hybrid designs randomize a reduced concurrent control arm while borrowing information from external data to increase effective sample size. The borrowing runs through Bayesian methods that “discount” the external data according to how consistent it is with the emerging concurrent control data, and three frameworks dominate.
The power prior (Ibrahim and Chen 2000) modifies the likelihood of the historical data by a parameter \(\alpha \in [0,1]\): \(\alpha = 0\) borrows nothing, \(\alpha = 1\) fully pools historical and current data. A fixed \(\alpha\) chosen before the trial controls the degree of borrowing; a data-adaptive \(\alpha\) estimated from the data consistency is more robust to historical-current heterogeneity but introduces operational complexity.
The Meta-Analytic-Predictive (MAP) prior (Neuenschwander et al. 2010) derives the prior from a meta-analysis of multiple historical studies, accounting for between-study heterogeneity through a hierarchical model. The resulting prior on the control event rate carries both the central estimate from historical data and the uncertainty about how far the current population may differ. MAP priors are implemented in the RBesT R package and have been used in pediatric trials, where adult trial data informs the prior for smaller pediatric randomized comparisons.
Commensurate priors (Hobbs et al. 2011) make the amount of borrowing a function of the observed similarity between historical and current control arm data, updating as current data accumulates: when the two are concordant, borrowing is high; when they diverge, it shrinks. That adaptation is the protection against the principal risk of Bayesian borrowing: bias introduced by over-weighting historical data that does not represent the current population.
How much effective sample size the borrowed data contribute depends on the discount applied. Regulatory agencies expect pre-specified borrowing parameters, with operating characteristics evaluated through simulation across scenarios of historical-current concordance and discordance. FDA’s January 2026 draft guidance on the use of Bayesian methodology in trials of drugs and biological products (U.S. Food and Drug Administration 2026) speaks directly to this setting, asking that borrowed information be valid, reliable, and interpretable, judged through five factors that include data quality, pre-specification of the borrowing, and the relevance and design of the contributing studies.
Registry-Based Randomized Controlled Trials
A registry-based RCT (RRCT) conducts randomization within an existing clinical registry and uses it for enrollment, baseline data, and some or all outcomes; some RRCTs rely on registries alone for outcomes, while others add trial-specific follow-up and adjudication. The TASTE trial (2010-2013) randomized 7,244 patients with ST-elevation myocardial infarction to thrombus aspiration versus conventional PCI within the SCAAR Swedish cardiac registry; outcomes were collected from registry records without source data verification. The VALIDATE-SWEDEHEART trial compared bivalirudin versus heparin in myocardial infarction within the same registry infrastructure but added nurse telephone follow-up at 7 and 180 days and blinded central adjudication of the primary end point (Erlinge et al. 2017, 2016). Both appeared in the New England Journal of Medicine, combining the operational efficiency of registry data collection with the inferential validity of randomization.
RRCTs are most powerful where an existing registry covers the relevant disease, collects the primary outcome reliably enough to carry the comparison, and can randomize at the point of care through its own interface. The Scandinavian registries (supported by universal personal identification numbers enabling linkage across data sources) have been the paradigmatic setting, but the methodology travels wherever a well-maintained registry exists.
Platform Trials and External Data
Platform trials (multi-arm adaptive protocols that can add and remove treatment arms over time) are treated in Section 13.6 and Section 12.5. Three of their features belong here.
External control arms in biomarker-selected platforms. Basket trials run one drug across tumor types sharing a molecular alteration; umbrella trials run several drugs within one tumor type, stratified by biomarker. Either way a biomarker-defined cohort can be too small to randomize internally, and the platform reaches for an external control arm, historical or registry-based, for that subgroup. The FDA master protocols guidance (U.S. Food and Drug Administration 2022) recommends a single common control arm wherever randomization is feasible and addresses nonrandomized, activity-estimating substudies where it is not; the design requirements for the external comparator itself, pre-specification of the external population and demonstrated baseline comparability, come from FDA’s externally controlled trials guidance (U.S. Food and Drug Administration 2023). Oncology platforms such as NCI-MATCH and LUNG-MAP have assessed molecularly selected cohorts against historical or external benchmarks.
Streamlined outcome collection in platform designs. The RECOVERY trial (Randomised Evaluation of COVID-19 Therapy) enrolled >40,000 patients across NHS hospitals in the UK with minimal trial data collection: coordinators enroll electronically, treatment is assigned, a single online follow-up form is completed at discharge, death, or day 28, and routine health care and registry data supplement it (RECOVERY Collaborative Group et al. 2021). The operational saving is dramatic. The REMAP-CAP trial (Randomized, Embedded, Multifactorial Adaptive Platform trial for Community-Acquired Pneumonia) embedded enrollment and randomization in intensive care routines but still recorded outcomes on trial case report forms, which one U.S. health system populated largely automatically from its EHR (Angus et al. 2020; King et al. 2024). Both are embedded pragmatic designs at platform scale, running a multi-arm adaptive protocol inside routine care.
Bayesian borrowing inside a platform. In multi-domain platforms (REMAP-CAP ran immunological, anticoagulation, and antiviral domains at once), one analysis model includes patients from every domain, but not as a shared control arm. Patients randomized in other domains sharpen the estimates of covariate effects such as site, age, and calendar time, while each intervention’s effect is estimated from patients who could have been randomized to it within its own domain. The borrowing runs between related treatment effects: REMAP-CAP nested tocilizumab and sarilumab under a common interleukin-6 receptor antagonist effect and shrank each estimate toward it when the two agreed (REMAP-CAP Investigators et al. 2021). It is the hierarchical logic of the MAP prior described above, applied to concurrent randomized comparisons inside one protocol instead of to historical controls.
Embedded and Pragmatic Trials
Pragmatic trials assess treatments under conditions approximating routine clinical practice: broad eligibility criteria, usual care comparators, real-world outcomes, and heterogeneous patient populations. Embedded pragmatic trials (EPTs) go further, living inside healthcare system infrastructure and using the EHR or claims for recruitment, randomization, and outcome collection. IMPACT-AFib is the canonical U.S. example: claims flowing through FDA’s Sentinel distributed database identified patients with atrial fibrillation in five health plans, and patients (not practices) were individually randomized to receive or not receive a mailed educational outreach. The primary analysis covered those with no claims evidence of oral anticoagulant use in the prior 12 months, and its outcome, anticoagulation initiation, came from claims (Pokorney et al. 2022). Claims replaced dedicated data collection, not research staff: two coordinating centers and investigators and programmers at the five participating health plans ran the distributed queries, checked refreshed data extracts, and arranged the mailings (Garcia et al. 2020).
FDA’s 2024 draft guidance on integrating randomized trials into routine clinical practice sets out the agency’s recommendations for EHR-based recruitment and outcome collection in an EPT. PCORnet and the NIH Health Care Systems Research Collaboratory have built the infrastructure enabling EPTs at scale.
14.4 Pharmacoepidemiology Design Principles
Observational RWE studies are subject to a set of biases that have no analog in randomized trials. These biases arise from the way treatment is assigned in clinical practice and from the structure of RWD sources. Understanding them is prerequisite to critically reading or designing an RWE study.
Confounding by indication is the most pervasive. Physicians assign treatments based on patient characteristics that may also predict outcomes: sicker patients get more aggressive treatment; frail patients get less. If those characteristics are not fully captured in the RWD, the treatment comparison is contaminated by the indication. The active comparator, new user (ACNU) design is the principal design-level solution.
The active comparator design draws the comparator group from patients who received a different active treatment for the same indication at the same time, not from patients who received no treatment at all. If both drugs are used for the same condition in the same clinical context, much of the confounding by indication cancels: the choice between Drug A and Drug B is less driven by disease severity than the choice to prescribe any drug.
The new user design restricts the cohort to patients who newly initiated treatment (no use in a washout period, typically 6-12 months). That aligns cohort entry with the start of treatment, so baseline covariates can be measured before exposure, mirroring a randomized trial that enrolls patients not already on the drug (a finite washout cannot show that they never used it). It also avoids prevalent user bias: prevalent users have already survived early adverse events and been selected as tolerating the drug, a systematically different population from incident users. The two-drug version of that problem is depletion of susceptibles. If early users of Drug A who experienced adverse events switched to Drug B, the remaining Drug A user population is enriched with tolerators and depleted of susceptibles, creating an apparent safety advantage for Drug A. Applying the restriction to both exposure groups removes that selection at cohort entry.
Immortal time bias occurs when the interval between cohort entry and first treatment exposure is incorrectly classified as follow-up time in the treated group (Suissa 2008). During that interval the patient could not have died: they are still alive to receive the treatment. Counting it as follow-up hands the treated group an artificial survival advantage. Target trial emulation (see Chapter 11) prevents this by construction, aligning the eligibility date with the treatment start date.
Index date alignment requires that baseline covariates be measured before the index date (treatment initiation) and outcomes after. Adjusting for a covariate that could itself have been caused by treatment, such as a post-initiation lab value, can block part of the effect being estimated or induce collider bias, depending on where it sits in the causal structure. The rule sounds trivial. It is routinely violated when analysts reach for “all available” data without strict time-ordering.
14.5 Statistical Methods for Real-World Evidence
The foundational methods are in Chapter 11. What observational causal inference adds begins before any of them, with the causal structure of the problem itself.
Causal Diagrams as the Starting Point
Three kinds of variable have to be told apart. Confounders are shared causes of treatment and outcome, and must be controlled. Mediators sit on the causal pathway from treatment to outcome, so controlling them blocks the very effect being estimated. Colliders are common effects, whether of treatment and outcome themselves or of their causes, and controlling one opens a spurious association. Directed Acyclic Graphs (DAGs) provide the formal language for the distinction (Pearl 2009): a DAG specifies the minimum sufficient adjustment set for the causal estimand, and just as precisely what to leave alone, mediators in many contexts and colliders unless the path they would open is blocked elsewhere.
DAGs are widely taught in pharmacoepidemiology and give an RWE analysis an explicit, reviewable rationale for its choice of confounders, and DAGitty makes the construction and analysis accessible without formal training in graphical models.
Propensity Score and Doubly Robust Methods
Propensity score (PS) methods (covered in Chapter 11) are the workhorses of observational RWE. The PS model predicts treatment assignment from baseline covariates measured before the index date, and variable selection follows its own rules: include the causes of the outcome, exclude instruments, and be cautious with variables associated only with treatment. Balance is demonstrated, never assumed. Report standardized mean differences (SMD) before and after adjustment, with SMD < 0.1 the conventional threshold for acceptable.
Doubly robust estimators (AIPW, g-computation with IPW, TMLE) are more resistant to model misspecification than PS-only or outcome-model-only approaches, and should be preferred for high-stakes primary analyses. Targeted Maximum Likelihood Estimation (TMLE) combines machine learning-based estimation of both the PS and the outcome model with a targeted bias-correction step that restores valid inference when data-adaptive models are used, provided the usual regularity conditions hold (adequate positivity, sufficiently accurate nuisance estimation, and cross-fitting or a comparable safeguard against overfitting).
Instrumental Variables
When unmeasured confounding is suspected, instrumental variables (IV) offer an alternative identification strategy. An instrument is a variable that: (1) affects treatment; (2) affects the outcome only through its effect on treatment; (3) shares no common cause with the outcome, so that it is unassociated with the unmeasured confounders of the treatment-outcome relationship. In pharmacoepidemiology, instruments include physician prescribing preference: some physicians systematically prefer Drug A, independent of patient characteristics. Formulary changes that shift prescribing toward one drug based on calendar time, and geographic discontinuities in access, are used the same way.
The three conditions alone only bound the effect. In the standard case of a binary instrument and a binary treatment, adding monotonicity (no defiers, patients who would do the opposite of whatever the instrument encourages) lets IV identify the local average treatment effect (LATE), the effect in the subpopulation whose treatment status is changed by the instrument, not the average treatment effect in the full population (Baiocchi, Cheng, and Small 2014). So IV earns its place when a strong instrument exists and the LATE is clinically meaningful. Weak instruments produce imprecise estimates.
Difference-in-Differences and Regression Discontinuity
Difference-in-differences (DiD) compares the pre-to-post change in outcomes in a treated group with the pre-to-post change in an untreated group, differencing out unmeasured confounders whose effects do not change over time. The identifying assumption is parallel trends: absent the treatment, the two groups would have followed the same trajectory. Because it concerns the untreated trajectory after treatment starts, it cannot be tested directly. Parallel pre-treatment trends make it more plausible and belong in the report, but a pre-trend test can lack the power to detect a violation large enough to matter (Roth 2022). DiD is common in policy evaluation and increasingly used in pharmacoepidemiology for studies of formulary changes, guideline implementations, or drug market entries.
Regression discontinuity (RD) exploits threshold-based treatment assignment: patients scoring just above a clinical cutoff receive treatment, patients just below do not, and near the cutoff the two groups are approximately comparable. RD identifies a local treatment effect at the threshold, useful wherever clinical practice sets one: CD4 count thresholds for starting antiretroviral therapy (Bor et al. 2014), lipid level thresholds for statin prescribing, age-based eligibility cutoffs.
Unmeasured Confounding Assessment
No observational method fully eliminates unmeasured confounding, so the response is to quantify how strong an unmeasured confounder would have to be to explain the observed result away. That quantity is the E-value (VanderWeele and Ding 2017), covered in Chapter 11: a large one means only a very strong unmeasured confounder could explain the result away. It addresses unmeasured confounding only; selection bias and outcome misclassification need their own checks.
Negative control outcomes and exposures are, respectively, outcomes the treatment under study cannot cause and exposures that cannot cause the study outcome, chosen to share the confounding and measurement problems of the main comparison. If the same analysis finds an association with a negative control that chance does not explain, the method is producing spurious results (systematic bias), and the primary estimate is suspect to the extent that the control shares its sources of bias (Lipsitch, Tchetgen Tchetgen, and Cohen 2010). It is a powerful validity check.
Quantitative bias analysis (QBA) goes further, into probabilistic sensitivity analysis: Monte Carlo simulation over plausible values of unmeasured confounder prevalence and effect size. What comes out is a bias-adjusted distribution for the effect estimate, in place of a single point estimate that assumes no unmeasured confounding.
14.6 Landmark Applications
Three cases carried real-world evidence from method into regulatory and clinical practice.
Oncology: Ibrance for Male Breast Cancer
Male breast cancer occurs in approximately 2,670 U.S. patients annually, and only a fraction of them have the hormone receptor-positive, HER2-negative advanced disease palbociclib treats, so a randomized trial in men would be slow to enroll; men had also been excluded from the trials behind palbociclib’s earlier approvals (U.S. Food and Drug Administration 2019). In April 2019 FDA approved Pfizer’s supplemental NDA for palbociclib (Ibrance) to add male breast cancer to the indication, the canonical landmark case for RWE-supported label expansion. The evidence came from three sources: Flatiron Health’s oncology EHR supplied real-world tumor response and treatment patterns; IQVIA insurance data added treatment patterns and concomitant medications; Pfizer’s global post-marketing safety database carried the adverse event profiles in male patients.
FDA treated this package as supportive rather than primary. The approval rested on palbociclib’s efficacy in women, established in randomized trials, and on a strong mechanistic rationale for the same effect in men; the Flatiron data, 3 responses among the 12 men on palbociclib whose tumors were assessed, added modest support, and FDA left the IQVIA results out of its benefit-risk assessment because of their limitations (U.S. Food and Drug Administration 2019). The label’s statement about men concerns safety: “Based on limited data from postmarketing reports and electronic health records, the safety profile for men treated with IBRANCE is consistent with the safety profile in women treated with IBRANCE.” FDA never called the real-world data equivalent to a randomized trial; it accepted the evidence as adequate for one specific regulatory question, whether the drug’s pharmacology and clinical profile in men are consistent with benefit in women, where randomized evidence exists.
COVID-19 Vaccine Effectiveness at Scale
Vaccines were authorized under Emergency Use Authorization on clinical trial data in 2020-2021, and real-world effectiveness still had to be measured across diverse populations, viral variants, and healthcare settings, questions the trials could not answer in time. What followed was a rapid, large-scale deployment of RWE.
The dominant design was the test-negative design (TND): among people with symptoms who were tested for COVID-19 by PCR, compare vaccination status between those who tested positive and those who tested negative. Restricting to people who sought testing helps control for differences in healthcare-seeking behavior, and the comparison estimates vaccine effectiveness against symptomatic disease. Lopez Bernal and colleagues applied it to England’s national surveillance data to estimate BNT162b2 effectiveness against the Delta variant (Lopez Bernal et al. 2021). Dickerman and colleagues took a different route, emulating a target trial in U.S. veterans data to compare BNT162b2 against mRNA-1273 directly, at continental scale (Dickerman et al. 2022).
Observational studies also documented protection waning with time since vaccination, and that evidence informed booster policy. The COVID-19 experience showed that RWE can provide actionable regulatory-grade evidence in a compressed timeframe when the methodological infrastructure is in place.
Registry-Based Trials: TASTE and SWEDEHEART
TASTE (Fröbert et al. 2013) and VALIDATE-SWEDEHEART (Erlinge et al. 2017), both described above, randomized thousands of patients for a fraction of traditional Phase III cost (Erlinge 2021), in a population more representative of clinical practice than the carefully selected cohorts conventional trials assemble. TASTE took its endpoints from national registries. VALIDATE-SWEDEHEART added its telephone follow-up and adjudication because the registry did not capture bleeding complications well enough. Where a suitable registry already exists, randomization can be built into its interface cheaply, but an endpoint the registry does not capture reliably still needs trial-specific follow-up and adjudication, which adds complexity and cost (Erlinge 2021). The evidence carries the inferential advantages of randomization with the population representativeness of observational data. The design is now being replicated internationally.
14.7 Infrastructure: OMOP, OHDSI, and Federated Analytics
Running any of this at scale takes computational infrastructure, and the OHDSI (Observational Health Data Sciences and Informatics) community has built it as an open-science ecosystem. The OMOP Common Data Model standardizes both the table structure and the vocabulary of observational databases: SNOMED CT for conditions, RxNorm for drugs, LOINC for measurements. Analysis code written once can then be run across the network’s standardized data sources, which in 2024 numbered 544 in 54 countries and together held more than 974 million unique patient records (Ryan 2024). Patient data never leave the institution; only aggregate results are shared.
The HADES toolkit (Health Analytics Data-to-Evidence Suite) comprised 37 R packages in 2024 for running standardized epidemiological analyses on OMOP-format data: cohort construction, propensity score estimation, outcome modeling, negative control calibration, and diagnostics (Ryan 2024). PhenotypeLibrary maintains a version-controlled repository of cohort definitions (phenotype algorithms) that undergo voluntary open peer review, with diagnostics run on volunteer data partners to check that each definition executes across the network (Rao 2026).
The DARWIN EU network applies OMOP-compatible standards across its European data partners, so the same standardized analytic code that runs on OMOP databases elsewhere can be applied to European data within a regulatory-grade framework.
Where a model has to be trained on patient-level data that cannot be pooled, federated learning trains it across the distributed sites, with model updates rather than patient data crossing institutional boundaries. It costs more complexity than distributed analytics and buys model-level analysis, but the updates are aggregates too and can leak information about the patients behind them, so privacy-sensitive settings (rare diseases, small populations, genomic data) call for safeguards such as differential privacy or secure aggregation (Ogier du Terrail et al. 2025). Owkin has applied it to construct external control arms from real-world data distributed across hospital networks (see Chapter 26).
14.8 Generalizability and Transportability
Even a perfectly unconfounded RWE study estimates the treatment effect in the study population. Whether that estimate carries to the target population, the patients who will actually receive the drug, is a separate question with its own vocabulary: generalizability runs from the study sample to the study population, transportability from the study population to a different target population.
Dahabreh and colleagues built a formal framework, combining evidence from randomized trials and observational data to estimate treatment effects in a target population beyond the trial’s eligibility criteria (Dahabreh et al. 2019, 2020). Measure and model the factors that make the trial population differ from the target, and inverse probability of sampling weights re-weight the trial evidence to represent that target. Weighting works only if those measured factors account for every difference between the populations that bears on the treatment effect, and if every covariate pattern in the target also occurs among trial participants; a group the eligibility criteria excluded outright, on a factor that modifies the effect, cannot be recovered by weighting (Dahabreh et al. 2020).
Trial eligibility criteria systematically exclude older patients, those with comorbidities, and those on concomitant medications, and those are precisely the patients who will receive the drug after approval. Within those limits, transportability methods estimate whether the trial effect size survives that translation, which bears on labeling claims and post-market commitments. The same machinery does the cross-border work: when a drug approved in one country is taken to another for approval or HTA, the approval trial’s evidence may not transport to the target country’s population, and MAIC (Matching-Adjusted Indirect Comparison) is the widely used HTA implementation of the idea.
14.9 When Does RWE Belong in a Development Program?
The conditions that make RWE credible share one logic: confounding is the irreducible threat, and each condition either limits its impact or keeps other biases from adding to it.
The strongest protection comes when the outcome is not subject to differential ascertainment. Overall survival qualifies because death is recorded regardless of which arm the patient was in; hospitalization is a billing event, generated the same way whichever treatment the patient received; treatment dispensing is a pharmacy record. None of the three is complete by default: U.S. EHR and claims data generally record a death only when it occurs under medical care, unless they are linked to vital statistics, and care or prescriptions obtained outside the contributing health system or insurance plan are missed (U.S. Food and Drug Administration 2024). Completeness has to be established for each source, often through linkage. Symptom scores, imaging-defined tumor progression, and patient-reported outcomes fare worse, because they depend on protocol-mandated assessments that routine care does not perform. A completely captured objective outcome leaves confounding by treatment selection as the problem to solve. A subjectively ascertained one adds outcome measurement bias on top of it.
Rare diseases and rare populations are a different case: a randomized comparison may not be feasible, and a comparison against natural history can still persuade when the untreated course is well characterized and predictable and the drug’s effect is large (U.S. Food and Drug Administration 2023). Rarity does not remove confounding. Treated patients can differ from natural-history patients in disease severity, prognosis, comorbidities, and prior treatment, and a historical comparison also absorbs changes in diagnosis and standard of care over time. Pediatric oncology and rare hereditary diseases, where the natural history is the only available comparison, are where regulatory agencies have been most receptive.
An RCT demonstrating efficacy, supplemented by RWE demonstrating real-world effectiveness, is substantially more compelling than RWE as the primary evidence. FDA’s endorsement of emulation benchmarking studies, which test whether RWE analyses can replicate the findings of completed RCTs in the same setting, follows that logic: where concordance has been established, regulatory confidence in RWE for similar questions increases.
All of this depends on pre-specification. FDA and EMA expect the external control population, the analytic methods, the confounders to be controlled, and the sensitivity analyses to be specified before any unblinding of the internal trial data. An analysis decided after the result is known is much weaker evidence.
The Ibrance precedent, the COVID-19 vaccine experience, and the growing body of emulation benchmarking studies are collectively moving the evidentiary floor, most clearly where RWE extends randomized evidence rather than replacing it. That movement is what the Advancing RWE Program is built to convert into effectiveness labeling changes, the application that has historically resisted RWE hardest.
Real-world evidence does not replace randomized trials. It answers the questions trials cannot: what happens to a broader population over a longer period under the conditions of clinical practice, using endpoints that routine care can measure. The methodological and regulatory infrastructure for generating credible RWE is now substantial, and its role already runs from informing trial design through supporting regulatory submissions to satisfying post-market commitments and HTA requirements. It will keep expanding as that infrastructure matures.
A sponsor is preparing a supplemental NDA for a targeted therapy in a rare solid tumor (approximately 800 annual U.S. cases) and plans to use Flatiron Health oncology EHR data as the primary external control. FDA’s fit-for-purpose framework requires evaluation of data relevance and reliability. Walk through the specific questions FDA would ask about this data source for the proposed regulatory question, and identify the two most likely reasons the submission could be rejected on data-quality grounds.
A retrospective study comparing a newer oral anticoagulant to warfarin uses a prevalent-user cohort: patients enter at their first database capture, by which time they have been on treatment for a median of 14 months, but their follow-up is counted from treatment initiation. Explain why prevalent user bias and immortal time bias are present in this design, what determines the direction in which each would bias the effect estimate, and what design changes would correct both.
An analyst emulating a hypothetical RCT comparing two antihypertensive drug classes using claims data sets the index date as the date of the first dispensing of the study drug. A reviewer notes that some patients have lab values recorded between the first prescription date and the first dispensing date, which the analyst included as baseline covariates. Does this necessarily violate the target trial emulation framework’s requirement for correct time-zero alignment? What would you need to know about when treatment actually begins, and what bias would follow if some patients had started treatment before their first recorded dispensing?
A hybrid Bayesian design for a rare hematologic malignancy with a binary primary endpoint uses a MAP prior derived from three historical trials, with a historical mean control-arm event rate of 30% and an effective sample size of 25 patients. The concurrent control arm enrolls 30 patients. At the interim analysis, the observed control-arm event rate is 42%, 40% higher than the historical mean. Describe what happens to the posterior estimate of the control arm event rate, how commensurate versus fixed power priors would respond differently to this discordance, and what you would recommend the DSMB do with this information.
NICE is evaluating a KRAS G12C inhibitor for second-line non-small cell lung cancer under a managed entry agreement. The sponsor must design a real-world evidence study to confirm effectiveness in routine NHS practice within 24 months of conditional reimbursement. What study design would you propose (specifying data source, population, comparator, outcome, and analytic method), and what are the two greatest threats to the study’s ability to produce a result that NICE will accept as confirmatory?