Comparative effectiveness and personalized medicine research using real-world data

Thomas Debray, PhD

Background

IMI GetReal: Incorporating real-life clinical data into drug development

GetReal aimed to show how robust new methods of real-world evidence collection and synthesis could be adopted earlier in pharmaceutical research and development and the healthcare decision-making process.
Together with Bern, we investigated statistical methods for network meta-analysis

ISCB Utrecht

Introduction

Randomization evenly distributes measured and unmeasured factors among intervention groups and thereby

Facilitates identification of causal relationships
Avoids bias and confounding
Offers a framework for assessing efficacy

The results of randomized, controlled trials are considered to be evidence of the highest grade

The Hierarchy of Evidence Pyramid

Limitations of randomized trials

Experimentation may be unnecessary
Experimentation may be inappropriate
Experimentation may be impossible
Experimentation may be inadequate

The efficacy-effectiveness gap

Variability in drug response may arise due to

Heterogeneity in case-mix (e.g., comorbidities)
Heterogeneity in concomitant medications
Heterogeneity in treatment adherence
Heterogeneity in prescription behaviour

The efficacy-effectiveness gap

Information from RCTs can only answer a minuscule fraction of the near-infinite number of questions about subpopulations, interactions, treatment settings, effects, etc., that are relevant to patients and healthcare professionals at the point of care

Real-world data and Real-world evidence

Randomized trials and real-world data

The Achilles heel of experimental studies is sample size and the lack of random selection
The Achilles heel of observational studies is error due to imbalance in (un)observed variables

‘’Real-world data can improve our understanding of health and social care delivery, patient health and experiences, and the effects of interventions on patient and system outcomes in routine settings’’ (NICE)

What is real-world data (RWD)?

Data relating to patient health status and/or the delivery of health care routinely collected from a variety of sources

Routinely collected data relating to a patient’s health status or the delivery of health care from a variety of sources other than traditional clinical trials

Data collected outside the context of a highly controlled clinical trial

What is real-world data (RWD)?

RWD is commonly understood as observational data

Primary use of data (e.g., data collected in an observational study, data collected with digital wearable devices or patients’ or physicians’ surveys)
Secondary use of data (e.g., electronic healthcare records, claims data, registries)
A combination of both

Sources of real-world data

Studies generating real-world data

Real-world data is not new in medical decision making

Phase IV trials
Post-authorisation safety studies (PASS)
Post-authorisation efficacy studies (PAES)
Studies to measure the effectiveness of risk minimisation measures (RMMs)
Studies to support life cycle benefit-risk evaluation

How is real-world data used?

Explore and quantify the unmet medical need within a particular disease area
Derive information on the current standard of care in different jurisdictions
Augment or replace the concurrent control of a randomized clinical trial
Monitor long-term safety and effectiveness
Tailor the usage, schedule and dosage of a treatment to an individual patient’s needs

What is real-world evidence (RWE)?

Evidence generated from the analysis of real-world data

How is RWE used?

RWE studies - what is the catch?

Selection bias
Information bias
Confounding

These biases may also appear in clinical trials, for instance when conducting subgroup analysis, principal stratum analysis, or indirect treatment comparisons.

Recommendations for RWE studies

flowchart TD
  Z[Study planning] --> A(Defining the research question)
  A --> B(Planning study conduct)
  B --> C(Choosing fit-for-purpose data)
  C --> D(Primary data collection)
  C --> E[Study conduct]
  D --> E

flowchart TD
  Z[Study conduct] --> A(Choosing study design and analytical methods)
  A --> B(Minimising risk of bias)
  B --> C(Assessing robustness of study results)
  C --> D(Using proportionate quality assurance processes)
  D --> E[Study reporting]

Analysis of RWE studies

Statistical methods for addressing confounding

Covariate adjustment
Propensity score analysis
Prognostic score analysis

These methods are also the building blocks for studies conducting indirect treatment comparisons or incorporating historical controls.

Analysis of RWE studies

Many other concerns should be explored:

Operational definitions of key study variables
Definition of time windows
Choice of patient eligibility criteria
Missing data and measurement error
Alternative model specifications
Treatment switching or loss to follow up
Non-adherence

What is next?

Regulatory decision making

The future is not about randomized controlled trials (RCTs) vs. real-world evidence (RWE) but about RCTs and RWE — not just for the assessment of safety but also of effectiveness
Patients will need to be followed up for prolonged periods of time to empirically confirm or refute a priori comparative effectiveness assumptions

Requirements

Need for additional sources of (non-RCT) evidence
Need for better methods for evidence synthesis
Need for revised regulatory (and other decision) frameworks

Data platforms

Catalogue of observational data sources for use in medicines regulation

Federated data network of allowing access to the data of EU citizens standardised to a common data model.

Individualizing treatment decisions

Causal treatment effects are estimated at the population level in randomised controlled trials (RCTs)
However, clinical decision is often to be made at the individual level in practice
Individualized absolute treatment effects provide a natural starting point to engage in shared decision making

Estimating individualized treatment effects

Subgroup analysis is often used to investigate heterogeneity in treatment effects
Although subgroup effects can provide interesting insights and generate new hypotheses, they may be difficult to use for guiding treatment
It may be more informative to move to the absolute risk scale and estimate multivariable models that adjust for individual patient characteristics

Estimating individualized treatment effects

Small RCTs
- Hard to improve beyond risk-magnification
- However, the price to pay to allow for treatment-covariate interactions was small when using both shrinkage and selection, especially for the hierarchical group lasso
Large RCTs
- Shrinkage and selection still needed
- Allowing for all interactions was beneficial

Estimating individualized treatment effects

precmed: Precision Medicine

A doubly robust precision medicine approach to fit, cross-validate and visualize prediction models for the conditional average treatment effect. Available from GitHub and CRAN.

Comparative effectiveness and personalized medicine using real-world data

Edited by TPA Debray, T-L Nguyen and RW Platt
21 chapters; publication expected in 2024
Contributions from ~ 40 authors (academia; industry; contract resesarch organizations; health technology assessment agencies; regulatory decision-making)
Example code on GitHub