Real-World Evidence: How Big Data is Changing Scientific Standards

How Should the FDA – and Journalists – Decide What Scientific Evidence Is Good Enough? Randomized clinical trials have long been the gold standard for drug and device approvals. Now Big Data powers “real-world evidence” that plays a growing role in medical decisions and FDA approvals. Former FDA head Mark McClellan explains the data, the evidence and how both have evolved.
by Chris Adams, National Press Foundation

In the age of Big Data, it’s increasingly possible to test drugs, devices and other treatments outside the traditional randomized controlled trial. Dr. Mark McClellan, who oversaw drug and device approvals as commissioner of the U.S. Food and Drug Administration and also led the Centers for Medicare and Medicaid Services, said that “real-world data” and “real-world evidence” sometimes offer advantages over randomized controlled trials, which have long been the gold standard in medical assessment. Two key advantages are cost and diversity. Randomized controlled trials are powerful and important, but also expensive. And it’s often difficult to test a drug or device on the subpopulations researchers would like to study. As McClellan noted: Clinical trials don’t generate evidence on all treatments and outcomes that matter for patients, providers and payers. “There are these more and more diverse kinds of questions that we’d like to answer about combinations of treatments, about comparing one treatment to another, about long-term outcomes that matter to patients that may be hard to capture in a traditional clinical trial,” he added. “… Real-world evidence can potentially really help fill evidence gaps.”

Spurred by Congress, the FDA is now drafting guidance for using real-world data and evidence. The Prescription Drug User Fee Act, which dictates much of the FDA’s drug-approval process, required the agency to enhance the use of real-world evidence for use in regulatory decision-making. The 21st Century Cures Act, signed in 2016, did the same. The FDA is implementing both congressional mandates and is scheduled to publish guidance by the end of 2021. The Duke-Margolis Center for Health Policy at Duke University, which McClellan directs, has been working with the FDA and other researchers to test the concepts and realities of real-world evidence. Their research is here. Big questions: “What is the clinical context? What kind of regulatory decision? What kind of policy decision? And what are the clinical features?” McClellan said, adding. “Just because we have a lot of data doesn’t mean we understand what those data mean. Healthcare data are notoriously complex.”

Just because data are available doesn’t mean they’re usable. Real-world data and real-world evidence mean information that comes from anything other than randomized controlled trials: data from patients’ electronic health records, insurance claims, wearables such as smartwatches, or from cancer and other types of disease registries. Some attempts to use such data have ended in disaster, leading to embarrassing retractions for prestigious medical journals. “Just because we’ve got a lot of data doesn’t mean we have a lot of evidence,” McClellan said. “There are very important methodologic considerations — both related to data and how the data are used — that are important to work out to have confidence in the results.”

The public debate over real-world evidence is relatively new, but the concepts has been around for decades. Over the years, real-world evidence has been used primarily for studies on a drug’s safety after it has been approved; the side-effects data reported once a drug is on the market provide fodder to study millions of people, not hundreds or thousands. But over the past 70 years, there have only been 34 cases when real-world evidence was used to approve new drug uses. Most were for drugs for rare diseases, cancers and other hard-to-test conditions. But such use has accelerated in recent years.

Real-world evidence that shows big effects is easy to use. Marginal effects will pose the toughest challenges. Testing a drug on a small subpopulation and finding an improvement over another drug of 30% or 40% or 50% makes for an easy decision, McClellan said. “You can be pretty confident that that wasn’t just occurring by chance,” he said. “… Where it’s been a little bit more challenging is when the effect sizes aren’t that large and there’s really not a good alternative because … there’s not that many patients with a rare indication.”

Dr. Mark McClellan, Robert J. Margolis Professor of Business, Medicine, and Policy, Duke University; Founding Director, Duke-Margolis Center for Health Policy, Duke University

This program is sponsored by the Professional Society for Health Economics and Outcomes Research (ISPOR), with support from the BMS-Pfizer Alliance. NPF is solely responsible for the content.