Real-World Evidence is Changing Medicine - Part 1

Journalists Need to Understand Uses and Shortcomings. As the FDA allows more big data for drug and device approvals, top researchers share studies that show its potential — and flaws. One drug-safety expert warns against abandoning the randomized controlled trial.
by Chris Adams, National Press Foundation

Real-world data can be used to replicate results from randomized controlled trials. While randomized controlled trials — RCTs — have long been considered the gold standard of medical evidence, researchers are now working to see if other pools of data can be used to answer the same questions. Dr. Sebastian Schneeweiss of the Harvard Medical School has or will conduct 30 studies in the “RCT DUPLICATE” initiative, funded by the U.S. Food and Drug Administration (first results here). The science behind the studies can get complicated, but at the core they use patient-level claims data from insurers to reach the same causal conclusions as an RCT. In an RCT, Patient Group A will get the drug under study and Patient Group B will get either a different drug or a placebo. Both groups are then observed, and researchers will eventually know with a high level of statistical satisfaction whether the new drug performed better than the alternative. In Schneeweiss’ studies [see video], insurance claims for existing patients show which ones are on which drugs. That allows researchers to design a cohort study by building a Patient Group A and a Patient Group B. These studies work for drugs that are already on the market but that researchers want to study for secondary reasons — say, a specific gender, or a certain age group. Schneeweiss has finished 20 of his 30 DUPLICATE studies, with generally positive results. “It is very encouraging news,” he said. “… Real world evidence may offer causal insights when RCT data are either not available or cannot be quickly and feasibly generated.”

The randomized controlled trial offers advantages that are hard to replicate. Dr. Steven Nissen, a prominent cardiologist and drug-safety expert from the Cleveland Clinic, cautioned against the move away from randomized controlled trials, argues that real-world evidence isn’t ready for prime time. “Real world studies should not be viewed as conclusive evidence,” he said. A clinical trial is a controlled experiment designed to isolate the new treatment by minimizing variability and bias. The problem, Nissen said, is that clinical trials are expensive. He sees the push for real-world studies as a cost-saving gambit but no substitute for RCTs. Hormone replacement therapy was taken by millions of women based on the strength of real-world studies, he noted, but later RCTs conducted in the early 2000s undercut the supposed benefits of the therapy. “Everybody believed based upon the real-world evidence that it would go the other way,” Nissen said. “And the evidence from observational studies was wrong.” He concluded that while real-world studies can be helpful in limited situations, “the randomized controlled trial is the gold standard for a very good reason.”

Real-world evidence can be used to supplement randomized controlled trials. Dr. Michele Jonsson Funk of the University of North Carolina at Chapel Hill detailed nine shortcomings of RCTs, including that they are too specific to answer broader questions, too small to study rare outcomes and too short to study long-term effects. Real-world data are helpful in addressing some of those flaws, such as the inability to study rare outcomes or subpopulations.

Speakers:
Dr. Sebastian Schneeweiss, Professor of Medicine, Harvard Medical School; Professor in Epidemiology, Harvard T.H. Chan School of Public Health
Dr. Michele Jonsson Funk, Associate Professor, Department of Epidemiology; Director, Center for Pharmacoepidemiology, UNC Gillings School of Global Public Health
Dr. Steven Nissen, Chief Academic Officer, Heart and Vascular Institute; Chair in Cardiovascular Medicine and Professor of Medicine; Cleveland Clinic Lerner School of Medicine

This program is sponsored by the Professional Society for Health Economics and Outcomes Research (ISPOR), with support from the BMS-Pfizer Alliance. NPF is solely responsible for the content.