Does Remdesivir Help or Not?
A drug called remdesivir has been approved for the treatment of hospitalized patients with COVID-19 by the Food and Drug Administration (FDA). The Infectious Disease Society of America (IDSA) recommends it for the treatment of severe COVID-19 infection.
Yet despite these approvals and recommendations, the World Health Organization (WHO) recommended against the use of remdesivir in late November, stating that there is insufficient evidence to support its effectiveness in the treatment of COVID-19. What is going on here? Are these august bodies of medical and scientific experts looking at the same studies?
The disagreement about whether to use remdesivir to treat COVID-19 highlights some of the principles of conducting clinical trials of medications that are important to understand. The WHO, IDSA, and FDA are in fact all reading the same papers about studies that had been completed as of this writing last month. In doing so, however, they come to very different conclusions, conclusions that could affect the lives of many people who are hospitalized with COVID-19. We think the difference of opinion hinges on the choice of what is called the “main outcome measure” for a clinical trial.
How Does Remdesivir Work?
We’ll explain what difference the choice of the main outcome measure can make, but first a bit about what remdesivir is and does. Some say it is a drug that has been looking for a disease because it has been tested on patients with several different viral illnesses, including Ebola (for which it didn’t work). Remdesivir is marketed as Veklury by drug manufacturer Gilead and is an antiviral drug that can only be given intravenously. To understand how it works, we will first supply a short review of the virus that causes COVID-19, called SARS-CoV-2
Cells in humans and mammals all contain the genetic molecule DNA, which holds all the instructions necessary to make the proteins that do the work in our bodies. The first step in that protein-making process is for the DNA to be “transcribed” into a molecule of RNA. RNA then binds to a protein called a ribosome and enzymes use its instructions to build proteins. SARS-CoV-2 and other coronaviruses skip the DNA stage and contain only a single strand of RNA, housed within a viral envelope. When the virus infects a human cell, say in the lung or gastrointestinal tract, it releases its RNA into the human cell, where it hijacks the human cells’ ribosomes to make its own enzymes and proteins. After making many copies of itself, the viral particles burst out of the human cell, killing it, and moving through the blood to infect other cells.
But in order to make copies of itself, the viral RNA must make copies of the RNA itself and it does this by assembling a series of molecules called nucleosides it steals from the human cell. This is where remdesivir comes in; it looks like one of those nucleosides and the enzyme that makes RNA copies mistakenly inserts it into the newly forming RNA strands. Because remdesivir is really a “fake” nucleoside, the enzyme’s activity grinds to a halt, resulting in defective viral RNA molecules that are incapable of making viral proteins or assembling new viral particles. In this way remdesivir is able to stop viral infection, at least in the laboratory.
Sounds like it should work in humans too, but there are some cautions. First, SARS-CoV-2 also has an enzyme that “proofreads” newly forming RNA strands and can clip out mistakes, like the insertion of a remdesivir molecule into the sequence instead of a proper nucleoside. Second, there are billions of viral particles fighting against billions of remdesivir molecules and it is possible that the virus will win out and cause severe infections even in people who get the drug.
What the Clinical Trials Show
That’s where clinical trials come in. Even though a drug might work as predicted in a laboratory test tube or even in a laboratory animal, there is no guarantee it will work in the much more complex human body. So scientists need to do experiments to test a drug’s effectiveness and safety. Some parts of these clinical trials are fairly standard; the most rigorous ones compare the active drug under investigation either to a placebo (sugar pill) or another active drug under “double-blind” conditions, meaning that neither the patients nor the investigators know who is on the study drug and who is getting the comparator. That reduces the chance for biased assessments of whether the drug works. Patients are randomized to get either investigational drug or comparator, so at least theoretically the only thing that is different between the two groups is whether they get drug or placebo. This kind of study is called a randomized clinical trial (or RCT). Under some conditions, such as are presented by a raging pandemic, compromises are often made in this most rigorous study design, such as not maintaining a double blind or even not having a comparator.
Some aspects of even the most rigorous RCT’s are left up to the investigators to decide about and one of these is what to declare before the study starts will be the main outcome measure. It is required that a main outcome measure—the most important outcome the investigators will use to evaluate whether the study drug worked—be set before starting the trial. There are two reasons for this. The first is statistical. There is an old saying that if you torture data sufficiently it will confess to anything. In this case, by simple chance if you keep testing data for whether drug worked better than placebo across multiple outcome measures–what scientists often call a “fishing expedition” –eventually one will turn up to be statistically significant. But that in itself is just a chance finding and of spurious value. The second is more obvious—if you wait until the trial is over and just pick the outcome variable that makes the trial look successful you may have avoided reporting many more that didn’t show any difference between drug and placebo. Scientists call this “cherry-picking the data.” It’s not allowed.
In the case of testing if remdesivir works for COVID-19, then, scientists have many choices for a main outcome measure. It could be how quickly patients recover, mortality rates, number of patients who require life support, number of symptoms of the infection, and so forth.
In the studies that FDA and the IDSA looked at, time to recovery in days was chosen as the main outcome measure by investigators conducting the trials, whereas in the studies WHO evaluated, mortality was the main outcome measure. It turns out that studies have shown a modest but statistically and clinically significant reduction in the number of days it takes for a hospitalized patient to recover from COVID-19 if the patient is given remdesivir compared to placebo. On the other hand, studies have not shown that remdesivir decreases the number of hospitalized patients who die from COVID-19.
For example, in one pivotal study called ACCT-1, 1062 hospitalized COVID-19 patients were randomized to receive either remdesivir or placebo. The patients who received remdesivir recovered in 10 days, compared to 15 days in the patients who received placebo. Giving remdesivir earlier in the course of illness was more effective than giving it after ten days, but the benefit of remdesivir persisted even when given later in the course.
The experts at the WHO looked at four studies that evaluated remdesivir, each with different designs. One of these studies used as its main outcome measure clinical status after receiving a ten-day course of remdesivir, with death as the worst possible clinical status. This study found no significant difference between remdesivir and usual care (that is, patients were provided regular COVID-19 care in both groups, but only one group got remdesivir). There was no difference between groups in the rate of COVID-19 death.
Two physicians wrote an op ed piece in the New York Times in November titled “Does remdesivir actually work against Covid-19?” They noted that Veklury (remdesivir’s brand name) is very expensive, doubted the evidence is strong enough for approval, and worried that its approval will discourage patients with COVID-19 from participating in trials of other potential treatments that might be more effective than remdesivir.
Those are all good points. As you can see, however, there is clearly room for scientific debate on whether remdesivir works well enough to justify FDA approval. The debate seems to hinge on a reasonable difference of opinion about what main outcome measure to consider most important. Is it sufficient that the drug speeds recovery even if it doesn’t reduce mortality risk? Clearly some experts think it is while others do not.
We highlight this issue in order to discuss the underlying concerns that inevitably arise whenever scientists disagree. In the case of remdesivir, and many other similar situations, the issue is not whether some experts are right and others wrong but rather that there are sometimes different ways to design studies and to interpret the results. The ultimate decision about whether to administer remdesivir to a hospitalized patient with COVID-19 will be made between individual patients and their physicians. Our prediction is that many in the U.S. will decide to take it based on the recovery data and that would not be a “wrong” decision. We also predict that remdesivir will be used less often in other countries where FDA approval is not relevant and WHO opinion more germane. That also will not be a “wrong” path to take. These are, of course, just guesses and only time will tell how much remdesivir is actually prescribed.
In most cases in which such disagreements exist, scientists call for more studies and more data. It is possible that such work will help with decision-making in the present case with remdesivir, but these are not usual times and as the pandemic rages on it is not easy to assemble and conduct rigorous RCTs on various proposed interventions. Here, the art of medicine becomes of paramount importance and having each physician-patient pair makes its own judgments about relative risks and benefits becomes the most important route to decision-making.
What would you do if you were hospitalized with COVID-19 and offered remdesivir?