COVID-19 models vary widely. What that means for leaders under pressure.
Where can public officials turn for trusted advice on COVID-19 when models of the pandemic’s arc have offered such divergent projections of death rates and hospital shortages?
Ideally, scientific advisers can help officials understand the underlying assumptions that modelers have used to piece together fragmentary data. But often those assumptions get lost in the fray, creating an aura of certainty – and dread – when projections are splashed across the news with grave images.
Why We Wrote This
Determining a rational course of action can be challenging when fear abounds. Understanding the underlying assumptions that have led to dramatically different projections of COVID-19 infection and fatality rates can help.
“I think we could be doing a lot of harm,” says John Ioannidis, an epidemiologist and statistician at Stanford University, who adds that that stress affects susceptibility to viral and respiratory infections. Others defend sobering models as essential to helping leaders and health care systems brace for a surge in demand and saving lives.
So far, there isn’t enough data to provide definitive transmission or fatality rates. Wide-ranging variations in the adoption of and adherence to social distancing further complicate the picture.
There is another variable that may well tip in our favor, says Herman B. “Dutch” Leonard, a professor of public management at Harvard Kennedy School. That’s America’s ability to innovate its way through crisis. “We are just beginning to do our best thinking,” he says.
As mayors, governors, and presidents weigh how best to guide their communities through the coronavirus crisis, they must navigate dramatically divergent models of the arc of the pandemic.
In the United States alone, leaders have already made decisions that have disrupted millions of citizens’ lives and cost trillions of dollars, with a disproportionate economic impact on low-wage earners and their families. As citizens across the country wonder whether they and their loved ones will be safe, how they’ll pay their bills, and when they'll be able to return to work, some are pressing their leaders to take stricter measures while others are questioning the models that prompted such unprecedented steps.
The modeling debate within the scientific community is due in part to uncertainty around key questions, including transmission and fatality rates. And the challenge is further complicated by wide-ranging variations in adoption of and adherence to preventative measures like social distancing and voluntary home quarantine.
Why We Wrote This
Determining a rational course of action can be challenging when fear abounds. Understanding the underlying assumptions that have led to dramatically different projections of COVID-19 infection and fatality rates can help.
Editor’s note: As a public service, all our coronavirus coverage is free. No paywall.
“I think we should use all data to inform our views, but we shouldn’t be overconfident in the results from any one data set or even any combination of data sets,” says Marc Lipsitch, a professor of epidemiology and director of the Center for Communicable Disease Dynamics at the Harvard T.H. Chan School of Public Health. “I think we’re still at the stage of, ‘Here are the caveats.’”
But the caveats and underlying assumptions are sometimes lost in the fray. For average citizens, the charts depicting expected spikes in fatality rates or hospital shortages can take on an aura of certainty – and dread.
“No matter what the numbers are, if they are accompanied by pictures of people dying and of graves and trucks carrying the dead ... [people] read the numbers through these narratives, through these stories,” says John Ioannidis, a professor of epidemiology at Stanford University. Such packaging in the media compounds an already stressful environment, he adds, noting that stress affects susceptibility to viral and respiratory infections. “In such a situation, with panic and horror being disseminated, I think we could be doing a lot of harm.”
Why the models diverge so widely
Two of the most prominent COVID-19 models are from Imperial College London and the Institute for Health Metrics and Evaluation (IHME) in Washington state.
Imperial’s Neil Ferguson and his colleagues garnered headlines with their projection that if nothing were done, COVID-19 deaths in the United Kingdom and U.S. over two years would total 510,000 and 2.2 million, respectively, and demand for critical care beds would peak at 30 times actual capacity.
Their March 16 report, which advocated social distancing, is widely credited with prompting Britain and many U.S. states to close schools, churches, and nonessential services. The Trump administration issued guidance the same day recommending that employees work at home and gatherings be limited to 10 or fewer people.
Dr. Ferguson, who is one of the most highly respected figures in his field, came under fire for allegedly reversing course nine days later when he told British Parliament that due to the U.K.’s lockdown and social distancing policies, he estimated the death toll at 20,000 and possibly substantially less.
The 20,000 estimate was in fact included in the original report – but as one of 46 different projected death tolls ranging from 5,600 to 120,000 depending on the rate of transmission and the scope of societal restrictions. (See page 13 in this PDF.)
It’s unclear how effective such restrictions are. A survey of 14 studies on quarantine in past disease outbreaks, for example, found that rates of adherence varied from 0% to 92%.
“I think we need to be very careful with narratives that say ... ‘Only 20,000 died because we did the right thing,’” says Dr. Ioannidis, who has made a career out of finding holes in medical research and cautions against a “bandwagon” effect among researchers.
Last month he published a provocative article in STAT warning of a once-in-a-century “evidence fiasco” around COVID-19, asking how policymakers could be sure they weren’t doing more harm than good by implementing draconian countermeasures based on patchy data. Dr. Lipsitch wrote a rebuttal piece arguing that despite the lack of data, inaction was not an option given the exponential nature of infection.
One of the key unknowns is the infection fatality rate. Under ideal circumstances, fatality rates are a matter of simple arithmetic – divide the number of deaths by the number of infections. But without widespread testing of the general population, it’s difficult to produce a reliable denominator. What’s more, the number of fatalities isn’t entirely certain, either. In Italy, for instance, there have been indications that deaths may have been either overstated or understated, casting doubt on totals elsewhere as well.
The World Health Organization initially estimated the fatality rate for global cases at 3.4%. But more recent estimates have come in lower. A March 13 article in the journal Science, for example, concluded that China’s fatality rate was closer to 0.5%. Imperial’s study put the U.K. rate at 0.9%.
The other model garnering a lot of attention, from IHME at the University of Washington, didn’t attempt to calculate fatality rates. It focused on projecting total deaths and the strain on health care systems, initially using data from Wuhan, China, and then other locations as more data became available.
IHME’s March 25 forecast projected lower death tolls than numerous other models – in part because it assumed that within one week, all U.S. states would have adopted four social distancing measures that China implemented. But two weeks later, 41 states have implemented two or fewer. Nevertheless, IMHE has since reduced its projections to 60,415 U.S. deaths by August, and lowered its estimate of total hospital beds needed at peak demand by nearly half.
Making assumptions clear
So how are public officials supposed to make sense of the debate within the scientific community – and fast?
Ideally, their scientific advisers can help sort through the relevant studies or models, including the fine print, says John Holdren, former White House scientific adviser under President Barack Obama.
“It’s the responsibility of the modelers [individuals and their agencies] to make the assumptions and associated uncertainties clear when they describe their results, and the responsibility of science advisors to policy makers to try to make sure these points are understood,” says Dr. Holdren in an emailed comment.
The Trump administration showed IMHE’s model at a March 31 press conference when it unveiled its own estimate of 100,000 to 240,000 fatalities, also citing studies from Imperial College, Harvard, and others. But the White House gave little visibility into how it arrived at those numbers.
“I’m not quite clear – and I’m a pretty experienced pandemic watcher – about the process of deliberation,” says Howard Markel, a physician and professor of the history of medicine at the University of Michigan who has studied pandemics from the Black Death to the 1918 Spanish flu and was part of a group of experts tapped to evaluate the Obama administration’s H1N1 influenza policies on a daily basis from 2009 to 2011.
Key in crisis: Scientists and politicians working together
One of the critical performance factors in handling a crisis is how successfully the scientific community and political actors interface with each other, says Herman B. “Dutch” Leonard, a professor of public management at Harvard Kennedy School who is working with mayors from around the world through the Bloomberg Harvard City Leadership Initiative.
Politicians are sometimes skeptical of scientists’ projections, he says, while scientists can underestimate the political difficulty of enforcing something like social distancing.
“The scientists often act as if the mayor has a wand and can order the public to stay home, but she’s using up really scarce political capital,” which could undermine her ability to act as the pandemic worsens, says Professor Leonard.
But Dr. Markel says that a public health official changing strategy as new data comes in is analogous to a physician who adjusts his treatment of a patient as symptoms change, and that should be communicated to the public. “It’s the patient-doctor relationship writ large because you’re taking care of a community,” he says.
As leaders strive to give an unvarnished view of the facts without creating undue fear, Professor Leonard says one variable may be higher than people realize: America’s ability to innovate its way through crisis.
“We consistently underpredict the resilience in our political and economic system because we can’t ourselves figure out what the answer is,” he says, even as an automotive factory is figuring out how to produce masks, for example. “We are just beginning to do our best thinking.”
Editor’s note: As a public service, all our coronavirus coverage is free. No paywall.
