fbpx

Latest Posts

Home

Facebook

Twitter

Search
About

Kevin Dayaratna: Failures of an Influential COVID-19 Model Used to Justify Lockdowns

This article was written by Kevin Dayaratna, a statistician at the Heritage Foundation.

Professor Neil Ferguson, who led the COVID-19 modeling team at Imperial College in London, resigned May 5 from his government advisory role after breaking the very same British lockdown rules that he had a role in influencing.

Ferguson led the Imperial College team that designed the computer model that, among others, had been used to justify the recent stay-at-home orders in England as well as in the United States. We now know the model was so highly flawed it never should have been relied upon for policy decisions to begin with.

Epidemiology—the study of the incidence, prevalence, and impact of disease—frequently calls upon models to forecast potential outcomes of diseases. Not surprisingly, once COVID-19 became a pandemic, policy experts from all across the world began relying on such models.

The Imperial College researchers ran one such model they had used in prior research and forecast a number of potential outcomes, including that, by October, more than 500,000 people in Great Britain and 2 million people in the U.S. would die as a result of COVID-19.

The model also predicted the United States could incur up to 1 million deaths even with “enhanced social distancing” guidelines, including “shielding the elderly.” Imperial’s modeling results influenced British Prime Minister Boris Johnson to impose a nationwide lockdown and influenced the White House as well.

I asked Ferguson and his colleagues for their model on multiple occasions to see how they got their numbers, but they never replied to my emails.

According to Nature, they had been “working with Microsoft to tidy up the code and make it available.” I also asked the U.S. Centers for Disease Control and Prevention for the codes it used to develop its COVID-19 forecasts, but got no response.

So, my colleague Norbert Michel and I decided to take a publicly available COVID-19 epidemiological model and forecast the prevalence and mortality of the disease under a variety of plausible scenarios.

The results varied, depending on the assumptions we made about mortality rates within hospital intensive care units, asymptomatic rates, and the specification of the R0 (pronounced R-naught) value, which measures how easily the virus spreads.

We found mortality rate predictions can be quite variable depending on the age and comorbidities of those contracting the virus.

Under varying assumptions regarding a mortality rate in intensive care units between 5% and 30%, we found that predicted mortality because of the disease could range from near 78,000 deaths to as many as 810,000 deaths in the U.S. by Aug. 1.

Recent testing data indicates that the asymptomatic rate for COVID-19 is likely not trivial, and data from Iceland indicates this rate can be as high as 50%. Assuming an asymptomatic rate ranging from 15% to 55%, one can project deaths in the U.S. of between 118,000 and 394,000 by Aug. 1.

Lastly, we looked at the model’s assumption about the virus’ basic reproductive number, the aforementioned R0 value. Popularized in the 2011 movie “Contagion,” the R0 value quantifies the average number of people who will get the virus from someone who is an infected.

Under assumptions of the R0 value ranging from 1.5 to 3.5—plausible estimates based on medical research as discussed in our paper—the model predicted from 44,000 dead to 1.1 million dead by Aug. 1 in the U.S.

According to the Johns Hopkins University coronavirus tracker, we are currently over 83,000 deaths, which exceeds our lower-end estimates. But the point our research made is that these types of models produce many plausible scenarios, depending on reasonable assumptions.

As we learn more about the new coronavirus, it is imperative to continue to update the assumptions used in these models.

After we had published our work, news surfaced that Microsoft had actually made some headway in making the Imperial College team’s model available. But the codes it released are a highly modified version of what the Imperial team actually used.

And, it turns out, the model has serious flaws, which a former software engineer from Google discusses at length in his blog.

The Imperial College code provides different answers using the same inputs. In particular, the same assumptions can provide results that differ by 80,000 deaths over a span of 80 days. The software engineer has noted there are apparently myriad other problems as well—including undocumented codes and numerous bugs.

This isn’t the first time bad models have made their way into policy. As we discussed in our work, statistical models can be useful tools for guiding policy, but they are only as credible as the assumptions on which they are based.

It is fundamentally important for models used in policy to be made publicly available, have assumptions clearly stated, and have their robustness to changes to these assumptions tested. Models also need to be updated as time goes on in line with the best available evidence.

Bottom line: The Imperial College model didn’t meet any of these criteria. And sadly, its model was one of the inputs relied on as the basis for locking down two countries.

The codes we used at The Heritage Foundation are available here. Our assumptions are clearly stated in our paper here.

Comments

Subscribe

Categories

Upcoming Events

  • Sick Seniors: Minnesota’s Failure to Protect the Elderly from COVID-19

    Location: Online

    Wednesday, July 22nd at 12:00 pm  High COVID-19 death rates are plaguing congregate care facilities across the nation. In some cases, COVID-19 positive patients are even being sent into congregate care homes to free up ICU beds, putting more seniors at risk. In Minnesota, where 80% of COVID-19 deaths are from long term care facilities (the highest percentage in the nation), Governor Walz’s sweeping statewide shutdowns failed to protect Minnesota’s vulnerable elderly population. Join Minnesota State Senator Karin Housley and Avik Roy, president of The Foundation for Research on Equal Opportunity, in a discussion about COVID-19 policies that are putting the elderly in…

    Register Now
  • Morning in Minnesota: St. Cloud

    Location: St. Cloud

    Sign up HERE! Courtyard by Marriott St. Cloud 404 West Saint Germain Street St. Cloud, MN, 56301 Please join Center of the American Experiment on Tuesday, July 21 for breakfast with Center policy fellow and education expert Catrin Wigfall as she explains K-12 education in the state and its persistent disparities despite decades of increased spending. Following her presentation, Catrin will lead a Q&A session. 7:30 AM Check In and Breakfast 8:00 AM Presentation 9:00 AM Conclude   Catrin Wigfall is a Policy Fellow at Center of the American Experiment. She is also the director of EducatedTeachersMN and EmployeeFreedomMN. Catrin’s…

    Register Now
  • Morning in Minnesota: Marshall

    Location: Marshall Golf Club

      Sign up for this event HERE! Please join Center of the American Experiment on Thursday, July 16 at Marshall Golf Club for a breakfast with Center economist, John Phelan, as he discusses Minnesota’s economic future. Following his presentation, John will lead a Q&A session. 7:30 AM Check In and Breakfast 8:00 AM Presentation 9:00 AM Conclude John Phelan is a graduate of Birkbeck College, University of London, where he earned a BSc in Economics, and of the London School of Economics where he earned an MSc. He worked in finance for ten years before becoming a professional economist. He…

    Register Now