Minnesota’s Covid-19 model is coming back. Will it be any more accurate this time around?

Back in June, the Star Tribune reported that:

Predictive modeling that guided Minnesota’s initial response to COVID-19 is being “recalibrated” with the latest pandemic data and will offer new forecasts about the spread of the infectious disease.

…

Updated COVID-19 forecasts by researchers at the University of Minnesota and Minnesota Department of Health should be released by mid-July.

These updated forecasts never materialized. I assumed that the model had been abandoned because they have, up to this point, been a total disaster.

VERSION 1 – 74,000 DEAD IN 16 MONTHS

Version 1 of the model, built over a weekend by the University of Minnesota School of Public Health and Department of Health and debuted on March 25th, forecast that, without mitigation, Covid-19 could kill upwards of 74,000 Minnesotans and the state’s 235 intensive care beds (ICU) would be full within six weeks. On this basis, Gov. Walz issued the stay-at-home order (SHO).

VERSION 2 – 3,700 IN ICU BY JULY 13TH

April 8th saw the release of Version 2 of the model. As I wrote in June:

This version modeled a number of scenarios…Gov. Walz chose Scenario 4 which extended the SHO for all by four weeks. Under this scenario, demand for ICU beds also peaked at 3,700 but on July 13th, and again, 22,000 Minnesotans were forecast to die of Covid-19 over the next 16 months.

VERSION 3 – 1,441 DEAD BY MAY 31ST, 3,397 IN ICU BY JUNE 29TH

On May 13th, the state government unveiled Version 3 its model. This forecast that with the SHO in effect to May 18th – the measures then in place – Minnesota would see 1,441 Covid-19 deaths by the end of May.

That seemed pessimistic to me. Up to that point, Minnesota had suffered 638 deaths from Covid-19 (subsequently revised down to 637) so to reach 1,441 by May 31st the state would need to see 45 deaths a day. Then, the record high of fatalities in one day was 30 (May 6th) and the average for the previous 18 days had been 22. In other words, the model which was driving state government policy forecast that the average daily death rate for the last 18 days of May would be double the average daily death rate for the previous 18 days. For the sake of curiosity, I made my own forecast that day. It was very basic: I simply took the current total number of deaths (638) and assumed that it would rise by 22 (the average of the previous 18 days) for the remaining 18 days of May. My forecast for total Covid-19 deaths in Minnesota by May 31st was 1,032.

The actual total was 1,039. Instead of averaging 45 deaths daily, Minnesota averaged just 22. Indeed, on no date did Minnesota see the 45 Covid-19 deaths a day the state’s model forecast. The state forecast was off by 401, or 28% lower than actually turned out. My back-of-an-envelope calculation, by contrast, was out by just eight, as Figure 1 shows. Indeed, as Figure 2 shows, nearly a month on, we are still below the number of deaths the model forecast for May 31st.

Figure 1: Minnesota Covid-19 deaths: forecasts and outturn

Source: Center of the American Experiment

Figure 2: Minnesota Covid-19 deaths: forecasts and outturn

Source: Center of the American Experiment

…

Version 3 of the model also forecast, with the measures in place, a peak ICU use of 3,397 beds on June 29th. At that time Version 3 was released, there were 199 people in ICU with Covid-19. As of Thursday, there were 162, down from a peak of 263 on May 30th. If we assume exponential growth, then, as Figure 3 shows, we should have 2,668 Covid-19 patients in ICU at present. It looks like the model will be at least as badly wrong about ICU usage as it was about mortality, and it was on the basis of these forecasts that Gov. Walz shut down the state and kept it shut down.

Figure 3: Covid-19 ICU hospitalizations, forecast and actual

Source: Minnesota Department of Health

VERSION 4 – ?

The nightmare scenario offered by Version 1 was a forecast of outcomes without mitigation. However unrealistic they may seem eight months on, it would be unfair to deem it a failure because mitigating steps were taken.

The same cannot be said for the predictions of Versions 2 and 3. In both cases, they made forecasts for outcomes if we took the measures we actually did. In each case, these forecasts were wildly pessimistic. Indeed, in May I outperformed the model simply by drawing a straight line.

But reports of its death may have been exaggerated. Last week, KSTP’s Tom Hauser tweeted:

Let us hope it isn’t as big a waste of time as its previous incarnations.

John Phelan is an economist at the Center of the American Experiment.