North Carolina's top doctor expresses concern about reliability of COVID-19 forecast models

As forecast models for the 2020 hurricane season are evolving, so is predictive modeling for the other threat facing North Carolina -- COVID-19.

We discovered two medical doctors who have been dispensing dependable guidance through the pandemic are expressing reservations about the reliability of virus modeling.

"Well, I think we have to take them with a chunk of salt, not a grain of salt," said Dr. Jennifer Ashton, chief medical correspondent at ABC News.

We went to Ashton for a reality check on COVID-19 models.

She told us it is similar to forecasting the impact of a hurricane on North Carolina when it is thousands of miles away.

"These models are very similar to predicting the path of a hurricane, you know when the human body is involved, when physiology is involved, when a virus is involved," said Ashton.

We looked at the University of Washington Institute for Health Metrics and Evaluation model.

On May 12, researchers released a worrisome prediction suggesting more than 4,400 people in North Carolina would die from COVID-19 by early August.

At the time, the death toll in North Carolina was 577.

Two days later, the Secretary of the North Carolina Department of Health and Human Services, Dr. Mandy Cohen, explained why we should not be looking at the dire prediction in that model and others.

"Any one of these models has its own limitations and that model in particular, the way they model is based on actions that were taken in Wuhan, China," said Cohen. "Obviously that is a very densely populated urban area, and so translating that to North Carolina is hard."

Cohen also expressed other concerns about the reliability of virus models.

"I believe none of these models truly take into account all of these nuances related to not just the geography and the nature of density in North Carolina, but also the nature of how we are doing the easing and putting on of restrictions."

The IHME tells ABC11 it has now expanded the information they use to incorporate locations and other data beyond China.

"While initially when we started modelling COVID several months ago, Wuhan was the only source of information about the shape of the curve of daily deaths, we have learned much more since from many other locations including detailed information on daily deaths, hospitalizations and positive tests from a much longer time series and predictors that determine the shape of the curve," said Dr. Theo Vos, professor of Health Metrics Sciences at the Institute for Health Metrics and Evaluation in a statement to ABC11. "In simpler terms: our estimates for North Carolina are no longer being informed by the shape of the Wuhan curve."

Dr. Ashton cautions against looking at virus prediction models the same way we look at a weather forecast.

"No one has a crystal ball here and we've never been in this situation before," said Ashton.