“The basic problem is that complex systems such as the atmosphere or the economy can not be reduced to simple mathematical laws and modeled accordingly. The equations in numerical models are therefore only approximations to reality, and are often highly sensitive to external influences and small changes in parameterization”
~ David Orrell
This will be a quick optimistic note on COVID and a pessimistic note about public policy and modeling. In case mainstream media hasn’t highlighted this sufficiently, Omicron is quite aggressively good news. One of the guys I’ve found most helpful in the mess of confusion around COVID-19 is Dr. John Campbell. He’s not extreme, but he is always accurate and often weeks to months ahead of the junk corporate news. Today he put out some Dutch data on Omicron I found very interesting from a belief in or reliance on models framework. If you remember the “bend the curve mantra” in the beginning of COVID-19. Models which drove lockdowns in all countries but Sweden were based on similar quality parameters and mechanics. In some sense so is climate change, but that’s all I’ll say here because I don’t need to anger anyone further today. Today good news only. Below is a simple graph showing the percentage of cases with green being delta and pink being Omicron. They will always be symmetric but the speed and completeness of the Omicron shift is remarkable, and should be ground for celebration.
Variants emerge as a factor of exposure numbers, and I assume they do so in accordance with relatively grounded tradeoff theory in which more spreadable (virulent) viruses out compete more deadly (pathogenic). The mode of transmission matters, but it should hold for human to human airborne transmissions. Of course, like seasonality, it was as if this theory didn’t apply when people talked about COVID-19 began, and frankly, it’s as if it still doesn’t apply. It is a fairly obscure part of evolutionary theory, so who would have paid much attention to it prior thee last few years. In any case, the theory consistent fact for this most important particular case is that Omicron is much less deadly. Recent reports seem to indicate about 100 fold less deadly than Delta.
The next chart is a multi scenario model of various cases with intensive care on the y-axis and time on the x-axis. This was in a report to the Dutch Parliament when the actual data were the black dotted line in mid December.
The second chart is the actual ICU bed occupancy experienced which is presented pretty much daily on the Dutch government website.
The third chart is when someone on Twitter added the actual data to the previous predictions. This yellow line. I want to point out that this is almost never ever done by professional modelers or their political consumers. Perhaps the modelers should be biased toward preparation in their scenario selection, but when the actual results exceed even the error bars of the extreme scenario it tells us something quite clear: the mechanics of the predictive model used are completely bogus.
So that’s it, the main thing I want to share with you was this magical moment when we captured so-called experts presenting rubbish. It happens all the time, but usually no one goes back and shows data from reality that is outside the model's confidence bounds. Mathematically this should not happen, it should undermine the whole enterprise, but it will do nothing. Often policy consultants use tractable models that do not really capture the critical, and often nonlinear, factors about how reality works. Yet they go ahead and present the models anyway which often frames thinking and policy in the linear but wrong outcome space. Perhaps it is better than nothing, but almost no policy maker knows how to correctly investigate the model limitations and often even the technical users do not. Statisticians are often trained to throw out outliers, but outliers are the clue that the world is working outside your frame. Many things we think are predictable are not, because our modelers cannot specify factors of relationships that they do not know, while whole fields do not look for mechanisms that defy their cultural presuppositions. It seems really hard to grasp how little we know when it seems we know so much more than people in the past. We tell ourselves nice stories often confusing authority for evidence. The best case for our future is that we are shown to be deeply in a dark age as they would define it.
If you are interested in the deeper epistemic topic please see much richer discussions by these authors:
The Future of Everything: David Orrell
Superforecasting the Science of Prediction: Philip Tetlock
Chaos: Making a New Science Revised, Gleick, James
Official Data Sources:
https://coronadashboard.government.nl/landelijk/sterfte
COVID Model Literature:
Modeling for COVID-19 has Failed
https://covid19scenariomodelinghub.org/index.html
https://arxiv.org/abs/2004.04734v4
Quotation Source:
https://www.sciencedirect.com/science/article/abs/pii/S0169207009000739?via%3Dihub