A reappraisal of COVID-19 epidemiology after the pandemic: models, data, and interventions

John P.A. Ioannidis

Professor of Medicine, of Epidemiology and Population Health, and (by courtesy) of Biomedical Data Science, and of Statistics and Co-Director of Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, USA; Corresponding Academician

Abstract

This is a summary of a presentation given in October 2023 at the Academy of Sciences in Bologna. COVID-19 was a major crisis that witnessed massive mobilization of the scientific workforce. Almost a million scientific articles were published on COVID-19 within less than 4 years, most of them of low quality. Several key components of the narrative that prevailed in the pandemic response were misleading and they affected both the credibility of science and public health and the health of billions of people worldwide. Rigorous evidence and a judicious approach to re-evaluating decision-making is needed to revert the tide and regain competence within the scientific community and trust among the general public.

Keywords

COVID-19, Epidemiology, Pandemic, Excess deaths, Infection fatality rate, Lockdown, Models.

© John P.A. Ioannidis, 2024 / Doi: 10.30682/annalesps2402b

This is an open access article distributed under the terms of the CC BY 4.0 license

COVID-19 caused a major crisis that resulted in a massive mobilization of the scientific community. Within less than 4 years in 2019-2023, almost 1 million scientific papers on COVID-19 were published. Many of these papers ventured in epidemiology while their authors had no previous substantive expertise in the field. Several prevailing narratives in the COVID-19 response were often shaped in the absence of serious epidemiological reasoning and evidence and were the composite of the impact of influencers, politicians, big tech, people in panic and occasionally even scientists with either conflicts or obvious lack of nuance and/or expertise. Eventually, 98 of the 100 most-cited papers published across the scientific literature in 2020-2021 were focused on COVID-19 [1] – a measure of the inordinate attention received by this crisis. Multiple evaluations of the quality of this scientific literature show major deficits in rigor [2-4]. Panicked interpretation confounded further the deficiencies of this corpus.

In the early days of the pandemic, most science and decision-making depended on models, and the influence of modeling continued to be strong during the entire pandemic. However, models are known to be easy to fail and they should be seen as offering sketches of possibilities rather than strong definitive evidence for decision-making. Broader considerations for failed forecasting during the pandemic included [5] poor data input on key features of the pandemic that went into theory-based forecasting (e.g.; SIR models), poor data input for data-based forecasting (e.g.; time series) with lack of consensus as to what is the “ground truth” even for seemingly hard-core data such as the daily number of deaths, wrong assumptions in the modeling, high sensitivity of estimates, lack of incorporation of epidemiological features, poor past evidence on effects of available interventions, lack of transparency, plain errors, lack of determinacy, looking at only one or a few dimensions of the problem at hand, lack of expertise in crucial disciplines, groupthink and bandwagon effects and selective reporting. Models became far more frequently used in the epidemiological literature during than before the pandemic, but most of them lacked minimum transparency such as data and code sharing [6]. The big-scale failure suggests that in the future pre-registration of predictions (so that they can be compared rigorously against eventual reality) may need to become more widespread [7]. Also, more transparency and pre-registration may help diminish the problem of selective reporting.

An illustrative major system failure demonstrating the impact of selective reporting was the publication of a notorious modeling paper in Nature [8] suggesting major benefits especially and almost exclusively from aggressive lockdown (rather than other, less disruptive interventions) in the first wave of the pandemic. This inference generated impetus for the devastating continuation of these harsh, irrational measures in subsequent waves as well, and lockdown practices with zero COVID theories were endorsed by powerful coalitions of enthusiastic advocates [9]. The same team of elite analysts from Imperial College had also generated a model at the same time that they did not publish in Nature but which would have shown practically no benefits from aggressive lockdown as compared with less aggressive measures; that non-published model had actually much better fit to the data [10] than the lockdown-favoring one that misleadingly found itself in the pages of Nature.

The final frontier of counting the composite impact of a pandemic is considered to be the count of excess deaths [11]. While there is certainly value to this approach, there can be large differences in the calculation of excess deaths depending on the assumptions used. For example, for Germany during 2020-2021 different models with different assumptions have shown values ranging from 22,000 to 203,000 excess deaths [12]. Values with proper age-adjustment of the changing structure of the population typically yield much lower estimates of excess deaths, but high-profile and widely disseminated excess death estimates failed to make proper adjustments [13]. Moreover, most published evaluations of excess deaths presented only one method for excess death estimation. Multiverse analyses, considering multiple different analytical choices for excess death calculations show that the absolute death count can vary substantially depending on these choices, but the relative ranking of performance of different countries differs far less regardless of which analytical choice and model is used [14]. Among European countries, Sweden did best during 2020-2023 regardless of what model is chosen [15]. With age-adjustment and with comparison against the reference years 2017-2019, Sweden had 3.5% fewer deaths during the pandemic than during the pre-pandemic years (a death deficit). Conversely countries with vulnerabilities, e.g.; a high proportion of the population living in poverty and with large inequalities (typical example being the USA where also many people were marginalized, not covered by health care, and subjected to concurrent epidemics of other lethal conditions, e.g.; overdose and alcoholism and multiple inequities) [16-18], had high excess deaths [15].

Throughout the pandemic there were major misconceptions about the personalized risk of death and severe outcomes after SARS-CoV-2 infection. Early estimates of infection fatality rate were misled by speculative assumptions from modeling, and a prevailing narrative that insisted on very high values of infection fatality rates. The most reliable data from national seroprevalence studies with the best representativeness of the general population show that the actual infection fatality rates were eventually far smaller in the first pandemic year (before vaccination) compared with what was believed and what was disseminated both in scientific and lay circles [19]. This misconception generated major imbalance and distortion in thinking about the benefits and risks of different interventions, in particular non-pharmacological interventions that often had profound negative impacts on education, mental health, the economy, and health care services for diverse diseases – among many other harms [20-24]. Most countries failed to protect the vulnerable populations more than the average person, although exceptions exist where such focused protection did happen (with beneficial results on the impact of the pandemic) [25] – yet, also other examples exist of countries that protected more those segments of the population who were at lower risk and exposed maximally the most vulnerable [25], e.g.; the elderly and those in nursing homes.

Nursing homes deserve special attention because they were the locations of major disasters in many countries, as a result of business systems that depended on for profit structures and that were malfunctional even before the advent of this major crisis [26, 27]. Empirical data suggest that deaths in nursing homes resulted in very limited loss of life span (probably less than 1 year) for those who were deceased [28]. However, the conditions of death in an atmosphere of dread, isolation, and abandonment damaged the very basic principles of human treatment of vulnerable people. Many other marginalized and weak segments of the population suffered disproportionately during the pandemic, while the wealthy bore very little impact of the crisis and often even benefited financially by the crisis.

As the large majority of the global population was already infected by early 2022 and as all the global population (with limited exceptions) had been infected by the beginning of 2023, the fatality rates of previously infected individuals (often infected multiple times) markedly decreased and an added shrinkage apparently happened as the result of widespread vaccination [29]. The relative benefits of vaccination versus previous infections in decreasing infection fatality rates need careful study to disambiguate. However, in the current setting (late 2023) the endemic picture is not one that should justify any aggressive measures – or probably any measures at all beyond the very basic routine standards of hygiene. Evidence has accumulated to-date suggesting that even during the height of the crisis mandates and aggressive measures were counterproductive and resulted also in loss of trust among large segment of the general population [30]. Decision-making (personal and public) needs to be multi-dimensional, considering multiple consequences of our actions (or lack of actions). Randomized trials are still very important to perform whenever there is ambiguity and the use of observational data in no excuse to argue against the conduct of such trials. Trials on masks with their surprising, mostly “negative” results offer a dire lesson [31], although it is not surprising that many will continue to defend the original narratives of effectiveness based on observational data [32].

COVID-19 led to major disruptions in every aspect of health and social life, worsening inequalities and damaging the health of the poor primarily [33]. The eventual blatant failure of overconfident zero-COVID-19 policies offers a major landmark of the triple threat of folly, panic, and conflict in the history of public health disasters. Careful scrutiny is needed to understand why we did so many things wrong and why even serious scientists were misled – or often led the misleading narratives themselves. The exact global footprint of the pandemic is uncertain, because even death registration in not reliable in most countries around the world. Moreover, some of the consequences of pandemic disruption may unfold over a time horizon of many years or even decades (e.g.; the impact on cancelled preventive services, disrupted education, and impoverishment). However, the existing data suggest that deaths due to the measures that we took to deal with the pandemic probably killed and damaged more people than the virus itself [34]. This may be particularly true in the poorest countries, e.g.; in Africa and India, where the age-structure of the population and other factors (perhaps even some cross-immunity from exposure to other pathogens [35, 36]) would have led to minimal deaths from SARS-CoV-2 itself. For more detailed discussion of these issues, see reference [37].

In the future, were we to be hit by a similar calamity, we should think of how we can affect the main factors underlying COVID-19 deaths. These include social injustice, inequalities, racism, poverty; smoking; other modifiable lifestyle risk factors, in particular obesity; poor protection of nursing homes; poor adoption of effective, simple, non-disruptive public health measures; adoption of harmful public health measures (e.g.; blind draconian lockdowns); suboptimal, ineffective or even harmful treatments and medical care; and policies that result in increased vaccine hesitancy and lack of trust in public health.

Instead of panic, use of poor models and surrendering to spurious big data that don’t speak by themselves and to an all-encompassing infodemic, we need to endorse a judicious, “slow” data public health approach where evidence is carefully collected to inform strategic actions and interventions that do more good than harm [38].

References

1. Ioannidis, J.P.A.; Bendavid, E.; Salholz-Hillel, M.; Boyack, K.W.; Baas, J. Massive covidization of research citations and the citation elite. Proc. Natl. Acad. Sci. U S A 2022, 119 (28), e2204074119.

2. Jung, R.G.; Di Santo, P.; Clifford, C.; Prosperi-Porta, G.; Skanes, S.; Hung, A.; Parlow, S.; Visintini, S.; Ramirez, F.D.; Simard, T.; Hibbert, B. Methodological quality of COVID-19 clinical research. Nat. Commun. 2021, 12 (1), 943.

3. Zdravkovic, M.; Berger-Estilita, J.; Zdravkovic, B.; Berger, D. Scientific quality of COVID-19 and SARS CoV-2 publications in the highest impact medical journals during the early phase of the pandemic: A case control study. PLoS One 2020, 15 (11), e0241826. doi: 10.1371/journal.pone.0241826.

4. Quinn, T.J.; Burton, J.K.; Carter, B.; Cooper, N.; Dwan, K.; Field, R.; Freeman, S.C.; Geue, C.; Hsieh, P.H.; McGill, K.; Nevill, C.R.; Rana, D.; Sutton, A.; Rowan, M.T.; Xin, Y. Following the science? Comparison of methodological and reporting quality of covid-19 and other research from the first wave of the pandemic. BMC Med. 2021, 19 (1), 46.

5. Ioannidis, J.P.A.; Cripps, S.; Tanner, M.A. Forecasting for COVID-19 has failed. Int. J. Forecast. 2022, 38 (2), 423-438.

6. Zavalis, E.A.; Ioannidis, J.P.A. A meta-epidemiological assessment of transparency indicators of infectious disease models. PLoS One 2022, 17 (10), e0275380.

7. Ioannidis, J.P.A. Pre-registration of mathematical models. Math. Biosci. 2022, 345, 108782.

8. Flaxman, S.; Mishra, S.; Gandy, A.; Unwin, H.J.T.; Mellan, T.A.; Coupland, H.; et al. Estimating the effects of non-pharmaceutical interventions on COVID-19 in Europe. Nature 2020, 584, 257-261.

9. Alwan, N.A.; Burgess, R.A.; Ashworth, S.; et al. Scientific consensus on the COVID-19 pandemic: we need to act now. Lancet 2020, 396, e71-2.

10. Chin, V.; Ioannidis, J.P.A.; Tanner, M.A.; Cripps, S. Effect estimates of COVID-19 non-pharmaceutical interventions are non-robust and highly model-dependent. J. Clin. Epidemiol. 2021, 136, 96-132.

11. Islam, N. “Excess deaths” is the best metric for tracking the pandemic. BMJ 2022, 376, 285.

12. Levitt, M.; Zonta, F.; Ioannidis, J.P.A. Comparison of pandemic excess mortality in 2020-2021 across different empirical calculations. Environ. Res. 2022, 213, 113754.

13. Estimated excess death count from the Economist. https://ourworldindata.org/excess-mortality-covid (last accessed November 2, 2023).

14. Levitt, M.; Zonta, F.; Ioannidis, J.P.A. Excess death estimates from multiverse analysis in 2009-2021. Eur. J. Epidemiol. 2023, 12, 1-11.

15. Ioannidis, J.P.A.; Zonta, F.; Levitt, M. Variability in excess deaths across countries with different vulnerability during 2020-2023. Proc. Natl. Acad. Sci. U S A 2023, 120 (49), e2309557120.

16. National Center for Health statistics. Provisional overdose death counts. In: https://www.cdc.gov/nchs/nvss/vsrr/drug-overdose-data.htm (last accessed November 1, 2023).

17. Sohi, I.; Chrystoja, B.R.; Rehm, J.; Wells, S.; Monteiro, M.; Ali, S.; Shield, K.D. Changes in alcohol use during the COVID-19 pandemic and previous pandemics: A systematic review. Alcohol Clin. Exp. Res. 2022, 46 (4), 498-513.

18. Magesh, S.; John, D.; Li, W.T.; Li, Y.; Mattingly-App, A.; Jain, S.; Chang, E.Y.; Ongkeko, W.M. Disparities in COVID-19 outcomes by race, ethnicity, and socioeconomic status: A systematic-review and meta-analysis. JAMA Netw. Open 2021, 4 (11), e2134147.

19. Pezzullo, A.M.; Axfors, C.; Contopoulos-Ioannidis, D.G.; Apostolatos, A.; Ioannidis, J.P.A. Age-stratified infection fatality rate of COVID-19 in the non-elderly population. Environ. Res. 2023, 216 (Pt 3), 114655.

20. Engzell, P.; Frey, A.; Verhagen, M.D. Learning loss due to school closures during the COVID-19 pandemic. Proc. Natl. Acad. Sci. U S A 2021, 118 (17), e2022376118.

21. Harrison, L.; Carducci, B.; Klein, J.D.; Bhutta, Z.A. Indirect effects of COVID-19 on child and adolescent mental health: an overview of systematic reviews. BMJ Global Health 2022, 7 (12), e010713.

22. Decerf, B.; Ferreira, F.H.G, Mahler, D.G.; Sterck, O. Lives and livelihoods: Estimates of the global mortality and poverty effects of the Covid-19 pandemic. World Dev. 2021, 146, 105561.

23. Joffe, A.R. COVID-19: Rethinking the Lockdown Groupthink. Front. Public Health 2021, 9, 625778.

24. Nadarajah, R.; Wu, J.; Hurdus, B.; Asma, S.; Bhatt, D.L.; Biondi-Zoccai, G.; Mehta, L.S.; Ram, C.V.S.; Ribeiro, A.L.P.; Van Spall, H.G.C.; Deanfield, J.E.; Lüscher, T.F.; Mamas, M.; Gale, C.P. The collateral damage of COVID-19 to cardiovascular services: a meta-analysis. Eur. Heart J. 2022, 43 (33), 3164-3178.

25. Axfors, C.; Pezzullo, A.M.; Contopoulos-Ioannidis, D.G.; Apostolatos, A.; Ioannidis, J.P. Differential COVID-19 infection rates in children, adults, and elderly: Systematic review and meta-analysis of 38 pre-vaccination national seroprevalence studies. J. Glob. Health 2023, 13, 06004.

26. Grinspun, D.; Matthews, J.H.; Bonner, R.; Moreno-Casbas, T.; Mo, J. COVID-19 pandemic in long-term care: An international perspective for policy considerations. Int. J. Nurs. Sci. 2023, 10 (2), 158-166.

27. August, M. Socialize, De-Commodify and De-Financialize Long-Term Care. Healthc Pap. 2021, 20 (1), 34-39.

28. Ballin, M.; Ioannidis, J.P.; Bergman, J.; Kivipelto, M.; Nordström, A.; Nordström, P. Time-varying risk of death after SARS-CoV-2 infection in Swedish long-term care facility residents: a matched cohort study. BMJ Open 2022, 12 (11), e066258.

29. Erikstrup, C.; Laksafoss, A.D.; Gladov, J.; Kaspersen, K.A.; Mikkelsen, S.; Hindhede, L.; Boldsen, J.K.; Jørgensen, S.W.; Ethelberg, S.; Holm, D.K.; Bruun, M.T.; Nissen, J.; Schwinn, M.; Brodersen, T.; Mikkelsen, C.; Sækmose, S.G.; Sørensen, E.; Harritshøj, L.H.; Aagaard, B.; Dinh, K.M.; Busch, M.P.; Jørgensen, C.S.; Krause, T.G.; Ullum, H.; Ostrowski, S.R.; Espenhain, L.; Pedersen, O.B.V. Seroprevalence and infection fatality rate of the SARS-CoV-2 Omicron variant in Denmark: A nationwide serosurveillance study. Lancet Reg. Health Eur. 2022, 21, 100479.

30. Nurmi, J.; Jaakola, J. Losing trust: Processes of vaccine hesitancy in parents’ narratives. Soc. Sci. Med. 2023, 331, 116064.

31. Jefferson, T.; Dooley, L.; Ferroni, E.; Al-Ansary, L.A.; van Driel, M.L.; Bawazeer, G.A.; Jones, M.A.; Hoffmann, T.C.; Clark, J.; Beller, E.M.; Glasziou, P.P.; Conly, J.M. Physical interventions to interrupt or reduce the spread of respiratory viruses. Cochrane Database Syst. Rev. 2023, 1 (1), CD006207.

32. Cash-Goldwasser, S.; Reingold, A.L.; Luby, S.P.; Jackson, L.A.; Frieden, T.R. Masks during pandemics caused by respiratory pathogens-evidence and implications for action. JAMA Network Open 2023, 6 (10), e2339443.

33. Schippers, M.C.; Ioannidis, J.P.A.; Joffe, A.R. Aggressive measures, rising inequalities, and mass formation during the COVID-19 crisis: An overview and proposed way forward. Front. Public Health 2022, 10, 950965.

34. Ioannidis, J.P.A.; Zonta, F.; Levitt, M. Flaws and uncertainties in pandemic global excess death calculations. Eur. J. Clin. Invest. 2023, 53 (8), e14008.

35. Ioannidis, J.P.A.; Contopoulos-Ioannidis, D.G. Prepandemic cross-reactive humoral immunity to SARS-CoV-2 in Africa: Systematic review and meta-analysis. Int. J. Infect. Dis. 2023, 134, 160-167.

36. Namuniina, A.; Muyanja, E.S.; Biribawa, V.M.; Okech, B.A.; Ssemaganda, A.; Price M.A.; Hills, N.; Nanteza, A.; Bagaya, B.S.; Weiskopf, D.; Riou, C.; Reynolds, S.J.; Galiwango, R.M.; Redd, A.D. Proportion of Ugandans with pre-pandemic SARS-CoV-2 cross-reactive CD4+ and CD8+ T-cell responses: A pilot study. PLOS Glob. Public Health 2023, 3 (8), e0001566.

37. Ioannidis, J.P. Revisiting COVID-19 models and epidemiology. In Critique of Pandemic Reason; edited by Cappelluti, F., Cesaretti, P., Laviano, F. Meltemi: Milano, 2025 (in press).

38. Chiolero, A.; Tancredi, S.; Ioannidis, J.P.A. Slow data public health. Eur. J. Epidemiol. 2023, 38, 1219-1225. doi: 10.1007/s10654-023-01049-6.