A critique of estimates of excess mortality from COVID-19 pandemic

In the past, The Lancet has published papers which have often enjoyed media attention with an enduring announcement effect which may continue to influence the public and health

authorities.

Recently, widely reported in the media, a paper [1] with 95 authors was published on “Estimating excess mortality due to COVID-19 pandemic: a systematic analysis of COVID-19-related mortality, 2020-2021”. Computer modelling which is central to the paper [1] may overestimate the COVID-19 mortality threat, as happened frequently during the pandemic, causing disproportionate policy responses.

Excess mortality is observational data defined in standard statistics by: Excess mortality = All-cause mortality – expected mortality. In this equation, all-cause mortality is data collected for all deaths observed for the year and expected mortality is an average, usually of the previous five years calculated from past mortality data. Expected mortality is known and easily calculated. Once all-cause mortality data are known from observations, excess mortality is simply an empirical fact, calculated straightforwardly from data.

The general purpose of calculating excess mortality is to quantify unexpected deaths leading to efforts to investigate causes which may explain the deviations.

Methods

The paper [1] accepts this definition stating “Excess mortality over time was calculated as observed mortality, after excluding data from periods affected by late registration and anomalies such as heat waves, minus expected mortality.” Therefore, to calculate excess mortality for COVID-19 pandemic till the end of 2021, it is essential to have observed all-cause mortality data to 2021, which are not available to the authors, at least not yet for most countries.

Without adequate collection of observational all-cause mortality data, excess mortality cannot be calculated.

To predict observations, the authors have developed a computer model (with adjustable parameters and covariates) which uses all-cause mortality data pre-2020, before the COVID-19 pandemic, as input to predict all-cause mortality data post-2020, during the COVID-19 pandemic.

Given its model projections of all-cause mortality, the paper then estimates the excess mortality due to the COVID-19 pandemic: Excess mortality due to the COVID-19 pandemic, defined as the net difference between the number of deaths during the pandemic (measured by observed or estimated all-cause mortality) and the number of deaths that would be expected on the basis of past trends in all-cause mortality, is therefore a crucial measure of the true toll of the COVID-19 pandemic.

That is, a model estimate of the “excess mortality during COVID-19 pandemic” is simply considered by the authors as an estimate for the “excess mortality due to COVID-19 pandemic”.

Results

The results of computer modelling of all-cause mortality for 191 countries are presented as estimated excess mortality in the 2020-2021 two-year period for each country and aggregated sub-region. The estimated excess mortality for all countries totalled 18 million triples the 6 million reported COVID-19 mortality.

The main result can be explained by a de facto assumption of substantial undercounting of mortality in major regions which has been adjusted by computer modelling. The levels of undercounting have been expressed by the “ratio of excess mortality to reported COVID-19 mortality” for each country as seen in Figure 4 of the paper:

Regions with high estimated ratios of excess to reported mortality (RERM) are coloured red/orange in the above figure which are predominately in Africa, India and China. These regions have large populations, accounting for more than half of global population, and yet they have some of the lowest reported COVID-19 mortality. By modelling these countries with very high RERM exceeding 20, global excess mortality would increase substantially.

Critique

The modelling of the paper lacks transparency, typical of COVID-19 models [3]. With insufficient technical details (even with computer codes) to replicate and potentially falsify the modelling, it is not possible to critique exactly how the model can predict all-cause mortality during COVID-19 pandemic from historical data without COVID-19 pandemic. Without possibilities of replication and potential falsification, the paper lacks the essential elements of science [4].

The computer model estimates 2020-2021 all-cause mortality for 191 countries during the COVID-19 pandemic, using past all-cause mortality data which have no similar historical precedence. The COVID-19 pandemic saw unprecedented public health interventions, including near universal vaccination which vary between countries. The COVID-19 pandemic did not preclude mortality due to natural disasters such as floods, droughts, cyclones etc.

Out-of-sample testing in the paper is problematic when data collected during COVID-19 pandemic are already uncertain subject to revision. It is difficult to glean from the paper how all-cause mortality in 2020-2021 can be estimated by computer models with any confidence. The regression model estimate of a global 2020-2021 all-cause mortality implies, from the above equation, the excess mortality during the COVID-19 pandemic.

By calling this excess mortality estimate the “excess mortality due to COVID-19 pandemic”, the authors have attributed all excess mortality to the COVID-19 pandemic, based on the assumption that “The full impact of the pandemic has been much greater than what is indicated by reported deaths due to COVID-19 alone.” This statement is not always true, as the paper itself shows excess mortality was negative in Australia and New Zealand, during COVID-19 pandemic.

The paper does not explain how the 12 million excess mortality in addition to the six million reported was related to the COVID-19 pandemic. The 18 million estimated excess deaths translate to an estimated global excess mortality rate of 228 per 100,000, not 120.3 per 100,000 as stated in the main table of the paper — a significant internal inconsistency. Theestimated global excess mortality rate of 228 per 100,000 is improbably higher than 180 per100,000 for the US, which is likely most vulnerable in the COVID-19 pandemic [5].

References

1. Wang H, Paulson KR, Pease SA et al. Estimating excess mortality due to COVID-19 pandemic: a systematic analysis of COVID-19-related mortality, 2020-2021, The Lancet , March 10, 2022. Retrieved from: https://www.thelancet.com/action/showPdf?pii=S0140-6736%2821%2902796-3 (accessed March 18, 2022).
2. Roser M, Ritchie E, Ortiz-Ospina E. World population growth. Published online at OurWorldInData.org. Retrieved from: https://ourworldindata.org/world-population-growth#how-many-people-die-and-how-many-are-born-each-year (accessed March 18, 2022).
3. Jalali MS, DiGennaro C, Sridhar D. Transparency assessment of COVID-19 models, The Lancet , October 27, 2020. Retrieved from: https://scholar.harvard.edu/files/jalali/files/transparency_assessment_covid19.pdf (accessed March 18, 2022).
4. Popper K . Conjectures and refutations: the growth of scientific knowledge . Routledge Classics, London and New York, 2002 (Page 48).
5. Sy W. Randomized controlled trial limitations: a case study of vaccine efficacy in COVID-19 epidemiology. March 27, 2022. Retrieved from: http://www.asepp.com/randomized-controlled-trial-limitations-a-case-study-of-vaccine-efficacy-in-covid-19-epidemiology/ (accessed March 27, 2022).
6. Centers for Disease Control and Prevention (U.S.) Provisional death counts for coronavirus disease (COVID-19), National Center for Health Statistics. Retrieved from: https://www.cdc.gov/nchs/nvss/vsrr/covid19/tech_notes.htm (accessed March 19, 2022).

Please Donate Below To Support Our Ongoing Work To Expose The Lies About COVID19

PRINCIPIA SCIENTIFIC INTERNATIONAL, legally registered in the UK as a company incorporated for charitable purposes. Head Office: 27 Old Gloucester Street, London WC1N 3AX.