The hidden biases that inflated Covid vax effectiveness and safety
Today, together with Dr. Marco Alessandria, Dr. Giovanni Trambusti, Dr. Giovanni M. Malatesta, and Dr. Alberto Donzelli, we published an crucial peer-reviewed scientific study titled “Classification bias and impact of COVID-19 vaccination on all-cause mortality: the case of the Italian Region Emilia-Romagna” [1]
In this study, we provide the first peer-reviewed evidence, based on real-world data, demonstrating how certain statistical methods have led to an overestimation of the effectiveness and safety of COVID-19 vaccines.
This paper will shock the world, because it proves that all scientific studies conducted so far that are affected by this bias should be reassessed.
We addressed a critical bias that can substantially distort real-world evaluations of vaccine effectiveness and safety, known as the “case-counting window bias”.
This bias, theorized by Fung et al. [2], occurs because individuals are classified as “unvaccinated” during the first 14 days after receiving the vaccine (the period believed necessary for the immune response to develop fully).
As a result, any adverse events, including deaths during this time, are incorrectly attributed to the unvaccinated group, artificially inflating its mortality rate, while simultaneously underestimating mortality among vaccinated individuals.
By analyzing detailed daily data on all-cause mortality and vaccine administration in the Emilia-Romagna region (Italy), obtained through a FOIA request by lawyer Lorenzo Melacarne (in accordance with the art. 5, comma 2 of the Italian Legislative Decree No. 33/2013), we found a clear temporal coincidence between vaccination campaigns and spikes in deaths among those incorrectly classified as unvaccinated during this critical window (Figure 1).
Figure 1. The chart illustrates the daily mortality rate per 100,000 individuals (age group 70-79), comparing those vaccinated (represented by the solid red line) with those unvaccinated (shown by the solid green line).
Additionally, it shows the cumulative number of vaccinations administered with at least one dose (indicated by the red dotted line) [taken from Alessandria et al., 2025].
Our statistical analysis demonstrated significant differences in mortality between vaccinated and unvaccinated groups, during the critical 14-day post-vaccination window when misclassification occurs.
Importantly, these mortality differences cannot be explained solely by COVID-19 deaths, which accounted for only about nine percent of all deaths in Italy in 2021. Even after excluding COVID-19-related deaths, the disparity between groups remained significant, indicating systematic misclassification rather than true vaccine benefit.
We also observed that the difference diminished with age, likely reflecting the increased comorbidity burden in older individuals that influences overall mortality risk (for more detailed information, please see the article, which is published in open access format and freely available to everyone).
Our findings suggest a harvesting effect, whereby vulnerable individuals succumb shortly after vaccination, but their deaths are wrongly counted among the unvaccinated.
This misclassification masks potential serious vaccine-related adverse events occurring shortly post-vaccination, such as severe allergic reactions, cardiovascular events, or autoimmune responses.
Moreover, the use of similar classification practices by many countries, including the United Kingdom, suggests that this bias is widespread internationally.
For example, UK public health guidelines classify individuals as unvaccinated for 14 to 21 days following vaccination, which leads to the misattribution of early adverse events.
It is crucial to recognize that the case-counting window bias is related to another well-established phenomenon in observational research known as the immortal time bias.
Prof. Norman Fenton and Prof. Martin Neil were among the first to identify how these biases shift cases and deaths in a manner that exaggerates the apparent efficacy and safety of vaccines by creating misleading temporal categorizations. Prof. Fenton himself has referred to such manipulations as a “cheap trick” — a statistical illusion that artificially enhances perceived vaccine effectiveness [3].
James Lyons-Weiler, PhD had already identified and highlighted this significant statistical inconsistency affecting vaccine efficacy data as early as October 2021, underscoring the need for critical scrutiny of how vaccination status and case counting windows distort perceived vaccine performance [4].
Importantly, in January 2022, Prof. Martin Neil, Prof. Norman Fenton, and colleagues published a pre-print study critically examining UK ONS Covid vaccine mortality data by comparing all-cause mortality between vaccinated and unvaccinated groups [5].
While initial reports suggested lower mortality in vaccinated older adults, the study uncovered significant data inconsistencies likely due to misclassification, reporting delays, and population errors.
The researchers found no support for explanations based on bias or socio-demographic factors, concluding that the data do not reliably demonstrate that vaccines reduce all-cause mortality and may indicate increased mortality shortly after vaccination in older populations.
Our findings have important implications: failing to account for these biases can lead to substantial overestimation of vaccine benefits and safety, resulting in misguided public health policies.
Therefore, the scientific community must recognize and adjust for these biases to produce more accurate and transparent assessments of vaccine risks and benefits.
In conclusion, our study shows that the case-counting window bias inflates mortality rates wrongly attributed to the unvaccinated, while simultaneously underestimating adverse reactions occurring shortly after vaccination.
To ensure reliable interpretation of observational vaccine studies and informed public health decisions, it is crucial to correct this bias alongside the immortal time bias. Furthermore, all existing vaccine effectiveness studies should be reassessed for these biases.
A key part of this process is having accurate and timely data on individuals’ vaccination status, which allows proper classification of cases and deaths and supports a more reliable evaluation of vaccine safety and effectiveness in real-world settings.
References
[1] M. Alessandria, G. Trambusti, G.M. Malatesta, P. Polykretis, A. Donzelli, Classification bias and impact of COVID-19 vaccination on all-cause mortality: the case of the Italian region Emilia-Romagna, Autoimmunity 58 (2025) 2562972. https://doi.org/10.1080/08916934.2025.2562972.
[2] K. Fung, M. Jones, P. Doshi, Sources of bias in observational studies of covid-19 vaccine effectiveness, J Eval Clin Pract 30 (2024) 30–36. https://doi.org/10.1111/jep.13839.
[3] N. Fenton, M. Neil, Vaccine efficacy “cheap trick” by exclusion, Where Are the Numbers? By Norman Fenton and Martin Neil (2023). https://wherearethenumbers.substack.com/p/vaccine-efficacy-cheap-trick-by-exclusion.
[4] J.L.-W. PhD, How The Definition of “Fully Vaccinated” Misleads People on COVID-19 Vaccine Safety & Efficacy: An Explanation For CNN’s Drew Griffin, Popular Rationalism (2021). https://popularrationalism.substack.com/p/how-the-definition-of-fully-vaccinated.
[5] M. Neil, N. Fenton, J. Smalley, C. Craig, J. Guetzkow, S. McLachlan, J. Engler, D. Russell, J. Rose, Official mortality data for England suggest systematic miscategorisation of vaccine status and uncertain effectiveness of Covid-19 vaccination, 2022. https://doi.org/10.13140/RG.2.2.28055.09124.
See more here substack.com