Unmasking the Greenhalgh Mask Study
COVID-19 sparked a polarising debate over the effectiveness of face masks in reducing the spread of respiratory infections.
Central to this debate are two major reviews of the evidence: the 2023 Cochrane Review by Jefferson et al., and the 2024 ‘state of the science’ review led by Greenhalgh et al. These reviews reach notably different conclusions. Understanding the differences between these reviews is essential for anyone seeking an evidence-informed view on the role of face masks in public health.
Published last year, the review led by Professor Trisha Greenhalgh and published in Clinical Microbiology Reviews titled ‘Masks and respirators for prevention of respiratory infections: a state of the science review’ gave comfort to many on the side of wearing face masks. It was considered to be a response to, and referred to the study by, Professor Tom Jefferson et al. in the Cochrane Library titled ‘Physical interventions to interrupt or reduce the spread of respiratory viruses’.
Absence of evidence
Considered by many, especially those sceptical about the effectiveness of face masks, to be the best review of the evidence up to that point, the review by Jefferson et al. could find no robust evidence that face masks of any kind were effective per se or that respirator type masks were any more effective than the disposable face masks much in evidence during COVID-19. Greenhalgh et al. purport to provide evidence that face masks are effective and that respirator type face masks are even more effective.
As science is a perpetual discussion, due to the stochastic nature of the outcome of scientific studies and the phenomenon of regression towards the mean, it would be improper to claim that either study was the final word or that there was definitive evidence in either direction. Therefore, it is not helpful to state which of the studies provides the correct answer.
But it is possible to compare and contrast the methods used in these studies, to examine the claims they make and how they arrive at their conclusions. In that regard, the study by Jefferson et al. makes no bold claims other than, according to its analysis, there is an absence of evidence for the effectiveness of face masks. However, Greenhalgh and colleagues make bolder claims about the effectiveness of face masks. It is worth considering why.
Greenhalgh et al. reflects on the Jefferson et al. study early on, highlighting its method of considering only evidence available from randomised controlled trials (RCTs) and claiming this is “controversial”. This is an unusual claim for two reasons. First, the method of gathering and evaluating evidence used by the Cochrane Library whereby only individual studies of the highest quality are included is considered the gold standard method of conducting evidence syntheses. Second, as a cursory search on Google shows, Greenhalgh herself is no stranger to the Cochrane Library given that she has led and participated in several of its reviews.
The Cochrane cookbook – tried and tested
The Cochrane method involves the methods of systematic review and meta-analysis. The former is a transparent and replicable method of retrieving relevant studies involving the use of search terms and date limits to interrogate specified databases.
For the studies retrieved, strict inclusion and exclusion criteria are set for which studies will be considered to hone many thousands of potential studies down to those specifically related to the research question. Next, the studies are evaluated for quality issues such as bias, adequacy of sample size and whether appropriate procedures have been used to follow the recruitment and dropout of participants.
Meta-analysis is a mathematical method of combining studies with similar interventions and outcomes. Steps are taken to minimise heterogeneity between the studies included and the degree of heterogeneity can be calculated and reported. Thus, the quality of the outcome of a Cochrane review itself can be evaluated.
In summary, the Cochrane Library uses an entirely transparent process, the protocol is subject to rigorous peer review prior to the study and, likewise, the final report. Clearly, while authors may have prejudices regarding the subject under review, the process of conducting such a review should be entered with an open mind and the methods ensure that the published outcomes are as free from bias as possible. This perfectly describes the methods applied in the review by Jefferson et al.
Starting with the answer
The review by Greenhalgh et al. did not follow Cochrane Library procedures. While selected groups of studies were meta-analysed, systematic methods were not used to retrieve studies in the first instance. Being a ‘state of the science’ review, the process used by Greenhalgh et al. is described as a “hermeneutic review” approach whereby the review team started with known sources, particularly those familiar to the team (which included experts in public health, epidemiology, infectious diseases, sociology, psychology and engineering).
There followed a process of iteratively refining the literature base by following up on references from key sources using Google Scholar. Thereafter, the authors actively sought recommendations from colleagues and researchers in relevant fields and crowdsourced references via posts on social media platforms. As a result they prioritised inclusivity and contextual understanding and eschewed methodological constraints (such as only including RCTs).
The problem with this approach is immediately obvious. The methods of retrieval are not transparent, systematic or replicable. The team, which one can presume to be biased in favour of face masks, far from approaching the issue with an open mind, began with studies its members considered showed that face masks were effective. They then mined the seams of articles in favour of face masks which were cited in those studies.
Let the echo chamber decide
The method of crowdsourcing further material is clearly biased as, with more to gain from an outcome in favour of face masks, the main respondents were likely mostly in favour of face masks and, subsequently, submitted studies reinforcing their bias. The review team had no way of checking for such bias.
I searched in vain for non-experimental studies showing face masks were ineffective and found only two suggesting the possibility of publication bias in this field. Rigorous experimental studies such as RCTs must be registered prior to commencing the study under the AllTrials agreement signed by all leading academic publishers specifically to minimise publication bias. No such agreement governs non-experimental work.
Moreover, beyond these methodological issues, the lead author Greenhalgh was a leading campaigner of fanatical proportions for masking during COVID-19. Her Twitter feed repeatedly urged the wearing of face masks, at some points more than one at a time, even during exercise such as jogging. That at least one member of the review team was biased in favour of face masks is demonstrated at the end of the review where a headshot of each member is shown. Professor C Raina MacIntyre is shown with a face mask under her chin.
Proving a point
In my estimation, Greenhalgh and colleagues set out to prove that face masks were effective and conducted secondary analyses of data to prove their point. While the main issue affecting most of the population of the world during COVID-19 was the enforced wearing of disposable face masks, these are given very little consideration in the study. The discussion quickly moves to respirator type face masks which, at least, are theoretically more effective. The authors also used modelling data to support the effectiveness of face masks. Ironically, while critical of the ‘controversial’ Cochrane approach to evidence synthesis, Greenhalgh and co. use meta-analysis of selected data and are not averse to citing RCTs where the outcome favours face masks.
Both reviews offer insight into the complex and often polarised discussion surrounding the use of face masks for respiratory protection. The Jefferson et al. Cochrane review exemplifies a disciplined and transparent approach to evidence synthesis, while the Greenhalgh et al. review embraces a more interpretive, expansive method that seeks relevance and applicability beyond the confines of RCTs. However, when scientific credibility is under scrutiny, methodological rigour, transparency and neutrality are essential.
In that regard, the contrast between these two reviews highlights not just differing conclusions but fundamentally different approaches to what constitutes reliable evidence. Sitting the Jefferson et al. and the Greenhalgh et al. reviews side by side, examining the methods, the possibility of bias amongst the teams and the conclusions reached, I doubt I am alone in concluding which of the reviews was the more robust and reliable.
See more here Daily Sceptic
Please Donate Below To Support Our Ongoing Work To Defend The Scientific Method
PRINCIPIA SCIENTIFIC INTERNATI ONAL, legally registered in the UK as a company 
incorporated for charitable purposes. Head Office: 27 Old Gloucester Street, London WC1N 3AX.
Trackback from your site.
VOWG
| #
The debate ended decades ago. They have never been of any value. Wearing a bacteria incubator on one’s face is not healthy.
Reply
BOB M
| #
There should have NEVER even been a debate, since the disposable face masks were never designed for such use.
Me thinks Ms. Greenhalgh is being paid by the big mask manufacturers or is in dire need of more funding for her pathetic studies using only selected data. And any “study” that uses “google scholars” as a resource, is immediately suspect.
Reply