Misinformation, Disinformation, and the Climate Sciences: Part 2, Our broken peer-review process
Introduction: observational bias
In 1963 Robert Rosenthal, a psychologist, tasked two groups of students to measure a group of rats’ abilities to complete a maze (https://www.gwern.net/docs/iq/1963-rosenthal.pdf). One group was told they were dealing with “bright” rats, the other received “dull” rats, though in reality both groups were randomly selected from the same lab rat population.
The students measuring the “bright” rats acted in ways to ensure their rats performed better, while the “dull rat” group ensured their rats performed worse. In essence, the students’ theoretical expectations influenced their observations, an effect known as observational bias.
Blinding the students to the expected outcome would have clearly resulted in better science. But what motivated these students to “cheat”? Perhaps they wanted to avoid personal consequences by demonstrating their “competence” to their professor and their peers by observing the “correct” result.
A more charitable explanation might be that they ideologically wanted to advance science by observing what their professor – their scientific authority – told them was the correct outcome.
Fear of consequences, academic progression, peer pressure and ideological fervor are powerful motivators for scientists. Part1 of this article demonstrated that since the 17th century the peer-review process has been Science’s best tool for ensuring scientific publications follow the scientific method and for eliminating often-unintentional observational and ideological bias.
Part 2 demonstrates the process is currently failing us.
The broken peer review process part 1: a Social Science hoax
In 1996 Alan Sokal, a mathematics and physics professor at University College London and New York University resp., successfully submitted an article entitled “Transgressing the Boundaries: Towards a Transformative Hermeneutics of Quantum Gravity” to Social Text, an academic journal of postmodern cultural studies with an openly progressive ideological orientation and scholarship.
Sokal specifically wanted to investigate whether “a leading North American journal of cultural studies – whose editorial collective includes such luminaries as Fredric Jameson and Andrew Ross – publish an article liberally salted with nonsense if (a) it sounded good and (b) it flattered the editors’ ideological preconceptions” (http://linguafranca.mirror.theinfo.org/9605/sokal.html).
Was it reasonable to expect the Social Text editors – who possibly had a poor understanding of physics – to determine that phrases such as “quantum gravity is a social and linguistic construct” are nonsense? Our trust in scientific publications hinges on our expectation that their editors and reviewers understand and critically review what they are publishing. Even half-alert editor should have been able to determine that Sokal’s article was gibberish.
A social construct is an object whose meaning is determined by society, for example class distinctions, money, language, governmental hierarchies, etc. The opposite of a social construct is an object whose meaning is not determined by society, that is an objective truth that does not change with society. The sentence’s null hypothesis – quantum gravity is not a social construct – is therefore a tautology whose falsification is impossible: a scientific discipline deals with objective truths.
“Anyone who believes that the laws of physics are mere social conventions is invited to try transgressing those conventions from the windows of my apartment. (I live on the twenty-first floor.)”
The broken peer review process part 2: Scientific fraud
In August 2011 Diederik Stapel, Marcel Zeelenberg, and Roos Vonk, three Netherlandish social psychology professors at the Tilburg and Radboud universities, organized a press-release – prior to any peer-review – about their investigation into the psychological and symbolic meaning of meat. Specifically, their research showed that meat-eaters are more egotistical and less social than vegetarians (https://joop.bnnvara.nl/nieuws/vleeseters-zijn-egoistischer-en-minder-sociaal).
The news articles quoting their conclusions were met with wide-spread public ridicule. The public’s reaction was likely the main catalyst in the subsequent investigation of Stapel’s work: in September 2011 Tilburg University suspended him pending their investigation into possible academic fraud. Their report (https://web.archive.org/web/20160627142859/https://www.tilburguniversity.edu/upload/547aa461-6cd1-48cd-801b-61c434a73f79_interim-report.pdf) concluded that Stapel fabricated data for at least 30 publications, and fiercely condemned the observation and confirmation bias in the social sciences peer review process..
“It is almost inconceivable that co-authors who analyzed the data intensively, or reviewers of the international ‘leading journals’, who are deemed to be experts in their field, could have failed to see that a reported experiment would have been almost infeasible in practice, did not notice the reporting of impossible statistical results, such as a series of t-values linked with clearly impossible p-values, and did not spot values identical to many decimal places in entire series of means in the published tables.
Virtually nothing of all the impossibilities, peculiarities and sloppiness mentioned in this report was observed by all these local, national and international members of the field, and no suspicion of fraud whatsoever arose. […]
A more general failure of scientific criticism in the peer community and a research culture that was excessively oriented to uncritical confirmation of one’s own ideas and to finding appealing but theoretically superficial ad hoc results. […]
Not infrequently reviews [of journal articles] were strongly in favour of telling an interesting, elegant, concise and compelling story, possibly at the expense of the necessary scientific diligence.”
Among the duped “leading journals” with poor peer-review processes was Science (https://www.science.org/doi/pdf/10.1126/science.1201068), who retracted the article “Coping with chaos: How disordered contexts promote stereotyping and discrimination”.
By 2011 it had become evident that the peer review process that enabled Sokal’s hoax and Stapel’s academic fraud was at least partially driven by material, social and ideological motivators, and not an altruistic desire to help society by weeding out poor science. Stapel himself (https://en.wikipedia.org/wiki/Diederik_Stapel#Reaction_by_Stapel) claims “the pressure to score, to publish” was his prime motivator.
Stapel’s claim that “I want to emphasize that the mistakes that I made were not born out of selfish ends” is stunningly self-delusional given his admission “the pressure to score” was driving him. He admits through his apology that his mistakes did not help – that is damaged – his career, his colleagues, his discipline, his university, and science in general. What remains is that he apparently believed he was unselfishly helping society by fabricating data to promote his ideological beliefs, while completely unconcerned that such beliefs may cause societal harm.
The broken social sciences peer-review process apparently promoted articles that confirmed shared ideological beliefs, thereby silencing any critical voices, stifling inconvenient scientific debates, and ignoring contradicting data. It took a public outcry of non-scientists over an article on the psychology of meat to provide the wake-up call to academia that much of his “science” was in fact disinformation.
The broken peer review process part 3: tripling down
In 2018 the “Grievance Study” (https://en.wikipedia.org/wiki/Grievance_studies_affair) hoax demonstrated that the social science peer reviewers were tripling down on their mistakes: three authors managed to publish 4 papers, and have a further 3 accepted, that were shockingly bad science. One paper suggested men anally penetrate themselves with sex toys in order to reduce their transphobia, another claimed dogs engage in rape culture.
It’s comical that a purportedly scientific peer-review is so blinded by ideology that it cannot discern science from drivel when even the most gullible members of the general public know these articles are gibberish. But the real-world consequences of such irresponsibility are far less amusing. The authors’ feminist rewriting of parts of Hitler’s Mein Kampf was accepted by social work journal Affilia.
Shouldn’t we be alarmed that a social scientific authority endorses ideas similar to those once promoted by Goebbels’ Ministry of Propaganda? Does it benefit society to promote a brand of feminism that shares an ideological base with Nazism?
Shouldn’t we be worried when students indoctrinated by such nonsense join the workforce as politicians, HR reps, teachers, or future peer-reviewers? Note that none of this disinformation would alarm the University of Exeter academics as the mainstream “scientific consensus” was supportive of this disinformation.
The broken peer review process part 4: Couldn’t happen here
“Show me the incentive, and I will show you the outcome.”
It doesn’t take a social scientist to determine that given the same incentive structure as the social sciences that similar shenanigans are occurring in the physical sciences, resulting in a peer-review process that promotes articles that match a reviewer’s ideological beliefs – especially when they quote the reviewer’s own work – and silences any critical voices that might damage a reviewer’s or publisher’s funding or professional reputation. The incentives are identical.
An anecdotal example shows how un-retracted misinformation published by Nature is still fueling an anti-science disinformation campaign.
The article concludes that:
“it will take at least a decade for the fastest growing species to recover and much longer for longerlived and slow-growing species.”
But the Great Barrier Reef then did something unforeseen by Hughes et al.: it quickly recuperated. Data from the Australian Institute of Marine Science (AIMS), who have been measuring GBR coral cover since 1985, show that by 2021 the GBR had almost fully recovered (https://www.aims.gov.au/reef-monitoring/gbr-condition-summary-2020-2021).
The AIMS observations falsify numerous Hughes et al. claims:
- “repeated episodes of coral bleaching [are] inexorably tied to an impaired capacity for recovery”
- “the GBR is currently in high risk of “widespread ecological collapse”.
- “The prospect of a full recovery to the original mature assemblages is uncertain, over a timeline that will almost certainly encompass further mass bleaching events”
- “the new recruitment dynamics of the Great Barrier Reef in 2018 represent a marked departure from the recent past, shifting to much lower rates of coral replenishment”
The falsification of these fairly alarmist claims and predictions demonstrate that the authors’ expectation case was extreme. Scientific use of such words as “almost certainly” should not be undertaken lightly. As was probably intended, given the title, the article was picked up and quoted by numerous news outlets, for example a CBS news article titled: “Global warming has caused such extensive damage to the Great Barrier Reef that scientists say its coral may never recover.” (https://www.cbsnews.com/news/great-barrier-reef-dying-climate-change-caused-decrease-in-new-coral-study-says/).
Note that even CBS included the weasel word “may” in its title, and so is less misinformative than the quoted article, whose main premise (title) is not falsifiable, and whose conclusion is contradicted by the data.
The misinformative Hughes et al. article continues to feed a campaign of anti-science disinformation. In June 2021 CBS published an alarmist follow-up article ideologically condemning the Australian government’s refusal to accede to the UNESCO’s demand to list the reef as “In Danger” (https://www.cbsnews.com/news/australia-united-nations-warning-great-barrier-reef-danger-climate-change/?intcid=CNM-00-10abd1h). The article quotes several disinformative, observationally-biased ideologues:
- UNESCO: “’urgent’ action is needed to combat the effects of climate change”, “”no possible doubt [that the GBR] is facing ascertained danger”
- The Climate Council: “the U.N. warning ‘brings shame’ on the [Australian] government.”
- Climate scientist Lesley Hughes: “The situation is dire, and our response should match that. It is important we remember that the Reef can be restored, but it needs a break from severe back-to-back bleaching events, and the only way to do that is to rapidly reduce our greenhouse gas emissions.”
The Australian government’s more evidence-based reaction – via their Minister for the Environment Sussan Ley – was:
“The Great Barrier Reef is the best-managed reef in the world and this draft recommendation has been made without examining the Reef first hand, and without the latest information.”
Were Nature’s editors and reviewers “strongly in favour of telling an interesting, elegant, concise and compelling story, possibly at the expense of the necessary scientific diligence” in a manner similar to Stapel’s?
The article’s main Marine Biology focus is whether corals can quickly recover from back-to-back heat events, a highly relevant study whose definitive answer in 2022 – yes, they can – has been lost in the Global Warming hubris.
Acroporidae have been around since at least the Jurassic, and have survived numerous severe “environmental disturbances” such as the Cretaceous, when global temperatures were 5-10°C higher than today (https://eos.org/science-updates/an-unbroken-record-of-climate-during-the-age-of-dinosaurs), the K-T meteorite impact that likely killed off half the species on Earth (https://astrobiology.nasa.gov/news/do-we-know-what-killed-the-dinosaurs/), the Toba super-volcano (Sumatra; 74000 years ago) that “dimmed the sun for six years, disrupted seasonal rains, choked off streams and scattered whole cubic miles of hot ash”(https://text.npr.org/163397584), and the 60+ meter Early Holocene sea level rise (12000 – 7000 years ago) that shifted the coastlines – and coral colonies – landwards by 10’s to 100’s of kilometers.
It is extremely likely that in their 200+ mln years of existence Acroporidiae colonies would have experienced numerous back-to-back heat stress events and would have evolved to survive such occurrences, so a highly speculative hypothesis suggesting that such a single occurrence could wipe out the world’s largest reef requires extraordinary proof.
The article’s main theory can be summarized as: Global Warming results in more frequent heat stress, which results in more coral bleaching and adult mortality, which results in lower rates of coral larvae production, which results in lower juvenile recruitment, which results in long reef recovery rates, which results in an existential risk for the GBR. Could the authors have debunked their own misinformation using their own data, had they and their peer-reviewers not been blinded by their observational and ideological bias, that is blinded by their desire to tell an “interesting, elegant, concise and compelling story” about how humanity needs to reduce greenhouse gas emissions?
Their evidence supporting their speculative hypothesis consists of “the strong relationship between the suppression of recruitment and the loss of adult brood stock [authors’ figures below] indicates that recovery of the key regenerative process of larval recruitment will be protracted, because of its reliance on the slow rebuilding of populations of sexually mature adults in coming years and decades.”
But their data doesn’t support their claims. These two graphs below from the article represent the change in recruitment and coral cover for the two main coral procreation mechanisms (left “Spawners”, right “Brooders”), plus the authors’ linear models used to reach the conclusion above (solid lines, grey shaded area is 95 percent confidence interval; note: colored shading and dashed line have been added). From the article:
“Corals that are ‘brooders’ release well-developed larvae that typically settle locally within a day, whereas broadcast spawning corals (spawners) release eggs and sperm that fertilize externally, followed by a peak in larval settlement 4–7 days later”, that is settle after ocean currents have possibly moved them a significant distance from their parents.
The graphs are unusual, confusing and misleading. First: why plot the data on a logarithmic scale? Two linear models (y=x; Δ recruit = Δ cover) are assumed by the authors (“For both spawners and brooders, the slope of the stock–recruitment relationship was indistinguishable from unity”), so plotting the data on a log-log scale is unnecessary, and results in an intuitively less-comprehensible display.
The authors’ linear models are the result of an ordinary least-squares regression analysis. The r2 value for a regression model can be seen as the amount of data variance that is explained (predicted) by the model divided by the total data variance; the regression models’ r2 value are therefore a measure of the model’s suitability.
Data points that are substantially different from the mean – outliers – are large contributors to the total data variance, so a linear model that connects the upper and lower extreme values will have a high r2 value but may be a poor overall predictor of the majority of values that lie in between. Note that r2 values of 0.658 and 0.517 are not typically considered to be high.
The fact that more than half (8/15) of the data points lie just at or outside both model’s 95 percent confidence interval (grey-shaded area) indicates the linear model is not able to explain much of the variance, and that other factors play a significant role.
The authors investigate whether changes in reef cover are the dominant factor determining changes in larvae recruitment, though their own data and anecdotal evidence show that this is not the case for the Spawners:
“a lthough coral cover at [Lizard Island] was reduced sharply within the tracks of two tropical cyclones in April 2014 and March 2015, recruitment by spawners remained high afterwards in early 2016”.
Numerous additional factors other than coral cover evidently play a role in spawner coral recruitment: heat prior to and during spawning, sedimentation load in the ocean, pollutants, nutrition, etc. (https://animaldiversity.org/accounts/Acropora_millepora/). The severe impairment outlier data (Δ cover < -95 percent; red zones; 3 data points) show that the change in spawner recruitment decreases linearly with changes in coral cover under severe stress: a certain threshold coral density might be necessary for sufficient eggs to be fertilized.
The brooder data show higher-than-unity recruitment is occurring, that is more recruitment than was predicted by the unity linear model, possibly due to the remaining adult brooders’ ability to produce numerous larvae that settle locally, in a faster-than-linear recovery even under the worst of cover change conditions.
The bulk of the data (white zone) are more representative (average) of the cover-recruitment relationship during back-to-back heating events than the outlier points. Most of the brooder data points (8/9) lie under the authors’ linear model, and are better predicted by a true unity linear model (added dashed line), possibly indicating brooding coral species such as pocilloporids may indeed recover slowly from severe bleaching events.
The spawner data however show a negative correlation (less negative change in recruitment for greater negative changes in coral cover) indicating that a) factors other than change in coral cover are likely dominating the recruitment process and b) their recovery from back-to-back bleaching events can occur rapidly, even during fairly large negative changes in coral cover.
It is self-evident that under benevolent conditions (green areas) coral reef growth is a function of juvenile recruitment: the reef can’t grow without new recruits settling. All recovery data points (Δ cover > 0) show Δ recruit values that lie above their (linear) modelled values, suggesting that recovery / growth – when it happens – occurs at exponential (note: log plots) not linear rates. This interpretation is confirmed by the AIMS spawner (Acropora) data:
“once established, [Acropora] enter an exponential growth phase which rapidly increases measures of percent hard coral cover”.
The data suggest that coral recovery is dependent on numerous factors other than back-to-back heat stress events. A working hypothesis for a follow-up study might be that some corals – through evolution – have developed coping mechanisms such as external spawning to deal with such environmental disturbances as decreases in coral cover and heat stress. Or that such coral bleaching events form an essential part of their resilience: genetically weaker adults die off in favor of genetically more temperature-resilient juveniles, a theory that is mentioned though not investigated for the GBR data by Hughes et al.
Such hypotheses should be investigated as they might greatly benefit global reef management strategies. While it’s possible – and perhaps even likely – that Global Warming is somehow responsible for an increase in the frequency of heat stress events in the GBR, the article’s assumption of “unprecedented back-to-back mass bleaching events caused by global warming” is asserted without evidence.
Hitchens’ razor – what can be asserted without evidence can be dismissed without evidence – therefore implies that the article’s title “Global warming impairs stock–recruitment dynamics of corals” is – for now – grossly overstating the global impact of the local evidence. The IPCC – the scientific authority on climate change – has established that between 1971-2010 the oceans mostly cooled to the northeast of Australia, the location of the GBR.
No evidence linking the back-to-back heat stress events to global warming is presented by the authors, so this unsupported – and possibly mistaken – assumption has unequivocally led to the misinformative conclusion that the only way to save the GBR is “to rapidly reduce our [global] greenhouse gas emissions.” Also note that linking the back-to-back heat stress events to Global Warming is a highly desirable outcome, as it demonstrates that the GBR results have global relevance. Scientists need to guard against such observational bias (preferred outcome) when globalizing this study’s conclusions.
The article’s investigation of the reaction of the world’s largest reef to heat stress events apparently took a back seat to the authors’ ideological agenda: the article calls for a reduction of greenhouse gasses in order to save a reef which apparently – for now – does not need saving.
This ideological bias very likely led to the observational bias apparent in their analyses of the cover-recruitment data. The GBR has long been a cause célèbre, whereby a slew of non-scientists are routinely encouraged to share their opinions about “environmental devastation” at purportedly scientific conferences, such as Leonardo di Caprio did at the 2014 Our Ocean conference in Washington DC. Unfortunately, the Hughes et al. article and others like it gives “Scientists claim …” credence to the disinformative CBS news stories as well as to the anti-science ideologues who clamor for the Australian government to divert resources to a non-problem, while the science indicates that those scarce resources are better employed elsewhere.
The article has not been retracted despite the fact it has now become obvious it is spreading misinformation.
Our scientific journals are in an ideological war with Science
Nature and Science have long been considered the gold standards of scientific integrity, though the anecdotal evidence above begs the question whether their editors’ ideological bias is currently driving some or most of their decisions, especially when dealing with ideological hot-topic issues.
It is extremely worrying when Nature’s editors publish an editorial on systemic racism in the sciences (https://www.nature.com/articles/d41586-020-01678-x) that does not provide any supporting evidence.
The article claims “We recognize that Nature is one of the white institutions that is responsible for bias in research and scholarship.”
Hitchens’ razor – what can be asserted without evidence can be dismissed without evidence – implies the editorial can be dismissed, though a subject that clearly has a huge societal impact – racism in the sciences has enormous potential to cause societal harm – plainly warrants a study that gathers the necessary evidence. How can science come up with solutions if we don’t know the mechanisms or dimensions of the problem?
The editorial confesses that Nature’s editors are ideologically / racially biased, but is presumably written by the same ideologically / racially biased editors that are causing or tolerating the problem.
Which creates a new problem: does any researcher feel motivated to waste resources on a study that might “fail” to demonstrate systemic racism in the sciences, knowing that these same
ideologically biased editors will likely recommend not to publish it? The 1963 Rosenthal experiment indicates that by abandoning the scientific method Nature’s editors are in effect encouraging observational bias in studies: find the evidence of our ideologically preferred theory and we will publish it, don’t and we won’t.
Scientific American has taken its war on science one step further: in articles highly reminiscent of the Sokal and Grievance Study hoaxes it has published articles that any competent peer-reviewer should have balked at.
In a 2021 virtually fact-free article “Climate Anxiety Is an Overwhelmingly White Phenomenon” (https://www.scientificamerican.com/article/the-unbearable-whiteness-of-climate-anxiety/) an overtly ideological author incoherently discusses her personal, anecdotal experiences that are rampant with observational bias, and reaches conclusions largely unsupported by any evidence whatsoever.
The one source cited, a Yale study that in 2010 concluded that “minorities are equally as supportive, and often more supportive of national climate and energy policies, than white Americans” (https://climatecommunication.yale.edu/publications/race-ethnicity-and-public-responses-to-climate-change/), doesn’t support such impenetrable assertions as “The white response to climate change is literally suffocating to people of color” or “Climate anxiety can operate like white fragility, sucking up all the oxygen in the room and devoting resources toward appeasing the dominant group”, even allowing that the author may have meant “figuratively” and not “literally”.
It’s little wonder that after publishing such twaddle that Scientific American would publish an article that rejects the scientific method altogether, that is an article that suggests we should abandon the method in favor of science by consensus (https://www.natureindex.com/news-blog/the-idea-that-a-scientific-theory-can-be-falsified-is-a-myth).The article is written by an author who is apparently unfamiliar with the scientific method: he believes it works by falsifying one of two competing theories, that is by “eliminating false theories we can eventually arrive at true ones”.
This is a mistake that is commonly made by inexperienced scientists (https://pubmed.ncbi.nlm.nih.gov/23249368/), but one that is unforgivable for an author writing about the scientific method in a scientific journal; the words “null hypothesis” are never used. It is highly disconcerting that a journal purporting to be scientific publishes an article on the scientific method when neither author, reviewer or editor appreciate how the best-practice scientific method actually works.
The “science by consensus” suggested by the author implies that he thinks science is a social construct, agreed upon by scientists, so in effect Scientific American have republished the Sokal Hoax, though this time the joke is on Science.
… and the future is not looking good
All of these examples can be dismissed as “a few rotten apples”, “a problem that’s being addressed as we speak”, etc. The trends however indicate the problem is getting worse. Springer Nature
has recently retracted 44 papers from its Arabian Journal of Geosciences after concluding they were gibberish, and has flagged 400+ more over concerns about “serious research integrity” in articles published in two of its journals (https://link.springer.com/article/10.1007/s12517-021-08471-8).
The publication of articles such as “Neural network–based urban rainfall trend estimation and adolescent anxiety management” and “Influence of soil-rock mixture based on big data and music network education in mountainous areas” indicate that far too few checks and controls exist in Springer Nature’s publication process: these obviously gibberish articles imply the publisher is clueless about their technical merit, and indicate Springer Nature should not be a scientific authority for Arabian Geosciences.
These nonsensical articles can be dismissed as a prank, in effect a present-day hoaxing of the peer-review process. The consistency of the articles however – the nonsensical combination of geological themes with music, sports and social science themes – is worrying: it suggests the articles might have been written by a – fairly incompetent – AI program.
Which in turn is worrying because the most recent generation of AI language models – GPT-3 – “generates AI-written text that has the potential to be practically indistinguishable from human-written sentences, paragraphs, [and] articles” (https://www.twilio.com/blog/what-is-gpt-3), and could certainly do a much better job.
The combination of an ever-improving AI technology, able to write scientific-looking articles from whatever ideological viewpoint, coupled to an insouciant peer-review process that values “interesting, elegant, concise and compelling” stories over objective truths forms an unprecedented, existential threat to the sciences.
The University of Exeter professors’ proposal to use AI algorithms to screen articles for misinformation will put Science on a path to a human-free dystopian future, whereby computers write, peer-review, and publish ideologically-motivated “scientific” articles that realistically should filed under “Fiction”.
Alternatively, perhaps we shouldn’t censor scientists for airing dissenting views, but should encourage vigorous scientific debates, combat misinformation with better information, and insist our scientific authorities uphold the highest scientific standards.
About the author: Koen Vogel PhD received his PhD in Geology from The Pennsylvania State University, worked in the Petroleum Industry for 25 years in a variety of technical and managerial roles, and since his retirement has been engaged in pursuing his intellectual interests. Such interests include reviewing the IPCC reports, authoring a book on Geostatistics (currently in review), and editing technical proposals for start-ups. His latest paper, ‘The Role of Geomagnetic Induction Heating in Climate Change’ is available for open peer review on Principia Scientific International
Please Donate Below To Support Our Ongoing Work To Defend The Scientific Method
PRINCIPIA SCIENTIFIC INTERNATIONAL, legally registered in the UK as a company 
incorporated for charitable purposes. Head Office: 27 Old Gloucester Street, London WC1N 3AX.
Trackback from your site.