Sloppy science, shortcuts, and COVID-19

Written by Research Outreach on August 8, 2023. Posted in Current News

The COVID-19 pandemic has led to one of the most public incidences of a phenomenon known as sloppy science

Shortcuts and simplifications were made during scientific research, which led to the public declaration of incorrect statements or ‘truthies’.

Professor Michael Lissack, from the College of Design and Innovation at Tongji University in China, and Brenden Meagher from nonprofit Jhpiego study this worrying trend.

The researchers explain the concept of slodderwetenschap and advise how good research design and rigorous interrogation can work together to prevent sloppy science.

Over the last decade, there has been growing awareness of slodderwetenschap – a Dutch term that means sloppy science.

Sloppy science occurs when shortcuts are taken by the scientific community in favour of reaching and proliferating preferred results. Sloppy science includes research riddled with faults, such as flawed experiments, incomplete analyses, and factually inaccurate reporting of results.

In its most extreme form, sloppy science can undermine the credibility of science due to unreliable findings and incorrect extrapolations. But, sloppy science doesn’t just refer to the original scientific idea.

Sloppy science also occurs when unreliable results are reported as ‘true’, presented to others as fact, and then used for decision-making.

Expert in the field, Professor Michael Lissack at the College of Design and Innovation at Tongji University, Shanghai, China, succinctly defines slodderwetenschap: ‘Sloppy science involves presenting ideas as conclusive facts without adequate scientific foundation.’

Sloppy science cements itself in our knowledge channels through declarations of purported authority rather than through academic merit. Lissack says ‘It’s dangerous to tolerate sloppy science. When we allow sloppy science to go uncorrected, we implicitly endorse misinformation.’

The birth of slodderwetenschap

While shoddy science or pseudoscience isn’t a new concept, slodderwetenschap – as the modern defined concept it now is – first arose in response to the Stapel Affair in 2011. The Stapel Affair was a case in the Netherlands where a prominent research psychologist, Diederik Stapel, forged fake data to produce near-perfect results for his students at Tilburg University.

The Dutch science community coined the term slodderwetenschap to describe both the research and the attitude of the politicians, journalists, peer reviewers, and grant providers who accepted these results.

An extreme focus on getting results led to these mistakes and shortcuts being tolerated – essentially condoning academic fraud.

Numerous other cases of sloppy science have occurred over the last decade. In response to this troublesome epidemic of slodderwetenschap, Lissack and Meagher highlight the dangers of an overly results-driven culture – in articles that illuminate practices that increase the proliferation of sloppy science.

The researchers identify that sloppy science is propped up by institutional practices that collectively have failed to stop its proliferation. For example, a general reluctance to discuss opposing theories – particularly around global warming – and overlooking mistakes, such as mislabelling samples for cancer research or tentative results as ‘facts’.

The worst of these, they suggest, is an over-reliance by science journals on peer review.

Sloppy science and COVID-19

Lissack and Meagher argue that the greatest production of sloppy science – in its most prolific and public iteration – has occurred during the COVID-19 pandemic. We had front-row seats to witness the media reporting claims of a breakthrough made one day, then dismissed the next.

It’s one of the first occasions the public has been able to clearly see how messy the scientific process can be – when it’s done sloppily.

One of the more public facets in the swirling whirl of COVID-19 misinformation was the continuing role of Dr Anthony Fauci, the Chief Medical Officer to the President of the USA.

Fauci insisted that his pronouncements of the moment, such as suggesting it would only take 15 days to slow the spread of the virus or that masks were ‘unnecessary’, were ‘science’ and as such not to be questioned.

Yet, the main method of science is to question. Skepticism should always be the default position.

Fauci was abusing his claims of expertise and in the process helping to erode the public’s trust in science itself.

The researchers argue that one of the drivers of sloppy science is that people find it hard to accept results that are a work in progress; they much prefer the neatness and superficial completeness that often comes with incorrect work.

It can mean that shortcuts are taken – and alternatives are ignored because they cause disruption.

Results that are desired are often declared correct due to political and financial pressures or even fears. This culture involves accepting storylines that are presented without further examination (eg, Fauci’s ‘I am Science’).

Naïve acceptance can cause real harm – especially when the initial claims need to be qualified or are disproved. What arose during the COVID-19 pandemic was the increasing proliferation of unsound science, which meant policy leaders – misled by misinformation – made terrible decisions with devastating ramifications.

The debate about the longevity of lockdowns as a means of dealing with COVID-19 and the seemingly deliberate suppression of the role of natural immunity post-infection stand out as two prominent examples.

Why does sloppy science exist?

As Lissack and Meagher say, good science requires us to constantly ask questions and interrogate findings. Within sloppy science, objective questioning is omitted – and shortcuts are taken to achieve subjectively desired results.

As humans, we have a limited cognitive capacity. To cope with what the world throws at us, we often try to limit the information we pay attention to. By reducing the number of unrelated items we attend to, we can avoid being overwhelmed and can, in theory at least, devote sufficient cognitive resources to allow narratives and meanings to emerge.

Another coping strategy to help limit our cognitive load is finding relationships between the things we do engage with. Yet, the researchers argue that these load-limiting activities can also increase the risk of slodderwetenschap.

When we make choices about what to deal with we often choose simple over complex, and shortcuts over engagement, leading us to act wrongly.

The 3Ts

Such cognitive dissonance is related to a concept Lissack and Meagher refer to as the 3Ts. The 3Ts – Truthies, TL;DR, and TCUSI (more on this below) – are persistent throughout every aspect of human inquiry and can negatively influence science.

The first T – truthies – is a phrase coined by comedian Stephen Colbert in 2005. It means that an individual believes what they say is right and nothing anyone else says is true. It’s the act of preferring concepts or facts that a person would like to be true over what has actually been proven to be true.

In reality, we cannot process the whole truth; the truth we have access to is a narrative created by our minds. It means we end up seeing what we want to see, as this takes up less cognitive energy than creating a new story.

The second T relates to the phrase ‘too long; didn’t read’ – also known as TL;DR. This refers to our short attention spans and means decisions are often made based on truthies, titles, or context rather than someone reading a whole article.

The growth of the internet and search engines has exacerbated the existence of TL;DR. The researchers ask a pertinent question: if people are unwilling to process information, how can we expect sound judgments about science and its reporting?

The final T is TCUSI, which stands for ‘too complex used shortcuts instead’. TCUSI refers to using simplification to try and increase efficiency. However, Lissack and Meagher argue that this is often at the expense of context and fit.

The complexity of our world is reflected in ambiguity and uncertainty – qualities we process as ‘unknowability’. However, this unknowability threatens our perceived control over and understanding of the world – humans find it hard to tolerate. Simplicity is preferable to complexity.

People are naturally inclined to simplify complex things to maintain the illusion of stasis, where opportunities appear predictable, context is controllable, and the emergence of the new is non-existent.

It leads to people erroneously ignoring complicated meanings and contexts, discarding multiple perspectives or intersectionality in favour of oversimplification. TCUSI approaches fail to tackle complex problems, challenges, and new concepts.

By assuming a stasis that seldom exists, TCUSI introduces fragility into one’s ability to deal with change.

Together, the 3Ts of truthies, TL;DR, and TCUSI – in the words of the researchers – ‘haunt modern life’ and can lead to sloppy science. But how do we avoid this? ‘Healthcare and science communications work best when the context and assumptions which give research meaning are made as transparent as possible’, says Lissack.

Effective science communication engineers two-way conversations between the public and the scientists when sharing authentic knowledge. It also acknowledges and respects the backgrounds and contexts that people use to frame and understand ideas.

However, there are more pitfalls to avoid.

Seven triggers of sloppy science

Lissack and Meagher outline seven critical mistakes where sloppy science can creep into the scientific process:

1. Jumping straight into giving explanations for unexpected observations. The impulse to be the first to obtain results makes shortcuts tempting.
2. Disregarding variables that could be of importance to the research. Selecting suitable variables is critical for good science – it’s a process that should not be rushed.
3. Not correctly considering the context of the experiment. For example, how the research relates to the real world.
4. Inflexible modelling. For example, only using a single model instead of an open-ended model to determine outcomes.
5. Making bad sampling assertions. Like applying statistical functions across populations as a whole – when they may only apply to specific subsets.
6. The overuse of labelling and categorisation. Mislabelling a category too quickly and without proper thought to meaning.
7. Prematurely applying scientific findings. Dangerously utilising early results; they may not be well understood or used by others to make misinformed decisions.

How can we prevent it?

The challenges presented by the COVID-19 pandemic gave rise to the most public display yet of widespread sloppy science. Reliance on truthies resulted in conflicting information, which adversely undermined public trust in scientists and officials – plus their policies, advice, or guidance.

The researchers say that a key takeaway from the explosion of sloppy science during the pandemic is exposure of the public’s (and the media and political leaders’) inability to deal with conflicting ‘facts’.

The media, politicians, public health officials, and scientists should not make public statements regarding the facticity of research – unless it’s been properly interrogated with its assumptions, context, and limitations included in the discussion.

We need to avoid a reliance upon assumptions as ‘truth’ to slosh away shoddiness from scientific endeavours. To promote sound science, ongoing vigilance is required by all parties involved.

Lissack says, ‘Science is a conversation, and sometimes a shouting match, between beliefs and evidence, though often the voices we most want to hear are barely audible.’

We need to ensure that rigorous, not sloppy, science is discernible at the highest volume – and drowns out the insidious claims of slodderwetenschap.

See more here researchoutreach.org

Header image: Forbes

Some bold emphasis added

Please Donate Below To Support Our Ongoing Work To Defend The Scientific Method

PRINCIPIA SCIENTIFIC INTERNATIONAL, legally registered in the UK as a company incorporated for charitable purposes. Head Office: 27 Old Gloucester Street, London WC1N 3AX. 

Trackback from your site.