Four Major Errors Found In Research Used To Support Climate Litigation

Motivated by and building on a reanalysis of Callahan and Mankin 2022 by statisticians Abraham Wyner and Ryan Brill, I reproduced CM22 and CM25 with the authors’ data and code, and have found major errors in each

This post reports what I found and shares my methods, data, and code, so others can check my work — downloads are available at the bottom.

I argue that there is a very strong case for the retraction of both CM22 and CM25.

CM25 characterize their analysis as supporting climate litigation:

“Will it ever be possible to sue anyone for damaging the climate? Twenty years after this question was first posed, we argue that the scientific case for climate liability is closed. Here we detail the scientific and legal implications of an ‘end-to-end’ attribution that links fossil fuel producers to specific damages from warming.”

I have often argued that an index of global temperature is not a cause of weather or impacts — which is the core of the CM methodology.

However, in the detailed analysis reported in this post, I replicate their work on its own terms, and find that both papers are fatally flawed — even taking them at face value.

Specifically, I found four errors in Callahan and Mankin’s 2025 paper that when corrected eliminate their headline conclusion:

  • Temperature scale. Callahan and Mankin get results that differ widely depending on whether temperatures are expressed in Celsius, Fahrenheit, or Kelvin. They multiply each hot-spell temperature by the region’s average temperature with results dependent on where zero falls in each measure. When I repeat their calculation using each temperature scale, I get losses of $28 trillion in Celsius (as they did), $43 trillion in Fahrenheit, and $359 trillion in Kelvin.
  • Currency. Callahan and Mankin measure growth in US dollars, so they mistake a national currency crash for a collapse in national production. When growth in local currency after inflation is used under their method, their headline result is cut in half.
  • Data breaks. In 2001, the regional data for Honduras switched to a new source. In Callahan and Mankin’s data, output in 14 Honduran regions jumps 76 percent to 395 percent that year, even though the national economy did not grow. Callahan and Mankin treat that statistical artifact as real economic growth. Removing those 14 data points, out of more than 20,000, cuts their $28 trillion result by a third.
  • Lag alignment. Callahan and Mankin line up the delayed effects of hot spells after skipping years with missing data, so some “one-year” delays span several calendar years. Properly aligning years raises their headline result to $39 trillion.

A reproduction of their methodology with all four errors corrected results in totals from a $5 trillion gain to a $2 trillion loss, scattered around zero and nowhere near the published $28 trillion loss.

By the standard Nature applied in the December 2025 Kotz et al. retraction, the journals have ample justification to retract both papers, and now face a test of their publication standards.

I invite Callahan and Mankin to respond to this analysis and I will be happy to publish their response here at The Honest Broker.

The analysis in detail

Callahan and Mankin claim to attribute specific losses to specific companies. With their data and code, I find no support for that claim.

With correction of clear errors in the application of their methods or the use of equally reasonable choices in applying their method, I can turn their reported $28 trillion loss into net gains, or increase the 2025 total and the 2022 loss.

The figure below summarizes what happens when these errors are corrected — the $28 trillion loss disappears.

In each row in the figure, I add one fix to the row above and otherwise implement the CM25 methodology. I mark each version of the 2025 total with a dot and the median of each row with a black bar.

The temperature fix alone cuts the total by about 90 percent to $3 trillion. With the currency fix added, the median result turns into a $2 trillion gain.

With all four fixes, results range from a $5 trillion gain to a $2 trillion loss.

Beyond the obvious errors, Callahan and Mankin also made three methodological choices that could reasonably make differently:

  • Comparisons across countries. Callahan and Mankin compare regions from different countries with one another. That creates a problem. When Indonesia’s currency crashed in 1997, every Indonesian province looked poorer on paper at the same moment. If Indonesia also had a hot summer that year, a comparison with other countries would blame the heat for losses that the currency crash caused. So I implemented a fairer comparison: I compared each region only with its neighbors in the same country in the same year. Indonesian provinces in 1997 all lived through the same currency crash, the same government and the same national economy. When I made the comparison that way, I found no heat effect.
  • Extreme growth rates. Callahan and Mankin keep every growth rate in their data, including jumps and drops of more than 100 percent in a single year, many of them data artifacts like the Honduran break. A handful of such extremes can sway the results. As an alternative, I capped the most extreme one percent of growth rates at each end to limit the influence of outliers.
  • A forced fade. Callahan and Mankin estimate the effect of a hot spell on growth in that year and the five years after, then impose a pattern in which the harm builds for three years and — seemingly arbitrarily — vanishes by the third year after the hot spell. As an alternative, I used Callahan and Mankin’s own estimates of a hot spell’s effect on growth in each of the six years, without forcing that effect to vanish by the third year..

In the figure below, I started from the calculation corrected for the four errors detailed above and applied the three alternative methods choices, alone and in combination.

In each row, I apply a different set of alternatives to the corrected calculation and again show four versions, one for each pair of temperature and currency fixes.

Across the 32 versions showing the results based on different methodological choices, I got a net gain in 26. I found the largest effect in the fade choice: when I drop the forced fade, the median moves from a $2 trillion gain to a $10 trillion gain.

With all three alternatives, I found a net gain in all four versions. Among the 32 versions, I found no loss larger than $15 trillion.

If Callahan and Mankin had measured a real effect of extreme heat, we would expect the corrected versions to agree on a loss of some size.

I found them scattered around zero, mostly on the side of a net gain, which — taken at face value — would assign economic benefits to the emissions of the 111 companies, which might seem like nonsense.

A yardstick: the Kotz retraction

In December 2025, Nature retracted Kotz et al. which had projected a 19 percent loss in world income by 2050 from ‘climate change’.

In replication, others identified bad data from one country, Uzbekistan, and found that Kotz et al. had ignored the shared shocks of neighboring regions in their uncertainty ranges.

After fixing both, along with other problems, the authors widened their likely range for the 2050 loss from 11–29 percent to 6–31 percent. The authors judged those changes “too substantial for a correction,” and Nature withdrew the paper.

I found similar problems in Callahan and Mankin’s 2025 paper, in comparable or larger form, along with three other problems.

By the standard Nature applied to Kotz and colleagues, the journals should retract both Callahan and Mankin papers.

See more here climatechangedispatch

Header image: MedPage Today

Please Donate Below To Support Our Ongoing Work To Defend The Scientific Method

Leave a comment

Save my name, email, and website in this browser for the next time I comment.
Share via
Share via