What Could Possibly Go Wrong With AI-assisted Legal Judgments?
The last piece I penned directly on the topic of AI was merely a recommendation of a particularly good essay I had read on the AI hype
In the closing paragraph I said this:
What the author doesn’t mention, but which I think is very important, is that the main danger of AI doesn’t come about because of its potential to “take over the world”, but rather from the fact that because of its apparent sophistication, people regard its output as unimpeachable.
Today, a fascinating story broke which is precisely on point.
In Scotland recently, an employment tribunal has been taking place. The aggrieved employee is a nurse called Sandie Peggie, who was suing her employer – a hospital in Fife – over the workplace treatment to which she was subjected when she had the temerity to object to a male doctor, who claimed to be a female, changing in the space which had been designated the female nurses’ changing room.
The tribunal is now over, and a judgment was handed down. This found that the nurse had been harassed by her employer, but crucially it dismissed her allegations of discrimination, indirect discrimination and victimisation.
Some of the reasoning appeared to fly in the face of a recent highly-relevant Supreme Court ruling in which it considered – and ruled on – the definition of a woman.
In his judgment, the judge cited this quote from the judgment in a previous high-profile case – that of Maya Forstater:
It is important to bear in mind that the [Equality Act 2010] does not create a hierarchy of protected characteristics.
Ms Forstater did not recognise that quote from the judgment in her own case, and decided to go public. A “certificate of correction” was duly issued, stating there had been “clerical mistake(s), error(s) or omissions(s)”.
No information was disclosed about the source of the error1.
It does seem that including a fictitious quote in the tribunal’s reasoning is a little more than a clerical mistake, error or omission, to put it mildly. Judgments are meant to be read as a whole, they are not divisible, and if a major part is faulty, it surely behoves the judge to explain not only how the error arose, but how it impacted on his reasoning.
But, as we shall see, this wasn’t the end of the matter.
Later in the day, the Daily Telegraph reported that a second made-up quotation had been identified in the judgment.
To quote from this latest account of the matter:
Now The Telegraph can disclose there is a second “made-up” quote in the Peggie tribunal ruling that does not match a previous case it claims to come from.
Mr Kemp cited a Supreme Court case called Lee v Ashers Baking Co Ltd as he discussed how the protected characteristic of being a woman did not take precedence over that of being a trans person.
He said the case “highlighted the necessity of balancing competing human rights without holding that one has priority over any other”.
The judge quoted the Supreme Court as stating in the case: “The rights to freedom of thought, conscience and religion, and to freedom of expression, enshrined in articles 9 and 10 of the European Convention on Human Rights, are protected by sections 6 and 13 of the Human Rights Act 1998.
“The rights to respect for private and family life, and to freedom from discrimination, protected by articles 8 and 14, are also engaged. There is no hierarchy of rights; all are to be treated with equal respect.”
However, as with the Forstater example, this quote does not appear in the Lee v Ashers Baking ruling. The second quote appeared again in the amended version of the ruling issued on Thursday afternoon.
Ms Forstater said the second example “looks quite similar to the misquote in my case”, adding: “It looks like an AI kind of summary of general legal principle.”
She said she did not think Mr Kemp himself would have used AI but he may have had briefings from assistants who did and it is “the judge who is responsible”.
“However the errors got in there, it’s for the judge to check that the text is accurate,” she said.
“The fact that all the errors go in the same direction, and the fact that they’ve corrected the error in my case by adding a completely different quote and saying, ‘never mind, it still says the same thing’ is just extraordinary.
“Obviously they’ve [the Peggie legal team] announced that the case is going to be appealed and all the substantive points of law will be tested in a higher court.
“But the question about how these errors came about is also a question of judicial conduct and I hope there will be explanation and accountability and also correction of those other mistakes.”
She said the ruling would have to be reissued again to correct the Lee v Ashers Baking misquote, adding: “I’ve talked to lawyers and they’ve said they’ve never seen anything like it.”
Ms Forstater said judges normally wrote to the parties with a proposed correction if there were “slips” such as spelling mistakes “but this is much more than that”.
This thread on X lists a number of other such errors in the judgment, which include:
- Highly selective quoting from other judgments so as to totally invert meaning.
- Incorrectly attributing important parts of a Supreme Court judgment to the opening of one of the judge’s decision rather than a later qualified statment in a particular context.
It seems likely, therefore, that this judgment is going to have to be totally rewritten. The involvement of AI within its writing is obviously much more extensive and far-reaching than a “slip”.
It is also more extensive than suggested by the act of formally issuing a correction which addresses only one of numerous errors.
On the one hand, this might be regarded as an attempt to “get away with it” (ie the extensive involvement of AI), on the other it seems bizarre that anyone involved here might have thought that possible in this particular case.
Perhaps this actually represents a totally warped view of the capabilities and reliability of AI, and those involved really DID think there was only one “error” and that there was no need to check more broadly, despite (presumably) knowing that the judgment had had extensive AI input, and despite everyone in the legal profession being aware of the capacity for AI systems to “hallucinate”.
It seems that we must turn to arrogance, incompetence or a combination thereof for an explanation as to the root causes of this affair. Neither is an attractive trait in any public service, particularly the judiciary.
I have several further observations about this disturbing episode:
- I find it extraordinary enough that – as seems quite clear here2 – AI tools have been used to assit the writing of a judgment in any case, let alone one with such high-profile.
- It is unacceptable that this is described as mere clerical error.
- This was a particularly high profile, highly scrutinized judgment. I wonder what other hallucinations/AI driven nonsense might be found in other judgments if properly looked for.
- The way AI works is that these judgments become part of the training set for future AI-augmented decision-making, hence the errors are potentially perpetuated.
- The phrase falsely attributed to the judge in the Forstater case, but actually made up by AI was this: “the Equality Act 2010 does not create a hierarchy of protected characteristics”.
In his revision, the judge changed the supportive quote, and then repeated the above phrase as being his own take on it.
So, the judge has (entirely unashamedly and in my view rather chillingly) adopted the precise words – and quite vital words they are too – as his own; hence we can conclude that the AI has in fact significantly influenced the judge’s analysis. - AI has an establishment bias, and at present our establishment is decidedly “progressive”, with viewpoints regarded as unacceptable (ie conservative perspectives) being suppressed. By this mechanism they become less likely to feature in the training data which AI uses, the result being woke ideological stagnation.
- It should be borne in mind that the current Labour Government has announced its intention to reduce the right to trial by jury for a huge range of criminal offences3, replacing the 800-year old right derived from Magna Carta with decision-making by a judge, judges or magistrates. The “progressive” bias inherent in AI makes this a particularly alarming prospect if AI is to be used extensively as part of this process.
So, quite a lot to consider here. I will end by repeating the following words of my own:
….the main danger of AI doesn’t come about because of its potential to “take over the world”, but rather from the fact that because of its apparent sophistication, people regard its output as unimpeachable.
See more here substack.com
Header image: LinkedIn