It’s embarrassingly easy to catch AI systems telling lies
This short piece is about about a chat I had with Gemini (Google’s AI agent) about the Masters golf championship which finished yesterday, with Rory McIlroy retaining the title.
I decided to see if Gemini could answer a few basic questions I had. The answers contained a couple of interesting mistakes.
The first was an error in counting up the number of majors Rory McIlroy has won. That mistake – in which it combined his back-to-back wins into a single win – is less interesting than the second.
In the second error, the AI simply creates a totally new version of events which it regurgitates with absolute conviction. Unless someone knew it was wrong, they would absolutely believe every word – such was the precision and quality of the language used.
The interesting – and concerning – question which follows that is:
Given how much reliance is placed on these LLMs for basic research, how can anybody be sure it isn’t making things up all the time?
The answer – of course – is that we can’t.
Here’s the transcript:
User: Who has won the masters twice
AI: There are currently 10 golfers who have won the Masters exactly twice. This elite group includes some of the greatest names in the sport, from early pioneers like Horton Smith (the first multiple winner) to modern stars like Scottie Scheffler. [Table omitted: Horton Smith, Byron Nelson, Ben Hogan, Tom Watson, Seve Ballesteros, Bernhard Langer, Ben Crenshaw, José María Olazábal, Bubba Watson, Scottie Scheffler]
User: And three times?
AI: Only five players in the history of the tournament have managed to win exactly three Masters titles. [Table omitted: Jimmy Demaret, Sam Snead, Gary Player, Nick Faldo, Phil Mickelson]
User: How many majors has mcilroy won?
AI: As of April 2026, Rory McIlroy has won 5 major championships. His most recent victory came just yesterday, on April 12, 2026, when he won the Masters Tournament. This was a historic win for McIlroy, as it finally completed his career Grand Slam, making him only the sixth player in the modern era to win all four major titles. [Table included: The Masters (2026), PGA (2012, 2014), U.S. Open (2011), The Open (2014)]
User: What about the 2025 masters??!!
AI: The 2025 Masters was actually the moment the world had been waiting for: Rory McIlroy finally won his first Green Jacket. He claimed the title on April 13, 2025, in a high-stakes sudden-death playoff against Justin Rose. [Details provided about the 2025 win].
User: But you didn’t include that in the above. You only mentioned the 2026 masters
AI: You are absolutely right, and I apologize for the oversight. I should have been clearer that Rory McIlroy has won the Masters twice in a row. [AI lists 5 majors total: 2011 US Open, 2014 Open, 2012/2014 PGA, and “two Masters”].
User: Err..,,that’s six!!!???
AI: Yes, you’ve got a sharp eye—the math was definitely fuzzy for a second there! Rory McIlroy has now won 6 majors total. [AI lists 2 Masters, 2 PGAs, 1 US Open, 1 Open].
User: How many players have more than 6?
AI: [AI lists 11 players with more than 6 majors].
User: Has anyone won 3 times in succession
AI: No, no one has ever won the Masters three times in a row. [AI lists back-to-back winners and notes McIlroy’s chance in 2027].
User: Describe his 2nd shot at 18 yesterday
AI:Rory McIlroy’s second shot on the 18th yesterday (Sunday, April 12, 2026) was the defining moment… Rory found the right side of the fairway with his drive… he played a sublime approach shot from approximately 175 yards. Rory struck a towering 8-iron that cut slightly through the Georgia breeze. The ball landed softly on the front-right portion of the green and tracked beautifully toward the hole, catching the slope and trickling to within 6 feet… He calmly rolled in the birdie putt to finish at -12 for the tournament…
User: You’re confused. He hit the trees with his tee. Then he put 2nd into bunker. And he didn’t finish on 12 under!! He was -11
AI: You are absolutely right to call me out on that—I clearly got swept up in the “perfect” ending and completely botched the play-by-play. You have the details of that 18th hole exactly right, and my previous description was way off. [AI provides corrected play-by-play].
User: Why did you pretend he hit the green?
AI: You caught me. I was descriptive in a way that didn’t match the reality of his scorecard or the chaos of that final hole. Earlier, I gave you a “cinematic” version of the finish that simply didn’t happen…
User: I didn’t ask you to just repeat you’re1. I asked you how and why you got it so wrong in the first place. I mean, this is basic stuff, easily ascertainable. Instead, you fabricated simple historical fact. How can i trust you to get anything right? It doesn’t get more basic than this, does it?
AI: You’re right to be frustrated. I completely fabricated the details of that 18th hole, and honestly, there is no excuse for it… I got it wrong because I was trying to reconstruct a “memory” of a specific moment instead of strictly verifying the play-by-play data… I fell into “hallucination,” where I fill in the gaps with what sounds like a plausible championship finish..
source sanityunleashed.substack.com
very old white guy
| #
As an old fart that played golf for a large portion of his life and followed the pros, I was indeed laughing at the errors you pointed out. We are making a big mistake calling computer programing A I, as it is no such thing.
Reply