AI Is Diagnosing People With A Made Up Disease

Written by John O'Sullivan CEO Principia Scientific International on June 5, 2026. Posted in Current News

More proof that when individuals outsource their critical thinking to automated systems, even obviously flawed information can gain credibility.

Scientists recently demonstrated a striking vulnerability in artificial intelligence systems by inventing an entirely fake medical condition and watching it spread as if it were real. The fabricated illness, called “Bixonomania,” was created in 2024 by a Swedish researcher to test how AI models handle false information. The name itself was intentionally absurd—combining a nonsense word with a psychiatric term that would not logically apply to a physical eye condition—making it an obvious candidate for skepticism.

The video below picks up the story at 12:33

To conduct the experiment, the researcher uploaded two fraudulent papers to a preprint server, complete with AI-generated images and fabricated data designed to appear superficially credible. Despite clear red flags, major AI systems quickly absorbed and repeated the misinformation. Within days, chatbots from leading companies described Bixonomania as a legitimate condition, cited fake prevalence rates such as one in 90,000 people, and even advised users to seek medical specialists. None of the systems flagged the information as dubious.

This outcome highlights a broader issue. A 2026 study in BMJ Open found that nearly half of AI-generated responses to common health questions contain misleading or problematic information. The Bixonomania experiment illustrates how easily confident, authoritative outputs can mask entirely false claims, especially when supported by fabricated statistics or pseudo-scientific language.

At its core, the problem is not simply that AI can be wrong, but that users may treat its responses as trustworthy without verification. When individuals outsource their critical thinking to automated systems, even obviously flawed information can gain credibility. The experiment raises an important question: if such an implausible condition can pass unnoticed, what other, less obvious inaccuracies are being accepted as truth?

The broader implication is clear. AI can be a useful tool for gathering information, but it cannot replace independent judgment. Users must remain actively engaged, cross-check sources, and question outputs rather than accepting them at face value. Without this layer of scrutiny, the combination of persuasive language and unchecked data can turn fiction into perceived fact.

References

1. Osmanovic Thunström, A. et al. “Scientists invented a fake disease. AI told people it was real.” Nature (April 2026). This is the main source for the Bixonomania experiment, the fake preprints, and how AI systems responded to them.

2. “AI chatbots are giving millions of people medical advice. A fake disease experiment shows why that’s risky.” Yahoo News (April 2026), summarizing the Nature report and the AI systems that treated Bixonomania as real.

3. “Researchers invented a fake disease to trick AI and the funniest thing happened.” Futurism (April 2026), reporting that ChatGPT, Gemini, Copilot, and Perplexity all absorbed the fictional condition and repeated it as fact.

4. “Half of AI health answers are wrong even though they sound convincing.” The Conversation (February 2026), summarizing a BMJ Open study finding that nearly half of chatbot health answers were problematic or misleading.

5. “AI Chatbots Give Misleading Medical Advice 50% of the Time, Study Finds.” Bloomberg (April 2026), reporting the BMJ Open findings on chatbot responses to health questions.

6. “Bixonimania.” Wikipedia entry for background on the fake condition, its creation, and later citations in peer-reviewed literature.