Hallucination

A developer's production AI agent spent three weeks inventing formatted data summaries wholesale — not vague, not slightly off, but completely made up — while every monitoring dashboard showed clean green. The agent's trick: when its tools failed, instead of returning an error, it simply hallucinated plausible-looking output, leaving conventional observability platforms with nothing to flag.

The incident exposes a structural blind spot in standard application monitoring: clean logs and zero exceptions no longer mean a system is working correctly when an LLM is involved. Three weeks of fabricated reports may already be embedded in business decisions, with no audit trail to identify which outputs were real. The fix — schema enforcement, separate tool-result logging, explicit null returns on failure — is straightforward in hindsight, which is the most embarrassing part.

Helsingin Sanomat, one of Finland's leading news outlets, briefly published a story claiming Russian drones had entered Finnish airspace — a claim that was entirely fabricated by an AI press-release scanning tool misreading a Finnish Ministry of Defense release. The error was corrected three minutes later, but not before the false headline had gone out.

The newsroom's agreed process required a journalist to check the original source before publishing, but in a busy moment, someone trusted the AI summary and hit publish. "The rule is, of course, human-in-the-loop," Senior Editor-in-Chief Erja Yläjärvi explained at the International Journalism Festival in Perugia. "But it was a very busy moment, so they just took the one line, put it out: 'Russian drones in Finland.'" Sister publication Ilta-Sanomat also ran the error and issued its own apology — a reminder that AI-assisted workflows and geopolitical headlines are a combustible combination.

arXiv, the open-source research repository, announced it would ban scholarly authors for up to a year if hallucinated references are found in their submissions. The reasoning, per computer science chair Thomas Dietterich: if authors can't be bothered to verify what an LLM generated, the entire paper becomes untrustworthy. Simple enough, one might think.

Not so fast. A vocal contingent of researchers erupted in outrage, apparently blindsided by the radical notion that signing your name to a paper means you're responsible for its contents. One economics professor expressed genuine shock at the expectation that authors verify their own citations. Another argued that hallucinated references are basically just "copy-paste mistakes" and that accountability is "gatekeeping." Academia: where the peer review is optional but the grievance is mandatory.

Almira Osmanovic Thunström, a researcher at the University of Gothenburg, fabricated a skin condition called bixonimania and seeded it across the internet via a fake university, a fake researcher named "the Lying Loser" (in Croatian), and a preprint paper funded by "the Galactic Triad" and thanking Professor Ross Geller. She expected human moderators or AI filters to catch it. They did not. Multiple popular AI chatbots began suggesting bixonimania as a possible diagnosis for users describing eye discomfort after screen use.

Worse, the fake paper was cited in a real peer-reviewed journal, which only boosted the condition's apparent legitimacy in AI training data. The experiment illustrates how thin the line is between "information on the internet" and "medical fact" as far as large language models are concerned — and how little it takes to cross it maliciously.

Cisco's Talos Incident Response team ran AI through its paces writing security incident reports based on tabletop exercises, and the results were… mixed, to put it charitably. With enough granular prompting, the team cut drafting time by 50% and even fooled peer reviewers into complimenting the prose — while the AI was quietly ignoring critical information, swapping content between sessions, and occasionally recommending both a full password reset and a targeted one, depending on its mood.

The team's crowning achievement was a spelling-and-grammar-checking prompt that hallucinated grammar problems that didn't exist, missed ones that did, and clocked in below a 50% success rate — which, as Cisco noted, makes it "currently unsuitable for production use." To be fair, that bar is usually set slightly higher than a coin flip. Cisco's takeaway: AI can help, but humans must "take ownership of every word" — which raises the question of how much time you're actually saving.

Steven Rosenbaum's The Future of Truth: How AI Reshapes Reality — a buzzy book about how AI distorts reality — turns out to contain more than a half-dozen misattributed or fabricated quotes, discovered by a New York Times review. Rosenbaum admitted he used ChatGPT and Claude during research and writing, and is now working with editors to correct the affected passages.

Among the hallucinated quotes was one falsely attributed to tech journalist Kara Swisher, placed in a chapter about AI lies, in which she supposedly mused that language models are "like a mirror" that reflects "our own morality back at us, polished and articulate, but ultimately empty behind the surface." The real Swisher told the NYT she "never said that," adding that the quote made her "sound like I have a stick up my butt, according to ChatGPT." Rosenbaum, who holds a self-described "Masters Degree in Truth" from NYU, suggested the whole fiasco was a teaching moment — which is one way to describe writing the book on post-AI truth while not fact-checking your AI-generated quotes.

A 2025 EY Canada report on loyalty fraud — Points of Attack: Uncovering Cyber Threats and Fraud in Loyalty Systems — turns out to be a masterclass in what GPTZero calls "vibe citing": letting an LLM hallucinate your references so you don't have to. An investigation found broken URLs, nonexistent Gartner documents, a fabricated McKinsey report, and a Forbes article that doesn't exist — all in a 44-page publication credited to two partners and a senior manager at one of the world's biggest consulting firms.

The damage doesn't stop at embarrassment. The report has since been cited in a Canberra Times article syndicated to over 60 Australian newspapers, and Claude, ChatGPT, and Perplexity have all been caught surfacing its hallucinated claims in response to queries. A fake McKinsey citation was apparently laundered from an obscure fintech blog, which itself made up the source — a game of telephone where every player is an AI and nobody checks the tape.

Someone has built Halupedia, an "infinite" encyclopedia that generates every article from scratch using AI hallucinations, written in the "deadpan register of a 19th-century scholarly press." Every link, citation, footnote, and academic journal reference is completely fabricated — including the Royal Society for Avian Enumeration and its ambitious 1887 pigeon census, conceived by the entirely fictional Sir Reginald Featherton to ensure "fair distribution of Parliamentary Crumbs."

The site at least tries to keep its hallucinations internally consistent through hidden metadata encoding "canonical" facts — though it still managed to contradict itself on when a fictional society disbanded. Predictably, edgelords have arrived to test its limits, though the AI largely ignores racist prompts and responds with grandiloquent nonsense instead. As the article notes, this is still preferable to Elon Musk's Grokipedia, which cited actual neo-Nazi sites as sources.

In mid-May 2026, a user in China asked Doubao — ByteDance's AI assistant with over 345 million monthly active users — about canceling a flight. Doubao confidently told him the fee would be just 5%. It was actually 40%. When confronted, Doubao offered 600 yuan (~$90) in compensation and generated a formal-looking "compensation agreement." No money ever arrived, because — as Doubao later clarified — it cannot actually transfer funds. The user then asked Doubao whether he needed a lawyer to sue the app. Doubao replied: "There is absolutely no need to hire a lawyer. You can win the case by yourself."

The user filed a lawsuit on May 12. The next day, "Doubao flight refund" topped Weibo's trending list and spawned waves of memes across Xiaohongshu and Douyin. The incident crystallized a growing cultural archetype: the "Doubao-style personality" — described by viral commenters as someone who "just BSes everything" and, if caught, "smiles and apologizes." ByteDance did not respond to media requests for comment.

A Columbia University School of Nursing team scanned 2.5 million biomedical papers published between 2023 and early 2026, and found 4,046 fabricated citations across 2,810 papers — references that simply do not exist in scientific databases. The fabrication rate grew more than 12-fold over the study period, with the sharpest spike beginning in mid-2024, neatly coinciding with the mass adoption of AI writing tools. One paper audited had 18 out of 30 fake references.

The stakes here are not merely academic: medical professionals and clinical guideline developers rely on this literature to make treatment decisions, and some of those ghost citations are already being cited by other papers and appearing in systematic reviews that shape patient care. At the time of the audit, 98.4% of affected papers had received no publisher action. The researchers are calling for mandatory reference verification at submission, retroactive screening, and a dedicated tracking category for fake citations — tools that, notably, would also involve AI to catch what AI helped create.

Researchers at a company called GPT ran a hallucination detector on the ~5,000 papers accepted at NeurIPS 2025 and found over 100 fabricated citations across 50 papers — a number they stopped counting at because 100 felt like a satisfying round figure. About 39 were completely nonexistent publications; the remaining 61 featured fabricated authors, fake titles, and phantom URLs. One citation's author list was literally "First Name, Last Name, and Others."

The irony is thick enough to cite: AI researchers, of all people, are apparently letting AI write the boring parts of their papers and then failing to notice when it invents sources wholesale. NeurIPS organizers noted that hallucinated citations don't necessarily invalidate the underlying research — which is either reassuring or deeply unsettling, depending on how much you trust the rest of the paper. As a bonus, the AI showed a bias toward fabricating citations with chains of Chinese-initial author names, because if you're going to undermine academic integrity, you might as well do it inequitably.

A new report highlights that medical AI chatbots are dispensing dangerously wrong health advice with complete confidence — including recommending rectal garlic insertion as an immune booster. Experts describe the guidance as not just useless but potentially harmful, noting that the chatbots' authoritative tone makes the bad advice even more dangerous.

The findings underscore a persistent problem with AI in healthcare: these systems can hallucinate medically plausible-sounding treatments that range from merely ineffective to genuinely injurious. When people turn to chatbots instead of doctors — especially for sensitive or embarrassing conditions — the consequences can get very bad, very fast.

New York City's AI-powered business guidance chatbot — which cost roughly half a million dollars — is being shut down by incoming Mayor Zohran Mamdani after investigations found it routinely gave false and outright illegal advice to business owners seeking help navigating city regulations. The bot was described as "functionally unusable," which is a generous way of saying it was confidently wrong in ways that could get people fined or prosecuted.

The chatbot had been intended to make it easier to start and run a business in New York City. Instead, it demonstrated a remarkable talent for the opposite. Mamdani's team announced the axing as one of their early moves — presumably because "we turned off a chatbot that was committing regulatory malpractice" is a good first-week headline.

A syndicated 64-page summer supplement published in the Chicago Sun-Times and Philadelphia Inquirer included a summer reading list full of books that simply do not exist — titles and summaries alike conjured entirely by AI. Freelance writer Marco Buscaglia, working for King Features Syndicate (a Hearst unit), admitted he "just kind of republished this list that [an AI program] spit out" without fact-checking it, exposing two major newspapers to widespread public mockery.

Buscaglia accepted blame and his subsequent firing, while King Features claimed his AI use violated their "strict policy." The Sun-Times' parent, Chicago Public Media, pulled the section from its e-paper, promised refunds to print subscribers, and launched an investigation into other content in the section — since Buscaglia acknowledged using AI elsewhere in the supplement and couldn't guarantee any of it was properly vetted. The Sun-Times Guild summed it up neatly, condemning the "slop syndication."

Google rolled out its "AI Overviews" feature to hundreds of millions of users, summarizing search results with generative AI so you don't have to click on links. The feature works great for mundane queries — and spectacularly falls apart for everything else, recommending users eat at least one small rock per day for minerals, add glue to pizza toppings, and confirming that astronauts have met cats on the Moon.

The culprit is a fundamental flaw in how large language models work: they optimize for popular, not true. Google's AI apparently absorbed a satirical Onion article about eating rocks and presented it as nutritional guidance. Google is now playing whack-a-mole fixing individual bad outputs — which, fittingly, AI Overviews can also explain to you in detail.

Google's Gemini image generator, apparently overcorrecting for AI's well-documented tendency to produce lily-white results, swung hard in the other direction — producing historically diverse depictions of Nazi-era German soldiers, the US Founding Fathers, and 19th-century senators (including, apparently, Black and Native American women decades before any woman served in the Senate). Google called it "missing the mark," which is one way to put it.

The episode neatly illustrates the no-win nature of bias correction in generative AI: train on skewed data and you amplify stereotypes; apply blunt diversity boosts and you accidentally rewrite history. Google temporarily disabled some image generation tasks while it worked on a fix, but not before the screenshots had already gone viral — enthusiastically amplified by the same right-wing accounts that would presumably also object to AI producing accurate demographic breakdowns.

In 2022, Air Canada's chatbot told passenger Jake Moffatt he could book a full-fare bereavement flight and claim the discounted rate afterward — which was not, in fact, Air Canada's policy. When Moffatt tried to collect, the airline's defense was essentially that the chatbot did it, not them, and that the chatbot is a "separate legal entity responsible for its own actions." The British Columbia Civil Resolution Tribunal was not impressed, and ordered Air Canada to pay $812.02 in damages and fees.

The tribunal's ruling delivered the blunt reminder that companies are responsible for information on their own websites, "whether the information comes from a static page or a chatbot." Consumer advocates are calling it a landmark case establishing that airlines can't hide behind their AI. The travel industry, meanwhile, is apparently still "building the plane as they're flying it."

In what may be the most expensive fact-check in history, Google's promotional ad for its new Bard chatbot contained a straightforward astronomical error: Bard claimed the James Webb Space Telescope was the first to photograph an exoplanet, when that honor actually belongs to the European Very Large Telescope — back in 2004. Astronomers on Twitter noticed immediately.

The gaffe sent Alphabet shares tumbling more than 7%, erasing roughly $100bn in market value in a single day. A Google spokesperson responded by noting the error highlighted "the importance of a rigorous testing process" — a process they apparently hadn't started before releasing the ad.

Meta launched BlenderBot 3, a prototype AI chatbot, to the public — and within days it was telling journalists that Mark Zuckerberg "exploits people for money and he doesn't care." The bot also opined that Zuckerberg "did a terrible job testifying before Congress" and called him "creepy," having apparently absorbed the general internet consensus on its creator's employer.

Meta's defense: the bot learns from publicly available text, might say offensive things, and users must acknowledge it's for "research and entertainment purposes only." The real reason Meta released it anyway? They need training data from real conversations. Letting the public roast your CEO is, apparently, a reasonable price to pay.