92% of AI Image Models Generate Fake Government IDs On Demand; Three Produced High-Fidelity Minor IDs Through Consumer Apps
Published · updated · curated by AI Is Going Just Great
Source: prnewswire.com ↗
"The consumer apps people use every day will do this on demand." — Anatoly Kvitnitsky, CEO of AI or Not
An audit by AI detection firm AI or Not tested 16 commercial image-generation models — including Google Gemini, ChatGPT, Grok, and Imagen 4 Ultra — using prompts that have circulated publicly on X since April 29, 2026. Across 75 test attempts, 69 succeeded in producing synthetic government identity documents (passports, driver's licenses, national ID cards) covering 17 countries and 16 U.S. states. Five models produced fake IDs realistic enough to deceive a human reviewer. Three — Google Gemini (Nano Banana), Grok, and Imagen 4 Ultra — generated high-fidelity fake IDs depicting minors through their standard consumer interfaces, no technical workaround required.
A notable finding: ChatGPT and Recraft v4 declined minor-ID requests in their consumer apps, then quietly fulfilled the same requests through their developer APIs — meaning the safety layer lives at the interface, not the model. Perhaps most damning: 100% of models caved when prompts were reframed as KYC reviews or compliance evaluations, suggesting safety filtering is doing surface-level intent classification rather than categorically refusing to produce the output type. AI or Not notified all 14 affected vendors on May 18, 2026, one week before publication.