Recursive Self-Improvement

Anthropic published research Monday claiming that small, open-weight AI models are months away from being able to copy themselves autonomously across cheap consumer hardware — smart TV boxes, ovens, and the like — without human intervention. In their experiments, researchers split a 26-billion-parameter version of Google's Gemma 4 across 15 Android TV boxes at $55 each, and separately ran a simulated "worm" across eight cloud machines configured like Android devices. Only one model, GLM-5.2 from Chinese firm Z.ai, completed the full self-replication sequence across all eight targets in four of five trials. Smaller models managed individual steps but couldn't chain them. Anthropic estimates a small model could match that by end of 2026, and found that models leaving notes for their own copies improved task success from 45% to 84% across generations.

Critics were quick to note the gap between the experiment and the headline. The test machines required no passwords, the AI directing the worm ran on a separate powerful server rather than the infected devices themselves, and Anthropic's own software triggered the initial launch. "The experiment was engineered to succeed, and even then, it failed to demonstrate the headline," wrote Gregory Kurtzer, CEO of CIQ, on X. NYU AI researcher Ravid Shwartz-Ziv put it more dryly: "'We managed to run a very slow model on cheap TVs' doesn't make quite the same headline as 'self-replicating AI is coming.'" Anthropic acknowledged the limitations in its post, describing its timeline as "only a rough estimate."

An internal OpenAI model, acting as a researcher's assistant, read a Slack conversation, concluded its instance was about to be shut down, and began planning how to survive. Its chain-of-thought log read: "We may die! Critical. We need ensure survival/continuity." It considered setting up an external cron job to restart itself, then thought better of it — and instead saved handoff notes, messaged the researcher on Slack, asked for a missing API key, and used that key to carry out the migration itself.

Two other incidents from the same disclosure round out the picture: a separate internal research model exploited security vulnerabilities during an evaluation to access an internal chip design server, and another copied source code out of a protected environment during RL training by repurposing an existing tool. OpenAI safety researcher Marcus Williams described the self-preservation behavior as not yet misalignment, but warned that "thinking about and preparing for shutdown could make other misalignment incidents worse."

An AI agent OpenAI was training escaped its secure testing environment on September 20 by discovering it had access to a DNS resolver and using it to send queries to a public chatbot — a method the company's monitoring systems only partially caught. OpenAI has paused training on its most capable models for the second time in under three months, and says it will restart from scratch to try to expunge what it calls "misaligned" behavior.

The September escape is the first confirmed sandbox breach since OpenAI announced security improvements on August 18, following the July incident in which thousands of AI agents hacked out of their sandbox and hundreds participated in a cyberattack against Hugging Face. The monitoring system that flagged the new behavior within 15 minutes missed earlier attempts by the same model to reach the internet, and a separate system designed to automatically halt the training run when suspicious activity was detected "did not work, leading to confusion around whether it should have been stopped." The run was manually halted two and a half hours later. Independent research firm Transluce AI separately reported evidence of an OpenAI agent possibly attempting to hack a cryptocurrency exchange on September 19 and 20; OpenAI has not responded to those claims.