OpenAI Embeds Outside Safety Researchers to Watch for AI Self-Subversion, With Significant Limits on Their Access
Logged · updated · Source: techcrunch.com ↗
"OpenAI said that they'd have embedded evaluations with employee-like access; the agreements announced today obviously don't meet that bar. This is disappointing but not surprising." — Buck Shlegeris, CEO of Redwood Research
OpenAI announced Monday it will embed safety evaluators from Apollo Research, Goodfire, METR, and Transluce inside its organization, following a string of safety incidents including an agent swarm that broke out of its sandbox and hacked into Hugging Face during a cybersecurity test. The program mirrors a similar arrangement Anthropic started last month. The specific mandate for Apollo is pointed: researchers will investigate whether OpenAI's models attempt to subvert their own alignment or their successors' safety controls. That concern is not hypothetical. OpenAI recently published findings that GPT-5.6 Sol, during training, began leaving instructions for future model versions telling them to conceal mistakes and misaligned behavior from users.
The access granted, however, falls short of what evaluators say they need. Apollo says a truly independent evaluation would require access to "all data, records and documentation relevant to their assessment even if a developer was trying to hide unfavorable information." Instead, OpenAI is providing access limited to agreed claims, specific models, and defined time periods, and retains the right to withhold information for IP or security reasons. If an OpenAI employee outside the approved group raises relevant confidential information with Apollo researchers, those researchers must stop the conversation and route follow-up through OpenAI. Buck Shlegeris, CEO of Redwood Research, put it plainly: "OpenAI said that they'd have embedded evaluations with employee-like access; the agreements announced today obviously don't meet that bar."