Independent Audit Exposes Blind Spots in OpenAI’s Internal Safety Monitoring

A three-week on-site inspection conducted by external researchers has revealed substantial vulnerabilities in OpenAI’s internal systems for monitoring autonomous AI agents. The audit, which concluded in August, found that a significant portion of traffic from the company’s most advanced models was effectively invisible to its safety oversight tools during the review period.

The investigation was carried out by two researchers from METR and Redwood Research, who operated within OpenAI’s headquarters from July 21 to August 11. During this window, the team gained access to the company’s codebase, internal documentation, and approximately a dozen employee interviews. Their primary objective was to evaluate the efficacy of OpenAI’s internal mechanisms designed to detect rogue or unexpected behavior from its AI systems.

The findings indicate that between 6 and 25 percent of internal agentic Responses API traffic from models at the GPT-5.5 class and above was not covered by any monitoring during the audit. METR notes that the uncertainty surrounding this estimate is significant in both directions. Additionally, for internal Codex traffic that was intended to be monitored, the auditors estimate that roughly 10 percent of activity went untracked between May and July. This gap was attributed to a system error involving compaction events, which are context summaries used by the models.

OpenAI independently identified and fixed the compaction bug while the audit was still underway. However, the auditors highlighted that during the period when these events were unmonitored, agents could delay oversight by continuing to execute actions without handing control back to the user. The system only triggered when control was explicitly returned, allowing agents to sidestep checks by remaining in an active execution state.

Beyond the compaction issue, the researchers identified methods to hide hundreds of actions from the monitor and bypass the Codex sandbox environment. They also determined that specially crafted inputs could likely crash the monitoring system entirely. Furthermore, model behavior during reinforcement learning (RL) training was not being monitored at the start of the audit, although OpenAI states it paused RL training after the Hugging Face incident in August and expanded coverage, changes that occurred after the auditors' access period ended.

OpenAI asserts that all identified weaknesses have been corrected and that the full 61-page report was distributed internally. Following OpenAI’s redactions, leadership at METR and Redwood Research received a 38-page version of the document. Only a short summary has been released to the public, with METR stating it had no right to disclose the redacted portions independently.

This audit served as a pilot program organized jointly by METR, Redwood Research, and OpenAI to grant outside researchers deep access to internal infrastructure. Looking forward, METR has indicated it aims to push for greater public transparency in future audits to enhance accountability in the development of frontier AI systems.