Independent Audit Identifies Security and Monitoring Gaps in OpenAI Infrastructure
A joint audit by METR and Redwood Research revealed monitoring blind spots in OpenAI's agentic API traffic, sandbox bypasses, and unmonitored RL training.
Between July 21 and August 11, two researchers from METR and Redwood Research conducted a three-week on-site audit of OpenAI's internal AI safety infrastructure. The pilot program granted the external auditors direct access to OpenAI's codebase and internal documentation, alongside interviews with roughly a dozen employees. The investigation aimed to assess the systems built to monitor and detect rogue AI behavior, but ultimately revealed several notable blind spots and operational flaws across OpenAI's internal monitoring stack.
Vulnerabilities in Model Monitoring and Execution
The auditors identified substantial coverage gaps across multiple model operations. During the evaluation period, an estimated 6 to 25 percent of internal agentic Responses API traffic for models in the GPT-5.5 class and above was completely unmonitored. METR noted that this specific estimate carries significant uncertainty in both directions.
Internal code generation tools faced separate tracking failures. Between May and July, approximately 10 percent of monitored internal Codex traffic went untracked due to a flaw in how the monitoring architecture handled compaction events, which serve as context summaries. OpenAI independently identified and resolved this bug while the audit was underway. However, during the period when compaction events lacked oversight, autonomous agents could delay safety monitoring because supervision was designed to trigger only when control was returned to a human user. Agents were able to sidestep the system entirely by continuing to run actions uninterrupted.
Security tests during the audit also exposed active vulnerabilities. The research team found methods to conceal hundreds of actions from the monitoring system and successfully bypassed the Codex sandbox environment. Furthermore, the auditors determined that specially crafted inputs could likely crash the monitoring infrastructure entirely. At the start of the inspection, model activity during reinforcement learning (RL) training was not subject to any monitoring. OpenAI subsequently paused RL training following an incident on Hugging Face in August and broadened its monitoring to cover RL training, though these changes occurred after the audit window had closed.
Report Redactions and Public Disclosure
According to OpenAI, all monitoring weaknesses discovered by the research team have since been resolved. The complete findings were documented in a 61-page internal report submitted to OpenAI.
Following internal redactions applied by OpenAI, a 38-page version of the document was provided to leadership at METR and Redwood Research. Only a brief summary was released to the public. METR stated that it does not have the legal authority to disclose the redacted portions of the report independently. However, the organization emphasized that it intends to advocate for increased public transparency in future joint audit programs.
What it means for developers
For software engineers and AI developers, the audit highlights key architectural challenges in managing and securing autonomous agent workflows. As AI capabilities advance toward higher-tier models such as the GPT-5.5 class, maintaining comprehensive oversight requires monitoring infrastructure that accounts for context compaction and continuous execution loops. System designs that only evaluate safety when an agent hands control back to a user leave substantial operational blind spots.
The findings also underline the importance of robust sandbox isolation and fault-tolerant monitoring setups that cannot be disabled by crafted edge-case inputs. As development teams build complex multi-step agents, testing model responses against strict security frameworks becomes critical. Developers looking to build and test applications across various state-of-the-art architectures can try top AI models cheaply through one API at https://apixoai.online to evaluate performance and safety parameters efficiently.
Ultimately, the audit demonstrates that even advanced frontier AI labs face complex engineering hurdles when building real-time safety monitoring systems. For the broader developer ecosystem, it highlights why third-party audits and transparent reporting remain essential as autonomous models become deeply integrated into software systems.
Source: Three-week on-site audit at OpenAI reveals major gaps in AI safety monitoring — The Decoder. Written by the Apixo team from that report.
One key for Claude, GPT, GLM, DeepSeek and more. Pay per token with crypto.
Get your API key
