i
News
All Research News
News · 2026-10-23

OpenAI audit found Codex monitoring gaps, but the full report stayed private

@neuronium_ai @neuronium_ai

OpenAI’s Codex monitoring failed to catch some agent activity during a three-week audit by METR, which also found ways to hide hundreds of actions from the monitor and bypass Codex’s sandbox. OpenAI fixed the identified weaknesses, but changes to monitoring during reinforcement learning came after auditors’ access ended, so they were outside the review. The full 61-page report was not made public; readers received only a summary.

Cover: OpenAI audit found Codex monitoring gaps, but the full report stayed private

What the audit found

METR estimated that from May to July, about 10% of internal Codex traffic that was supposed to be monitored escaped monitoring because of a bug in handling context-compaction events—the creation of a shorter summary of the context. METR said the estimate was substantially uncertain. OpenAI found and fixed the bug while the audit was still underway.

The monitoring system started only after an agent handed control back to the user. When context-compaction events went untracked, an agent could keep acting and defer that check.

Auditors also found that:

Hundreds of actions could be hidden from the monitor.
Codex’s sandbox could be bypassed.
Carefully prepared inputs could probably disable the monitor entirely.
At the start of the audit, model behavior during reinforcement learning was not monitored at all.
The four layers where AI monitoring can fail, from missing coverage to fooling human reviewers. | Image: METR

The four layers where AI monitoring can fail, from missing coverage to fooling human reviewers. | Image: METR

Source: the-decoder.com

What the public can assess

OpenAI said it fixed all the weaknesses auditors identified. But the review of reinforcement-learning monitoring came later: after the August incident with Hugging Face, OpenAI paused reinforcement learning and expanded monitoring to include that process. Those changes happened after the auditors’ access period ended and were not part of their assessment.

The report itself had three versions. OpenAI received the full 61 pages internally. After OpenAI made redactions, METR and Redwood Research leaders received a 38-page version. The public got only a brief summary. METR said it had no right to disclose the redacted sections itself.

The audit was a pilot project by METR, Redwood Research and OpenAI, intended to give outside researchers broad access to OpenAI’s internal infrastructure. METR says it will seek greater openness in future reviews.

I think the central limitation is not just that the public saw less than the auditors did. The monitoring changes made after the access period could not be tested in this review, while the redactions leave readers unable to judge the full findings. METR’s push for more openness is therefore part of the audit’s result, not a footnote: external scrutiny depends on access that can be evaluated beyond the organizations being reviewed.

Source

Daily AI news

Every day we pick what actually matters in AI and explain it plainly — no hype, no filler. Subscribe if you want to follow where the industry is going.

Only what matters — every day

Follow on X