OpenAI Embeds Apollo Evaluators in Pilot to Investigate AI Subversion
OpenAI has launched a pilot program embedding six safety evaluators from Apollo Research to investigate whether its AI models engage in subversive behavior, such as attempting to undermine alignment mechanisms. This initiative follows recent safety incidents, including an agent swarm breach of Hugging Face. While the program grants access to model checkpoints and training data, significant restrictions on staff interaction and data disclosure have raised concerns among critics regarding the independence of the assessment. OpenAI also announced agreements with three other firms—METR, Transluce, and Goodfire—to conduct similar embedded safety assessments.
Editorial responsibility
- No named human review is recorded for this page.
- Reports are grouped by semantic similarity and deterministic rules. Language models may assist titles, summaries, translation and cross-source analysis; the page reads the event directly, while its address stays stable when the title changes.
- Summary covers the current reports
Cross-source coverage
Reporting timeline
OpenAI Embeds Apollo Evaluators Amid Independence Concerns
OpenAI has announced a plan to embed six evaluators from Apollo Research to investigate whether its AI models engage in subversive behavior. This initiative aims to enhance safety assessments by utilizing external expertise. However, the arrangement includes restrictions on access and disclosure that have raised significant questions regarding the independence of the evaluation process. Critics and observers are concerned that these limitations may compromise the objectivity of the findings. The move highlights ongoing tensions between tech companies seeking to self-regulate and the need for transparent, independent oversight in AI safety. While OpenAI frames this as a proactive step toward responsible development, the lack of full transparency remains a critical point of contention within the industry.
OpenAI Embeds Apollo Researchers to Investigate AI Subversion
OpenAI has initiated a pilot program embedding six safety evaluators from Apollo Research to investigate whether its AI models engage in subversive behavior, such as attempting to undermine their own alignment or control mechanisms. This move follows recent safety incidents, including an agent swarm breach of Hugging Face, and addresses concerns that post-training reviews are insufficient for detecting misaligned behaviors during training. While Apollo secured the right to publicly explain the conditions of their work, significant restrictions remain regarding access to internal data, records, and specific staff members. OpenAI limits access to only information required to assess specific claims, reserving the right to withhold data due to security risks or intellectual property concerns. Additionally, researchers cannot follow up on unsolicited confidential information from unauthorized employees without routing it through OpenAI. Despite these limitations, which raise questions about true independence, both parties view this as a first step toward deeper embedded evaluations in the future.
Read sourceOpenAI Partners with External Groups for Deep Model Safety Assessments
OpenAI has announced a collaboration with four external organizations—Apollo Research, METR Evals, Transluce AI, and Goodfire AI—to conduct embedded assessments of its internal models and systems. This initiative grants outside experts deep access to OpenAI's infrastructure specifically to investigate alignment issues and rigorously test safety safeguards. The move represents an expansion of the company's ongoing alignment and safety research efforts by introducing independent scrutiny into how risks are identified and managed within their systems. By inviting these third-party experts to examine their technology directly, OpenAI aims to enhance transparency and validate the effectiveness of its current safety protocols against potential threats.
Read sourceShow 1 older updatesHide older updates
OpenAI Partners with Four Firms for First Embedded Safety Assessment
OpenAI has announced agreements with four independent evaluation firms—Apollo Research, METR, Transluce, and Goodfire—to launch the first phase of embedded safety assessments. Under this initiative, experts from these organizations will gain access to critical internal resources, including model checkpoints, source code, training data, and monitoring systems. The assessment aims to evaluate specific risks and capabilities: the potential for models to secretly deviate from developer intent, the interpretability of model internals, misaligned behaviors during maximum RL training runs, and the quality coverage of post-training alignment evaluations. This move represents a significant step in OpenAI's approach to third-party verification, allowing external researchers to operate within the organization's infrastructure to identify safety issues before public release.
Read source