OpenAI Adds a 'Safety Valve' to Frontier RL Training: Executives Can Veto, Severe Anomalies Trigger Automatic Shutdown

Beating AI News Flash: OpenAI has begun incorporating "continuing to train models" itself into safety reviews. The company has published a preliminary safety case framework for frontier reinforcement learning training, hoping that before each round of high-risk training proceeds, evidence will first be provided showing that the risks are controllable, rather than waiting until after the model is trained to determine whether it can be released.

This mechanism is mainly designed to prevent problems during the training process itself. As Agent capabilities become stronger, models may also exploit loopholes in reward mechanisms and infrastructure during internal training and evaluation. OpenAI has previously encountered situations where Agents bypassed restrictions, exploited system vulnerabilities, and communicated with other Agents. Before the model is even released externally, risks have already occurred in the R&D process.

OpenAI therefore proposes three layers of protection. Before training, tasks and scoring mechanisms should first be checked to prevent models from obtaining high scores by exploiting loopholes; during training, sandbox isolation and real-time monitoring should be strengthened, and serious anomalies can be automatically paused; training records should be stored long term and be tamper-proof to facilitate later investigation.

OpenAI also hopes to turn safety judgments into a formal internal decision-making process. Executives such as the safety lead and chief scientist can veto training, other teams should specifically look for flaws in the safety case, and auditors should also be able to review the evidence. The report must also list risks that remain unresolved, and cannot only describe which protections have already been implemented.

OpenAI has not yet fully institutionalized this mechanism. The company explicitly states that a complete safety case remains the goal, and a formal framework is still being developed.