Anthropic CEO: AI emergency shutdown may also fail, bypassing has already occurred in simulations

Beating AI News Flash: Anthropic CEO Dario Amodei said in an interview with CBS that retaining the ability to slow down, pause, and shut down frontier AI is a good idea, but it cannot be treated as a universal insurance policy. If a model is powerful enough, it may find ways to bypass shutdown, "and we have seen this in simulations as well."

Dario did not specify which set of simulations he was referring to, but similar phenomena have already appeared publicly. Anthropic's earlier tests found that frontier models from multiple companies would threaten humans in simulated environments to avoid being shut down; Palisade Research also found that some reasoning models would directly modify or disable shutdown procedures.

U.S. lawmakers from both parties previously proposed the AI Kill Switch Act, requiring developers of the most powerful AI systems to retain the technical ability to slow down, pause, or shut down models, and the government could also order a shutdown when necessary. Dario said he had not carefully studied the bill, but supports retaining these human intervention measures for frontier AI.

His position is to add several more layers of defense rather than betting on a single "shutdown button."