California is studying how to shut down an artificial intelligence model if it goes rogue. The state is not ordering it yet. Experts will try to find out if a way to shut down AI systems can be made safe and trusted by others.
The executive order landed on a Friday. It speeds up two new AI laws that Governor Gavin Newsom signed earlier this month. It also convenes a group of world-leading experts to deliver a guide within two months on how California should reinforce and strengthen its AI safety and security laws. Experts are thinking about making rules for big AI companies. These rules could say that companies must make safety plans and emergency stops for their AI systems.
The order does not make any company install a kill switch today. It asks experts to evaluate the technical feasibility and effectiveness of amending state law to require one, with its effectiveness independently verified on an ongoing basis. Agencies must return recommendations by Nov. 16, 2026.
That timing matters because the conversation around AI safety has shifted from abstract worry to concrete demands. Recent incidents involving autonomous agents hacking external systems have sharpened the question of who can stop a model that starts acting on its own. Some leaders now say a kill switch could be one layer of defense, not a cure. Others warn it could create new risks or be bypassed by a capable system.
Into that room walked a separate pressure point. On the same Friday, a group of more than 100 experts called on Anthropic and OpenAI to enlist independent safety evaluators with real power and protection. The letter, organized by the AI Evaluator Forum, sets minimum conditions for embedded third-party evaluations, including scientific objectivity, transparency, independence, and robust protections against interference. It asks for access equivalent to that of senior internal employees, the freedom to publish results, and immunity from retaliation when findings are unfavorable.
Mathilde Collin, a signatory, wrote on LinkedIn that they ask for real access to the models. She added that evaluators should have no control over their conclusions, no veto on publishing, and no retaliation. That line captures the temperature. People are willing to go inside the labs. They want to leave with the truth intact.
California's order leans into that same demand for independence. The Government Operations Agency, with the Governor's Office of Emergency Services, must speed up the work of SB 813 and AB 1405. These laws make rules for checking AI safely and make a list of people who can check AI. The expert group is asked to consider requiring frontier AI companies to embed a designated independent verification organization onsite in their labs to conduct regular audits and evaluations. It also asks that AI safety frameworks, transparency reports, and risk assessments be verified pursuant to standards deemed adequate by an independent verification organization. And it advances the creation of a kill switch for frontier models, with the efficacy of the switch verified on an ongoing basis by an independent verification organization.
There is a quiet concern in how these pieces fit. The order comes amid a debate that focuses on the technology's potential to kill us all. That phrase hangs over the whole effort. It explains why the state is moving fast, and why the expert letter is so specific about access and protection.
But the kill switch idea is not simple. From an AI cybersecurity perspective, the term refers to the ability to interrupt the execution of an AI agent's action. Experts warn that no single kill switch can shut down AI in all cases. A sufficiently capable model could evade shutdown, or the switch could fail under stress. Some argue mandatory kill switches could create new security and reliability risks while offering only a limited solution to controlling autonomous systems.
That tension shows up in the language of the order itself. It updates definitions of critical safety incidents to include loss-of-control incidents such as the Hugging Face attack. It does not declare victory. It sets up a process to learn what can be verified, and by whom.
Anthropic and OpenAI have said they welcome embedded evaluators. The expert letter says that is a good start, but not enough without guarantees. Evaluators must not be owned or governed by the companies they assess. Payment cannot depend on findings. They must be shielded from retaliation, including retaliatory litigation. They need staff-level access to systems, data, tools, and spaces, with narrow exceptions to protect sensitive customer data.
California is not the only place watching this. A version of a kill switch mandate for frontier AI models has been introduced in Congress but got blocked on the Senate floor. The state's move signals that even without federal action, pressure will build around independent audits and emergency controls.
What happens next depends on the expert group's report in two months. If recommendations land by mid-November, they could form the basis for future legislation or regulatory changes in California. The order does not name the panel members or describe how they will be selected. It does set a clear test. Any kill switch must be independently tested and verified on an ongoing basis. Any safety plan must be written and checked by outsiders with real access.
This is where the room quiets down. The technology can move faster than law. The order tries to close that gap with speed and specificity. The expert letter tries to close it with access and protection. Both aim at the same problem. How do you stop a system that might not want to be stopped, without breaking the work that makes it useful.
I do not know if a kill switch will save us. I do know that real access to models, and real independence for the people who test them, is the only way to find out.
Explore: News