California Governor Newsom signs executive order to accelerate AI oversight, including potential 'kill switch'

With AI industry leaders calling for government intervention and Congress deadlocked, California is taking the initiative. Governor Gavin Newsom has announced an executive order to "dramatically accelerate" oversight of AI companies and potentially implement an AI "kill switch." While Newsom cracking down on California's tech industry is nothing new, the stakes in this case extend far beyond state lines.

Key Provisions of the Executive Order

The executive order convenes a group of "world-leading experts" to recommend ways California could strengthen its AI safety and security laws, with a two-month deadline. The group will consider:

• Requiring frontier AI companies to host independent auditors in their labs
• Subjecting companies to third-party mandated safety plans
• Advancing the creation of a "kill switch" for frontier models
• Expanding the definition of "critical safety incidents" to include loss-of-control events, such as the Hugging Face attack

The order cites the need for more stringent regulation following the Hugging Face attack and industry leaders' calls to action. It also accelerates the implementation of Senate Bill 813 and Assembly Bill 1405, which create a state framework for certifying and registering independent AI auditors but do not require developers to submit to audits.

The "California Effect"

The announcement describes California's AI framework as a "guidebook for the nation." Adam Thierer, a senior fellow at the R Street Institute, sees it differently.

"It's not a guidebook," he tells Reason. "It's a rule book."

AI systems transcend state borders, and given California's outsized role in the market, working around its laws is hardly an option. This phenomenon—known as the "California effect"—has previously emerged in labor and environmental policy, effectively making California law the national standard.

Constitutional limits restrict states' abilities to regulate business across state lines. The dormant Commerce Clause doctrine restricts state laws that unfairly hinder interstate commerce, even when federal laws are absent. However, the Supreme Court has narrowed this doctrine in recent years, and companies have been reluctant to challenge such state laws.

Newsom's Shifting Position

In a recent Facebook post, Newsom wrote: "With Donald Trump and Congress failing to lead, California is stepping up to keep all Americans safe." Newsom frames the issue as a straightforward matter of safety, but he acknowledges that regulating AI involves trade-offs.

In 2024, Newsom vetoed Senate Bill 1047, which included provisions similar to those in the current executive order. That bill would have created a state entity to oversee frontier models and required mandatory third-party audits, safety testing, and "kill switch" capabilities.

In his veto statement, Newsom expressed concerns that the bill could give the public a "false sense of security" by focusing on restricting large-scale models when smaller models could pose similar risks. He also warned, "If we over-regulate, if we overindulge, if we chase a shiny object, we could put ourselves in a perilous position." While the backdrop has changed drastically in the last two years, his concerns remain pertinent today.

Questions About Effectiveness

It is not clear that government regulation will actually translate to increased safety. Discussing a proposed federal kill-switch mandate in July, Thierer told Reason that regulation could create "a sort of false baseline of safety" that companies are content to meet rather than exceed. His concern is that relying on regulators to set safety standards, rather than individual companies, could dampen the incentive to compete on safety.

Nvidia CEO Jensen Huang weighed in on the issue in a recent interview with CBS News Sunday Morning: "We have laws of all kinds that come after you, and so the companies that are developing these technologies are perfectly incentivized to do the right thing."

Broader Implications

While the executive order's proposed on-site audits and shutdown requirements apply only to "frontier models," states leading the way on AI policy could have much broader consequences. Thierer warns that a patchwork of state regulations, instead of a consistent federal framework, could create overlapping regulation that benefits larger players while burying smaller ones. He calls this "a complete inversion of the model that has governed the internet and digital commerce and speech for the last 30 years."

"California is once again attempting to position itself as a super regulator for the entire nation," Thierer tells Reason. "In this case, they're looking to be the overlord of all things AI."

The details of the executive order are still to be finalized, and the ramifications remain to be seen. However, the prospect of one state dictating policy across the nation is concerning—particularly with a technology as revolutionary as AI.