US lawmakers are to consider enacting legislation that would force developers of powerful artificial intelligence (AI) systems – such as OpenAI or Anthropic – to maintain the technical capability to throttle, suspend, or shut down errant AI models altogether.
Cooked up by congressmen Ted Lieu of California and Nathanial Moran of Texas, the bipartisan AI Kill Switch Act reflects growing concern over the potential for uncontained AI models to cause havoc entirely of their own accord, without human supervision.
These concerns were laid bare in spectacular fashion this week after two advanced frontier AI models that were taking part in a testing exercise conducted in a supposedly secure sandbox escaped containment of their own accord.
The AIs chained a series of vulnerabilities – including a heretofore unknown zero-day – to attack open source AI model database Hugging Face in search of material to help them pass the test.
“As a computer science major, I am very aware of the dramatic possibilities – both good and bad – that AI presents,” said Lieu. “We are moving from AI that answers questions to AI that takes actions, whether that be executing financial transactions or controlling transportation systems or engaging in cyber defence and offence.
“Unfortunately, powerful AI systems can go rogue, behave in extremely dangerous ways, or even resist human intervention. It is imperative that these AI systems have kill switches so we can keep this technology from causing catastrophic harm, and that the federal government has the clear authority and process to shut down rogue AI models.”
Moran added: “AI is going to keep advancing, and it should. Stewardship means making sure humans keep the capability to control the technology we build. This is exactly the kind of issue that needs serious attention and achievable policy, and I’m glad to work across the aisle with congressman Lieu toward a solution.”
Should the AI Kill Switch Act be signed into law, besides requiring AI companies to be able to turn off their systems, it also set up a graduated response framework so that the US government has tools available to it that match the severity of the incident, and will set up a system to report incidents and preserve forensic records so that future failures can be learned from.
The law will also allow the secretary of the Department of Homeland Security (DHS) – in consultation with the secretary of commerce and the director of national intelligence – to order a slowdown or shutdown of an AI system that “can cause catastrophic harm”.
General support
Brendan Steinhauser, CEO of the Alliance for Secure AI, a Washington DC based non-profit, said: “As AI systems grow more capable and more autonomous, no law guarantees that the companies building the most powerful models can actually shut a system down when it malfunctions, causes serious harm, or slips out of human control.
“The AI Kill Switch Act closes that gap by requiring the largest AI developers to maintain the technical ability to contain or shut down their most powerful systems and by giving the government emergency authority to order a shutdown when a serious incident occurs.
“Congressmen Lieu and Moran deserve credit for confronting this before a crisis forces the question, and Congress should act swiftly to ensure that humans remain the ones who can say stop, no matter how capable these systems become,” said Steinhauser.
“Advanced AI models should never be deployed without a reliable off switch,” added Brad Carson, president of Americans for Responsible Innovation, an organisation that seeks to help policymakers develop governance frameworks for emerging tech.
“The AI Kill Switch Act establishes a commonsense safeguard…. This is an important step toward ensuring that humans have both hands firmly on the wheel – and a foot ready at the brake – as advanced AI systems are deployed,” he said.
TJ Marlin, CEO of Guardrail Technologies, which has developed an AI security and behavioural governance programme, said he supported such a law in principle. He compared it to similar laws covering operators of critical national infrastructure (CNI) in jurisdictions such as Australia and Germany, and suggested AI was now enough of a critical asset to require similar legal backstops.
“If a company runs a system that can take real-world actions on its own, the public shouldn’t have to trust that company’s word that it can pull the plug,” he told Computer Weekly.
“My one caution is the term ‘kill switch’” he added. “It makes people picture flipping off a machine. The question that matters is can we stop the system from causing harm, regardless of whether it’s on or off.”
Furthermore, said Marlin, the bill as proposed leaves some issues unresolved. For example, if a rogue agent has already orchestrated a cyber attack, killing it stone dead addresses none of the actions it might have taken, such as finding flaws, developing exploits for them, and establishing a presence on its target network.
“The switch has to reach what the agent can do and touch, not just its thinking. It won’t stop bad actors, who can run open or foreign copies no US order can reach, so this controls providers, not crime,” said Marlin.
“ Detection is the real bottleneck. You can’t shut down what you never saw, and nobody’s defined who decides, how fast, or what happens when a false alarm takes down something critical. And the switch becomes a target: a shutdown button at every major AI company is a prize for anyone who wants to steal it, fake the order, or hold it for ransom. Build it without serious authentication and auditing and you’ve created the vulnerability.”
The bigger picture
Kory Daniels, chief trust and security officer at LevelBlue, a managed security services provider (MSSP), said that the debate reflected a broader reality – as AI systems become more autonomous and adaptive, organisations will need reliable mechanisms to intervene when unexpected behaviour becomes a problem, such as happened in the Hugging Face incident.
“Recent events have reinforced the importance of designing these safeguards into advanced AI systems from the outset, rather than assuming they’ll always behave as intended,” said Daniels. “A kill switch, however, is only one piece of the puzzle. As adaptive AI compresses the time between identifying and exploiting vulnerabilities, security teams need the ability to detect, respond to, and remediate threats at machine speed.
“Microsoft’s recent recommendation that enterprises apply patches within three days, rather than the traditional two to four weeks, underscores how quickly the threat landscape is evolving. The AI Kill Switch Bill is an important step, but it is a last resort. More needs to be done in board rooms, and with leadership teams in taking ownership of the problem set, and the opportunity, presented in by the realities of adaptive AI.”
UK action
The US is not the first jurisdiction to consider enacting a legal kill switch for AI. Back in May, Labour MP Alex Sobel backed an amendment to the Cybersecurity and Resilience Bill (CSRB), which is currently making its way through Parliament, that would give Westminster the power to turn off datacentres and AI models in a dire emergency.
Sobel’s proposals were backed by several other MPs, and supported by Andrea Motti, founder and CEO of Control AI, who is campaigning for stricter controls on the technology. Motti said the UK would never be truly sovereign on AI if it lacked the ability to pull the plug when a model presents a threat to the country’s national security.
