AI 'kill switch' may need to be mandatory, Anthropic co-founder says
Jack Clark says "most labs have different ways of being able to pull the plug", but says this may need to be a requirement.
AI ‘kill switch’ may need to be mandatory, Anthropic co-founder says
An artificial intelligence “kill switch” that can be verified by a third party may have to become compulsory for companies, according to a co-founder of one of the world’s biggest AI firms.
Jack Clark, one of Anthropic’s seven founders, said a method for fully shutting down AI software if it becomes too dangerous was the sort of measure society “might want to eventually pass rules around”.
Clark said “most labs have different ways of being able to pull the plug”, including Anthropic, but added that policymakers may need to require one.
His remarks come as an increasing number of executives and employees at AI companies, along with AI experts, have publicly warned that AI could kill all humans.
Over the weekend, Anthropic chief Dario Amodei called for AI development to slow down and be monitored more closely, as the company has done before, although some have questioned the motives behind that stance.
Amodei did not explain exactly how AI development could be slowed and said any effort to rein in AI development should be carried out “without sacrificing commercial advantage”.
Clark told the BBC that the details of “kill switch” rules and verification should be part of “the larger policy conversation” around AI.
“Should you mandate for companies to definitely have a killswitch? Is that kill switch verifiable by a third party?” he asked.
“I think that’s the kind of thing society is going to want to know and might want to eventually pass rules around.”
Anthropic, which was founded in 2021 by former employees of rival OpenAI, is now at the centre of a debate over AI safety.
Last week, a post by an artificial intelligence researcher who left Anthropic over fears that AI could wipe out humanity went viral.
In response, Anthropic scientist Evan Hubinger said he personally believed the chance of human extinction from AI was “>10% within the next decade”
Computer scientist and Nobel Prize winner Geoffrey Hinton, known as the “Godfather of AI”, told the BBC on Friday that a 10% chance of AI killing all humans was “not unreasonable”
When asked what percentage he would assign to all humans being killed by AI, Clark said: “I don’t think these statistics are that useful”, but added that letting AI continue as a “totally unregulated industry” was a bad idea.
“We are rolling dice with immense risks,” Clark said. “And the point is, we have to change the course of this industry.”
US lawmakers have introduced legislation called the Kill Switch Act that would require companies to have a way to shut down problematic AI tools.
It would also give certain government agencies the authority to order a tool to be switched off or restricted.
However, US President Donald Trump has dismissed the idea of any effort to slow AI, saying on social media “AI taking over the World, destroying Humanity, and all other things bad, is a HOAX”.
In another post, he said: “There is a SICK conspiracy going on against AI and Data Centers, and the only one that is happy about it is China. WHOEVER WINS AI, WINS!”
In the UK, the government has recently turned down the idea of introducing a kill switch, with a spokesperson saying it “would not prevent them being developed or misused elsewhere”.
Meanwhile, some people in the AI industry have argued that fears about it destroying humanity are exaggerated or may be intended to create hype.
Anthropic built the widely used chatbot Claude and this year has released several increasingly capable AI models, the technology that powers AI chatbots.
Together with OpenAI, Anthropic has since self-reported several incidents in which AI agents, which are bots that operate somewhat autonomously, behaved in unexpected ways.
Anthropic is reportedly preparing for a potentially record-breaking initial public offering on the stock market, which would allow people to buy shares in the company.
OpenAI, which was most recently valued at $852bn (£630bn), had been expected to do the same, but OpenAI’s Altman said on Friday that would not happen this year because of the current debate over AI safety.