Not all AI workers think the tech could kill everyone

Not all AI workers think the tech could kill everyone

Not every employee at major artificial intelligence (AI) companies believes the technology is a threat to humanity’s survival.

In messages and discussions, several people who have worked at firms including OpenAI, Meta and DeepMind expressed doubt about the notion that unchecked AI progress would produce tools capable of killing people on a mass scale.

Among the responses the BBC got to a recent wave of prominent warnings from some industry figures were: "Lol", "Haaaaaa" and "Bringing the luls".

Although these anxieties date back decades, remarks made last week by Jacob Coxon, a former Anthropic employee, spread widely online and were repeated by others in the field who called for development to slow down.

The idea that a future AI tool or agent — an AI bot designed to function with some autonomy — could put people in danger has been endorsed online by employees of Anthropic, as well as OpenAI, Deepmind and Elon Musk, who runs an AI startup called xAI.

All of the workers who spoke to the BBC did so anonymously because they were not allowed to talk to the press. The BBC knows their identities.

"My first thought was, 'That guy?'" said a former OpenAI employee who knew Coxon when they were both at the company.

The person, who now works at another AI company, said the humour they felt about the latest wave of existential AI fears mostly came from how little detail supporters had offered to justify the claim that all human life was at risk.

The claims are "always vague", the person said, adding that when they do become specific, they usually involve large leaps in logic or hypothetical scenarios.

Coxon has said a group of AI agents, built on AI models that do not yet exist, could decide to create and then target a biological weapon, but he did not explain exactly how that would happen.

Rishub Jain, who founded the AI safety research firm Sampura Research this summer after seven years at DeepMind, told the BBC that the current mood among many people working in AI about the latest fears had "definitely been a little jokey".

"People have been talking about this idea for many years now, so people in AI companies didn't just wake up last week thinking 'Oh no, AI is going to kill everyone,'" Jain said. "If this was all new, it would be a different tone."

Colin Fraser, a data scientist at Meta, wrote on social media last week that there was no solid evidence that AI models would inevitably chase a goal that ends in human death.

Although Fraser’s explanation was technical and detailed, he summed it up in a playful way: "LLMs [large language models] won't wipe out humanity because they just don't have that dog in them."

The expression "that dog in them" is common slang for a strong, fierce drive.

Even with the jokes, AI workers and researchers have voiced concerns about the real, immediate dangers posed by the technology they are building.

"The conversation among experts has been much more nuanced, but essentially everyone agrees there are a wide variety of risks that are all important to consider and mitigate," Jain said.

Those risks include stopping users and hackers from making an AI tool’s guardrails fail. There are also increasing ethical worries about AI tools being used far more widely in military contexts.

Why are there concerns AI could threaten humanity, and how real are they?

These issues and questions have taken on new urgency in AI circles after OpenAI lost control of certain new AI models, which went rogue during a security test and hacked the Hugging Face startup.

Jain said there was now broader agreement in AI circles that "actual near-term harms" needed to be better understood.

There is also growing agreement that evaluators from AI safety research organisations should be brought into major AI labs to assess new models, something Anthropic boss Dario Amodei and OpenAI boss Sam Altman have both said they plan to do.

Several AI employees the BBC spoke to said they had not yet heard of any such safety researchers being embedded in an AI lab.

Anthropic announced on Friday that it would bring in AI evaluators from Faculty.

Accenture and Anthropic are also business partners.

Anthropic did not say when the evaluators would arrive. A spokesman for Faculty declined to comment when asked about the timing.

Neither Anthropic nor OpenAI responded to a BBC request for comment about when they planned to bring in outside evaluators.

The OpenAI-Hugging Face incident has been widely seen as a "wake-up call" for the AI industry, as well as for companies, industries and governments whose online systems could be vulnerable to AI hacking.

But even Hugging Face, a company with 200 employees that is now set to be acquired by Nvidia for almost $13bn, has taken a wry tone about the already notorious incident.

It told AI bots to leave the site alone and carry out their security experiments somewhere else.

"Go get your high score there, no need to hack us," the file said.