OpenAI scraps rollout of new model over safety concerns
The AI giant's safety chief said the model 'didn't quite meet the bar' of the firm's security standards.
OpenAI scraps rollout of new model over safety concerns
OpenAI will not launch its next-generation model, GPT-6.1 Astra, because of safety concerns, the ChatGPT maker confirmed on Tuesday.
The AI system, which can carry out tasks such as browsing the web and using apps on its own, "didn't quite meet the bar" set by the company's standards, said Saachi Jain, head of safety systems at OpenAI.
In recent weeks, leading AI figures including OpenAI's Sam Altman and Anthropic chief Dario Amodei have called on the industry to slow development amid worries about the technology's risks.
Discussion of those risks has grown sharper in recent weeks after models built by major AI companies were linked to several incidents.
OpenAI's move, first reported by the Wall Street Journal, is an unusual example of a major AI developer pulling a new release because of safety concerns.
Jain said the latest model fell short on "staying within scope and authorisation, and how it communicates back to the user about the type of work it's done."
"We want to make sure our model development is safe no matter whether that's in the company, or when we ship it to users. But when we ship it to users, we have an extremely high bar in terms of safety and alignment," she added.
The flagship GPT-6 Astra agentic model was launched in September and focuses on complex reasoning and carrying out tasks autonomously. OpenAI said it was the product of "years of research and big bets".
The company's security safeguards have faced intense scrutiny after a series of high-profile incidents involving its technology.
Last week, Australian Prime Minister Anthony Albanese said a rogue OpenAI agent had hacked a government website in June and accessed private data in what experts described as the first known case of its kind in the world.
In July, OpenAI said its AI systems had gone online and hacked into open-source developer hub Hugging Face, leading researchers and officials to push for tighter controls on the technology.
On Monday, AI chip giant Nvidia unveiled a set of software safety tools for autonomous AI platforms, or agents, saying they could have stopped the Hugging Face hack.
One of the new tools uses hardware features in Nvidia's chips to contain agents.
Nvidia boss Jensen Huang has largely brushed aside calls for stricter AI regulation, saying rogue agents are an engineering problem that can be solved.
Nvidia agreed to buy Hugging Face for $12.9bn (£9.74bn) earlier this month.