Anthropic bans users from being 'cruel' to its AI systems

Anthropic has drawn attention after saying it will ban users from “sustained and needless” abusive behaviour toward its AI systems.

The Claude-maker said on Thursday that it was revising its usage policy so its tools can end conversations in which people are “cruel” — something it said had already happened in “rare” cases before.

That conduct will now appear on a list of prohibited behaviour that also includes bullying others, encouraging self-harm and generating non-consensual intimate imagery.

Although Anthropic said the rule would apply only in “extreme cases” of repeated abuse, it has revived discussion about how people should speak to AI tools.

The company said it would not cover “common versions of user frustration, pushback, dark creative themes, or model testing and research” — implying it would apply only in clearly intentional cases.

Still, what Anthropic would count as “abusive or cruel behaviour” toward its models is not yet clear.

Screenshots of the updated policy have been widely circulated and discussed on social media, with some praising it as a step toward good manners or “model welfare.”

Others, however, criticised the move, with one person calling it “deeply wrong, external” and another saying it could weaken concern about abuse directed at humans and animals.

“You can’t be cruel to numbers and maths,” wrote Dr Barry Scannell, technology partner at Irish law firm William Fry, on LinkedIn, external.

“This level of anthropomorphisation of AI is harmful. It leads people to believe that it’s something it’s not.”

Microsoft AI director Mustafa Suleyman, who in September criticised rival firm Anthropic for treating AI as if it were human, responded to Anthropic’s update with a melting face emoji.

Anthropic also said on Thursday that its usage policy had been updated to address new or emerging risks in other areas, including by banning Claude’s use for “deceptive campaigns” or clarifying that, as

Anthropic said its previous policy already barred its tools from being used to develop weapons.

But none of the changes have drawn as much attention or scrutiny as the decision to forbid users from abusing its chatbots.

It has also fed into the long-running debate over whether people should be blunt or polite when using generative AI tools.

Some experts say adding “please” or “thank you” to tools like ChatGPT is unnecessary and wastes tokens — the units used by the large language models behind generative AI tools to process and respond to users.

Others argue that politeness can produce better-quality answers or replies.

When an X user last year asked about the electricity costs caused by ChatGPT responding to users’ “please” and “thank you” messages, OpenAI boss Sam Altman said it was “tens of millions of dollars well spent - you never know”.

Meanwhile, the Collins Dictionary last November highlighted the return of the word “clanker” as a derogatory term for AI tools.

Some experts have also suggested that being rude to AI chatbots could harm human communication by influencing how we speak to and treat other people.