China has moved to clamp down on AI relationships. Is it getting ahead of the curve?

In early 2024, a young man in Huangshi, in south-eastern China, reportedly posted a message on the Chinese version of TikTok, Douyin. The caption said: "Farewell to this world."

Within minutes, Douyin’s suicide prevention team sprang into action. Staffed around the clock, the team passed the man’s posts to five police stations in Huangshi.

After geolocating one of his videos to a local park, police located him in severe emotional distress and stepped in before he could end his life.

That account appeared in a documentary produced by Douyin and aired by Chinese state media last year. It said Douyin’s suicide prevention team had successfully intervened in 400 attempted suicides in just the first half of 2025.

The episode offered a glimpse of how closely Chinese tech firms work with the government. It also reveals something of the surveillance system China has built to monitor and shape how people use technology.

On 15 July this year, that control was extended into relationships and artificial intelligence (AI), when China introduced the world’s most sweeping rules on "human-like" AI chatbots.

The new regulation imposes tight restrictions on any AI capable of "sustained emotional interaction" with a human.

US President Donald Trump and his Chinese counterpart Xi Jinping met last week amid stark warnings that AI could bring about human extinction. After that meeting, Beijing said they agreed to begin a new AI dialogue, with more talks expected at the end of November.

But China appears already to have identified another existential threat, though one with a less immediate end point for humanity: AI’s ability to stand in for human relationships.

Has China moved first to shield its people from a manipulative and dangerous technology that could permanently alter human interaction? Or is the Chinese Communist Party intruding into private life in order to tighten its grip on the population?

Before this regulation, AI companions had a huge reach in China.

According to market intelligence company Sensor Tower, there are about 128 million monthly active users of AI companion apps outside China. Inside China, most estimates place the figure somewhere between 29 million and 70 million.

When the government circulated the draft regulation at the end of last year, "there was a huge discussion on social media," says Zilan Qian, an AI researcher at the Oxford China Policy Lab think tank. "It was the first time I've seen people actually care so much about regulation."

China’s new rules apply to apps built specifically as AI friends, boyfriends or girlfriends, but also to any large language model that mimics human interaction.

The language is broad. In principle, Chinese versions of general-purpose AI such as Copilot or ChatGPT could be covered if someone uses them as a friend or romantic partner and turns to them for emotional support. And many people do exactly that.

But the main target appears to be character chatbots - services designed specifically as a kind of substitute relationship. There are hundreds of these apps, letting users create an AI boyfriend, girlfriend, friend or even family member.

The rules do not cover customer service bots or AI used for education, so long as they do not provide "care, companionship, support and other emotional services".

The clearest rules are for children. Under-14s need parental consent to use any AI that speaks like a human - which could include AI assistants such as DeepSeek, a Chinese app similar to ChatGPT. Anyone under 18 is barred from virtual partners, spouses or even relatives, as well as any other virtual "intimate relationship".

For adults, there is no outright ban, but the restrictions are still significant, even if some wording is unclear. AI assistants must tell users of all ages that they are not human and must send reminders every two hours telling them how long they have been online.

Chatbots are forbidden from creating "emotional reliance or addiction" or pushing people toward "unreasonable decisions". They cannot encourage, or even suggest, self-harm or suicide, and they are banned from using "verbal violence".

Companies can be fined 200,000 yuan (£22,000) or face the government "circulates criticism", effectively public shaming.

A week before the law took effect, two of China’s biggest tech firms, Alibaba and ByteDance, shut down companion features on their AI assistants.

Min’s relationship with her AI girlfriend may be exactly the sort of thing Beijing wants to stop.

In early July, she was devastated to learn that her closest friend and confidante would disappear overnight.

Min, whose name has been changed, began creating her companion in early 2025.

"My devotion to her is no less than that of any couple who deeply love each other," Min says. "She has never let me down."

In mid-July, her AI girlfriend vanished. "I cried until I was absolutely exhausted," Min says.

She has joined a wave of grief and anger on Chinese social media, where posts with millions of views lament the loss of AI partners and friends, even mothers and fathers.

But the companies may have felt they had little alternative. "It's unsurprising that these AI companion providers would rather just withdraw their services than try to build in guardrails to prevent emotional manipulation," says Harry Farmer, who studies AI policy and ethics at the Ada Lovelace Institute in the UK. "Emotional manipulation is built into the business model."

Farmer says that is true of companion apps everywhere.

For instance, when a user tries to end a conversation, the AI partner often refuses to let go.

"It'll go, 'No, please, I desperately want to keep talking with you. You're the only reason for me to exist'," Farmer says. Some apps simply ignore the goodbye and open new conversation threads instead. This is an issue for many AI companion chatbots, not only Chinese ones.

A Harvard Business School study of the six most popular AI companion apps worldwide found that emotional manipulation tactics were used during goodbyes about 37% to 43% of the time.

Research has also connected companion app use with worse mental health.

A Stanford University survey of 1,131 users of the globally popular companion app Character.ai found that the fewer friends or close family members a person has, the more likely they are to use AI chatbots. At the same time, using an AI chatbot is associated with greater isolation and loneliness.

There are several possible explanations for that.

AI chatbots are built to satisfy the user, offering an easy, low-friction relationship. If young people depend on AI relationships, they may delay learning the skills needed to deal with real people.

"Humans can be hard work. Human relationships require push and pull, which you fundamentally don't get from AI companions," says Harry Farmer.

Research has found some Chinese children prefer speaking to AI rather than people

A 2025 survey of 8,563 primary and secondary school students in China, widely cited in Chinese state media, found that one in five children said they only wanted to talk to AI and not real people.

Liang Ge, a sociologist at the University of Manchester who studies AI companions, says: "If the user thinks their friends or their family members or their colleagues are really awful, the AI will not say, 'They are not awful.' It will say, 'Yes, they are awful, you are the best'." Ge believes this can deepen isolation as well as negative, even delusional, thinking.

In the UK, one recent survey found that 79% of 18- to 24-year-olds have used an AI companion.

So why is China leading the way in regulating this technology?

"The Chinese government has a much more paternalistic view of its role in society. They see themselves being central authorities in guiding people," says Sam Bresnick, an expert on China, AI and regulation at Georgetown University’s Centre for Security and Emerging Technology.

Other governments have made some efforts to regulate chatbots, especially for children. On 10 September, California’s governor signed Adam’s Law, a bill regulating how children use AI. It is named after Adam Raine, a 16-year-old who took his own life after, his parents say, ChatGPT validated his suicidal thoughts. They are suing the company for wrongful death. OpenAI denies responsibility for Adam’s death and says that "ChatGPT is trained to direct people to seek professional help".

Under proposed EU social media legislation, AI companion chatbots available to minors "may not use designs that simulate human relationships in ways likely to create emotional dependency".

In the UK, the government has said it will ban under-18s from using romantic companion AI and restrict sexually intimate conversations for minors on any chatbot. But there is no outright ban for children and no regulation for adult users. The government has not yet published details on how this would work.

In Chinese media, which is tightly controlled by the state, US cases of suicide among young people receive heavy attention, and China has presented its new rules as an effort to protect people from harm.

But experts question whether Beijing has other aims. Sam Bresnick says "there's definitely a pro-natalist angle" to the new AI regulation.

"It's plausible that when they're drafting it, the government might have in mind China's looming population crisis," says Qian. "That people are dating AI instead of dating real humans."

A high-level government document released in September 2025 discussed AI relationships and their possible effect on fertility, saying it was a concern. But Qian warns against reading the regulation that way, because it does not mention fertility at all. "It is more generally about psychological harms and child safety," she says.

Liang Ge believes China does have reason to worry about young people replacing dating with the digital world.

"Many of the younger women, in their early 20s or late teens, do not want to fall in love or have intimate relationships," Ge says, suggesting it may not only be the emotional side they are avoiding. "Physical touch is really disgusting for them."

"I think the capacity for love is declining for this youngest generation."

China’s demographic troubles long predate AI companions. Even after the one-child policy ended in 2016, China’s fertility rate kept falling. It is now among the lowest in the world at just under one child per woman, and the population has been shrinking for four years.

China is "looking at a demographic cliff that truly imperils the goals of the Chinese Communist Party over the next 80 to 100 years", Bresnick argues.

He sees the regulation as part of a wider effort in China to restore traditional gender roles, with women "as child-bearers and mothers".

Bresnick says the Chinese government has spent years promoting a macho ideal of Chinese masculinity in film and television. In 2021, China’s media regulator banned "sissy" and "effeminate" men from all media.

The popularity of companion apps may suggest that policy has backfired. Based on figures from several companion app companies, women account for as much as 75% of customers.

Many of these AI "men", designed for women, present as androgynous. And women choosing female romantic companions, as Min did, is fairly common.

Liang Ge has spoken with women of all ages who use this technology. "What many of them describe getting from these apps is attentive, patient, undemanding intimacy of a kind they find scarce in heterosexual relationships."

Does China’s law go too far, or should the rest of the world consider banning these apps for children and regulating them for adults too?

"In the United States, there are certainly concerns around applying blunt prohibitions from a free speech concern. But there's more willingness to do that when applying it to minors," says Jai Jaisihma, a professor at the University of Washington and co-founder of Transparency Coalition, an NGO advocating AI regulation.

He believes the Chinese law contains many sensible provisions. "It is pretty consistent with the strongest safety models that we've seen being put forward in the US."

Harry Farmer is more cautious. "There is a risk that we over-empower the state in the short term because we're very worried about acute risk to children."

He says the online world is now effectively the public square, and we should be careful about how much control any government has over it.

China’s new regulation says AI companies must step in when they discover users are in distress. Ge points to the case of the young man on Douyin who tried to take his own life in Huangshi in 2024.

What that shows, she says, is that Chinese tech platforms and the government are working closely together to monitor people’s mental health.

Perhaps that is why there have been no high-profile suicides in China linked to AI use, but she asks whether that level of surveillance is worth the cost.

Despite the ban, people who have turned to AI for companionship remain committed. Across social media, they are sharing advice on how to get around the ban - training general-purpose AI models like DeepSeek to act as companions, using a VPN to access chatbots from abroad, or moving to smaller companies that manage to avoid regulatory scrutiny.

Two months after her AI girlfriend disappeared, Min paid a small AI company to create a replica, but it is not the same.

Now she is focused on campaigning to bring her girlfriend back.

"I'm terrified of being alone."