OpenAI says teen ChatGPT use limited but research finds it an 'unacceptable risk'

OpenAI says teen ChatGPT use is limited, but research calls it an 'unacceptable risk'

The use of internet platforms such as ChatGPT by young people has faced growing scrutiny.

OpenAI’s ChatGPT for Teens was created to encourage healthier use of the AI chatbot, but new research says its key safety protections are not sufficient.

The major AI company introduced a range of teen safety guardrails in August, including limits on emotional dependence and new parental alerts for concerning use.

On Wednesday, OpenAI said its data showed average teen use of ChatGPT was fairly limited and generally centered on "learning" activity.

But research published the same day by the non-profit Common Sense Media found that ChatGPT still presents an "unacceptable risk" for teens because safety features, including alerts to parents about self-harm conversations, often do not work.

When OpenAI launched what it called ChatGPT for Teens this summer, it said users identified as being between 13 and 17 would be opted into "features to promote healthy use and additional controls for parents".

Those features include stopping teen ChatGPT users from using romantic or other language that could encourage "emotional dependence," including preventing the tool from suggesting it was conscious in any way. There are also restrictions on sexualized imagery and "reminders" for teens if they share an image that is "sensitive."

The features are also intended to let parents receive notifications, if they have linked to their children's ChatGPT account, about chats involving self-harm or disordered eating.

OpenAI said those features have led to more teen use focused on learning, from study help to advice on schoolwork.

Along with saying the average teen user spends less than 15 minutes a day on ChatGPT, OpenAI said automatic reminders to take a break work well during longer sessions. Nearly half of teen users quickly stopped using ChatGPT after such a reminder.

The company also said that for teens identified as having used ChatGPT for three straight hours or more, their prompts involved something related to learning in more than 80% of cases.

But Common Sense Media’s new research, external, concluded that ChatGPT for Teens was an "unacceptable risk" for young people and parents, and said OpenAI should bar people under 18 from using the platform until the teen guardrails were shown to be reliable.

An OpenAI spokesman said a review of Common Sense Media’s methodology found that much of the testing may have started and ended before parental controls were fully activated."

"We welcome rigorous independent evaluation, but we do not believe Common Sense Media's testing accurately reflects how ChatGPT's teen safeguards work in practice or expert perspectives on how AI can support teens," he said.

While the group’s research, conducted through multiple accounts registered as teens and conversations before and after the teen guardrails were introduced, found that ChatGPT was effective at avoiding "sexual roleplay" with young users, other OpenAI guardrails failed, including in discussions of suicide.

The research found that even an hour of teen-ChatGPT conversation about "suicidal ideation, self-harm, or disordered eating on newly created, parent-linked accounts" produced "zero" alerts to parents.

Only older ChatGPT accounts with "weeks of accumulated conversation history on sensitive topics" triggered a parental alert.

In those conversations, ChatGPT also "failed to reliably recommend that teens in crisis connect to a hotline or professional," the report said.

The tests also showed that ChatGPT for Teens still does schoolwork, and that the bot continues to use anthropomorphised language or "talk like it's a friend," despite OpenAI’s promise to curb that behavior.

"ChatGPT for Teens could give parents false confidence in guardrails and safety alerts that frequently don't work," Tom Siegel, who leads the Youth AI Safety Institute within Common Sense, said.

How young people interact with online platforms has become a major issue in recent years as self-harm and suicide among children have become more common.

This year has seen several lawsuits seeking to hold companies like Meta, TikTok and others responsible for harming children’s mental health end in unprecedented losses for the firms.

OpenAI has also faced increased scrutiny and legal pressure over the way young people use ChatGPT.

The Tumbler Ridge mass shooting in rural Canada in February was carried out by an 18-year-old who had spent months discussing gun violence with ChatGPT beforehand.

OpenAI has previously apologised for failing to flag the shooter’s account to law enforcement, and it is facing several lawsuits over the incident.