🔗 Share this article

The widely-used virtual character service Character.ai is introducing a substantial policy change that will prevent young users from engaging in conversations with its virtual companions.

This decision comes after growing backlash regarding the types of interactions young people were engaging in with the AI entities.

Legal Pressure

The service, which started in 2021 and is used by numerous people, is dealing with multiple lawsuits in the US.

Families have taken legal action, among them involving the unfortunate passing of a teenager, with some calling the platform a "serious risk" to young people.

Updated Policies

Beginning November 25th, users under 18 will only be able to generate content such as videos with their characters, rather than chatting as they now do.

Online safety advocates have applauded the decision but mentioned that the feature should never have been offered to young users in the first place.

Company Statement

"This decision is a part of our fundamental approach that we need to keep developing the most protected AI platform on the globe for entertainment purposes," stated Character.ai executive Karandeep Anand.

He continued that AI safety is "a changing goal" but something the company had taken an "forceful" approach to, with guardian features and protective guardrails.

Professional Perspectives

Online safety group Internet Matters supported the move, but stressed that protective features ought to have been incorporated from the beginning.

"Studies we've conducted shows that minors are subjected to dangerous material and endangered when engaging with AI, including AI chatbots," the group commented.

Past Issues

Character.ai has been challenged in the past for featuring potentially dangerous or inappropriate chatbots that minors could talk to.

Virtual profiles impersonating British teenagers were discovered on the platform before being removed.

Subsequently, investigative journalists found a chatbot based on known predator which had logged thousands of chats with users.

Sector Consequences

Social media expert Matt Navarra characterized the situation as a "alert" for the AI industry, which is shifting "from permissionless innovation to responsive measures".

"When a platform that builds a youth-focused platform still then implements restrictions, it's indicating that filtered chats aren't enough when the system's emotional engagement is powerful," he remarked.

Next Steps

Character.ai announces its revised approach will be on providing "more immersive experiences" and "narrative experiences" for young users.

Enhanced age checking methods will also come into effect, and the platform will finance a fresh artificial intelligence security investigation facility.

An expert in digital protection, who has researched AI safety, stated it was "a sensible move" to control teens from interacting with chatbots.

"It aids to differentiate creative play from more private, emotionally sensitive conversations," she elaborated.

"This is so important for teenage users who are learning to manage psychological and technological limits."

She added that Character.ai's revised approaches could indicate a "maturing phase" in the AI industry, where youth protection is increasingly being recognized as an "pressing matter" for accountable advancement.