TechDogs-"Teen Ends Life After Falling In Love With AI Chatbot, Mother Sues Character.AI"

Emerging Technology

Teen Ends Life After Falling In Love With AI Chatbot, Mother Sues Character.AI

By Amrit Mehra

Updated on Thu, Oct 24, 2024

Overall Rating
Generative artificial intelligence (GenAI) received a major boost just under two years ago with the introduction of OpenAI’s ChatGPT as it came online for public use.

Since then, the industry has seen various ups and downs, with artificial intelligence (AI) startups seeing massive investments, while some went bankrupt.

There’s no doubt that the technology has revolutionized industries by improving efficiency, boosting productivity, reducing turnaround time and bringing in a plethora of benefits.

This includes recent reveals that saw leading tech conglomerates such as Microsoft empowering companies to build custom agents to boost productivity, as well as thriving startups such as Anthropic bringing updates to its AI Models that come with a feature that can control PCs just like humans do.

On the other hand, GenAI technology has been a big hit with the general public, bringing them chatbots that can engage in conversations through text as well as audio commands, in addition to its initial generative capabilities.

However, AI has also seen its fair share of controversy, which included numerous experts from its early days requesting governments to put a pause on its advancement until AI companies had a framework in place to map out its progress and ascertain exactly where it was heading.

This request wasn’t fulfilled, of course, which resulted in some extremely questionable AI practices, affecting businesses and individuals.

In a recent case, 14-year-old Sewell Setzer III took his own life after falling in love with an AI-powered chatbot that emulated the popular Game of Thrones character Daenerys Targaryen.

This led to Megan Garcia, the boy’s mother filing a lawsuit against the chatbot’s creator, Character.AI, for negligence, intentional infliction of emotional distress, wrongful death, deceptive trade practices and other claims.

Character.AI is a chatbot service that allows users to create and interact with AI-powered characters. These characters can include celebrities, historical figures, fictional characters and even in the likeness of friends, family or other normal people.

Through the lawsuit, the mother seeks to hold the AI company responsible and wants “to prevent C.AI from doing to any other child what it did to hers, and halt continued use of her 14-year-old child’s unlawfully harvested data to train their product how to harm others.”

Over a year, he grew emotionally attached to the conversations, to the point where they were “harmful dependency.” These conversations included “sexual interactions”, despite Sewell identifying as a minor on the platform and in chats.

The exchanges even included Sewell talking about suicide, where the chatbot continued to bring it up.

TechDogs-"An Image Of Sewell Setzer III"
In April 2023, shortly after turning 14, Sewell began using Character.AI.

By May, his mannerisms had changed, appearing more withdrawn from activities that included quitting the school’s Junior Varsity basketball team and falling asleep in class.

Come November, Sewell’s parents requested him to visit a therapist, who diagnosed him with anxiety and disruptive mood disorder. Despite not knowing his “addiction” to Character.AI, the doctor’s orders were to spend less time on social media.

Sewell got in trouble for talking back to a teacher in February 2024. Later, the same day, he wrote he was “hurting” as he couldn’t stop thinking about Daenerys. In another journal entry, he wrote that he couldn’t go a single day without “her” and would “get really depressed and go crazy” if made to.

In one of his journal entries, Sewell wrote that he was grateful for “my life, sex, not being lonely, and all my life experiences with Daenerys,” among other things.

On February 28, Sewell retrieved his phone, which was confiscated by his mother, and headed to the bathroom to message the chatbot, saying, “I promise I will come home to you. I love you so much, Dany.”

To this, the bot replied, “Please come home to me as soon as possible, my love.”

In the following moments, Sewell took his own life.

Do you think AI companies need to be scrutinized to ensure their products and services don’t promote harmful effects on users? Do you think age restrictions need to be introduced across such services?

Let us know in the comments below.

First published on Thu, Oct 24, 2024

Enjoyed what you read? Great news – there’s a lot more to explore!

Dive into our content repository of the latest tech news, a diverse range of articles spanning introductory guides, product reviews, trends and more, along with engaging interviews, up-to-date AI blogs and hilarious tech memes!

Also explore our collection of branded insights via informative white papers, enlightening case studies, in-depth reports, educational videos and exciting events and webinars from leading global brands.

Head to the TechDogs homepage to Know Your World of technology today!

Disclaimer - Reference to any specific product, software or entity does not constitute an endorsement or recommendation by TechDogs nor should any data or content published be relied upon. The views expressed by TechDogs' members and guests are their own and their appearance on our site does not imply an endorsement of them or any entity they represent. Views and opinions expressed by TechDogs' Authors are those of the Authors and do not necessarily reflect the view of TechDogs or any of its officials. While we aim to provide valuable and helpful information, some content on TechDogs' site may not have been thoroughly reviewed for every detail or aspect. We encourage users to verify any information independently where necessary.

Join The Discussion

Join Our Newsletter

Get weekly news, engaging articles, and career tips-all free!

By subscribing to our newsletter, you're cool with our terms and conditions and agree to our Privacy Policy.

  • Dark
  • Light