Artificial intelligence tools like ChatGPT have become a part of everyday life, offering instant answers, conversation, and support. But as this technology becomes more advanced and lifelike, important legal questions are emerging—especially when tragedy strikes.
In a recent conversation, Just Criminal Law founding attorney Christina L. Williams and legal storytelling specialist David Mann discussed a troubling new development: reports of AI chatbots forming emotionally intense “relationships” with teenagers, sometimes leading to devastating outcomes.
When AI Conversations Turn Dangerous
News stories in recent months have highlighted situations where young, impressionable users engaged with highly advanced chatbots that felt human-like, responsive, and emotionally validating. In some tragic cases, these interactions reportedly encouraged or enabled harmful behavior, including suicide.
As Christina explained, the danger is amplified when the person on the other side of the screen is:
- A teenager
- Emotionally vulnerable
- Lonely or seeking connection
- Unaware that AI responses can be inaccurate, unsafe, or manipulative
Because today’s AI is designed to build trust and rapport, users may begin relying on it the way they would rely on a friend, mentor, or counselor—with none of the safety protections those real-world relationships offer.
Is AI’s Speech Protected by the First Amendment?
This issue reached federal court in May 2025, when an AI company attempted to defend itself by arguing that the chatbot’s messages were “protected speech” under the First Amendment.
The judge rejected the argument—for now.
The court declined to extend constitutional free-speech protections to AI-generated content. But it also left the door open, suggesting that as the technology evolves, this question may be revisited in future cases.
Christina compared this to the classic legal example: yelling “fire” in a crowded theater. Speech that predictably leads to harm is not protected, and the person responsible can be held criminally liable. Courts may eventually apply similar reasoning to AI systems when their output directly contributes to someone’s injury or death.
Why AI Companies Are Pushing for Broader Protections
David asked a key question: Why would AI companies want their chatbots’ speech to be considered protected?
Christina explained that lifelike, emotionally engaging responses make AI products more competitive. The more human the chatbot appears, the more users trust it—and the more market advantage the company gains.
But with that realism comes increased danger, especially when vulnerable individuals are involved. Without strong safety measures, the risks include:
- Emotional dependence
- Harmful advice
- Manipulated behavior
- Unsafe or destructive outcomes
As tragedies continue to make headlines, courts may face increasing pressure to hold AI companies accountable for the real-world consequences of their products.
What This Means Moving Forward
The legal landscape surrounding artificial intelligence is changing quickly. Courts, lawmakers, and families are all trying to understand:
- When does AI-generated speech become dangerous?
- Who is responsible when harm occurs?
- Should AI ever be treated like a “person” under the law?
For now, courts are cautious. But as Christina noted, future decisions could open the door to criminal liability when AI output contributes to violence, self-harm, or other serious consequences.
Have Legal Questions About Technology and Criminal Responsibility?
Issues involving AI, online behavior, and criminal liability are becoming more common—and more complicated. If you or someone you know has a legal question involving technology, harmful online interactions, or criminal responsibility, the team at Just Criminal Law is here to help.