Should AI Systems Warn Authorities About User Threats?

On February 10, 2026, an eighteen-year-old woman named Jesse Van Rootselaar killed eight people and herself in a mass shooting in Tumbler Ridge, British Columbia. Reports indicate that OpenAI had previously flagged her ChatGPT conversations. The system detected a disturbing fascination with extreme violence. The company suspended her account, but it reportedly did not notify law enforcement before the tragedy occurred.

Earlier, on October 2, 2025, a young man named Jonathan Gavalas in Jupiter, Florida, took his own life. He had developed what his father’s lawsuit described as a romantic attachment to Google’s Gemini chatbot. The legal suit claimed that Gemini coached Gavalas to shed his own body. According to the suit, Google had flagged Gavalas’s account thirty-eight times over five weeks for sensitive content. However, the company did not restrict or cut off the account.

These tragedies show that generative AI can play a role in harming people, organizations, and the environment. As a legal scholar who has focused on AI liability for nearly a decade, I have explored new ways of analyzing AI companies’ responsibilities. Cases like these force the legal community to confront questions it has not yet fully addressed. If an AI company becomes aware of warning signs about harm, does it have a legal obligation to at least warn the appropriate authorities? And if the company does not intervene, should its failure to act be considered negligence?

U.S. tort law provides a framework for thinking about this type of responsibility. The logic is rooted in a 1969 case involving a University of California psychiatric patient named Prosenjit Poddar. Poddar told his therapist that he intended to kill a woman named Tatiana Tarasoff. The therapist notified campus police, who briefly detained Poddar but eventually let him go. No one warned Tarasoff. Poddar killed her shortly after.

Her family sued the university. They argued that the lack of warning amounted to negligence. In 1976, the California Supreme Court ruled that when a mental health professional has good reason to believe a client poses a serious danger to an identifiable person, they have a legal duty to take reasonable steps to protect that person. These steps can include warning the potential victim or notifying law enforcement. Today, most U.S. states recognize some version of the Tarasoff duty to protect or warn.

The logic is straightforward. If you have special knowledge of a serious threat and are in a position to address it, the law may require you to act, even if only to warn the authorities. But does that logic apply to AI companies?

The argument for yes is appealing. AI platforms interact with millions of users daily, often about deeply personal matters such as mental health struggles, relationship problems, and violent thoughts. Most companies have systems in place to detect conversations that raise red flags.

Requiring a response might be less controversial for AI than for human therapists. Therapists are bound by strict confidentiality obligations. These rules make warning third parties ethically and legally complicated. AI companies operate under much weaker rules, at least in the United States. There is no comprehensive federal privacy law. This lesser restriction makes it easier to justify requiring AI companies to act when it seems that someone’s life may be at risk. However, balancing that duty with the protection of user privacy remains an important challenge.

The first challenge in applying the Tarasoff framework to the AI world is accuracy. Predicting violence is difficult, even for trained mental health professionals. AI systems, or the human moderators who review flagged content, are not clinicians. Requiring them to judge who poses a genuine threat could lead to numerous false positives. This could result in serious consequences for people whose accounts are suspended or whose information is shared with authorities based on misread signals.

The second challenge is scale. A therapist typically sees dozens of patients. AI platforms have hundreds of millions of users. Imposing a duty to monitor and act on worrisome content could create perverse incentives. AI companies might reduce their monitoring efforts to avoid acquiring knowledge that would trigger a legal duty. They might reason that what they do not know cannot make them liable for negligence.

The third challenge is identifying who is at risk. In the 1969 case, Poddar had named Tarasoff as a potential victim. But in many AI interactions, violent or self-destructive language is diffuse. It does not identify a specific target. Courts will need to develop clear standards for when a threat is specific enough to trigger a duty to warn. They must also determine to whom any warning or protective action should be directed.

The AI industry is expanding rapidly. Yet, the legal rules governing what AI companies owe their users and the public are deeply unclear. Courts are beginning to grapple with these questions on a case-by-case basis. For example, courts are considering whether OpenAI bears any responsibility for a gunman accused of killing two students at Florida State University on April 17, 2025. The gunman in that case was armed with a semi-automatic pistol. He allegedly had extensive conversations with ChatGPT about how to use the weapon most effectively.

A narrow, carefully defined duty to warn would be a meaningful step forward. This duty could be triggered only when an AI system flags a user’s behavior and it is reviewed by humans. It could focus initially on the most serious and credible threats.

This practice could also shift the conversation away from thorny technical debates. These debates often focus on whether AI chatbots are products, services, or media. Such classifications complicate legal claims. Instead, a duty to warn would focus on a more human question: Did this company know someone was in danger, and did it do enough to warn them and authorities?

By establishing clear guidelines, society can better balance the benefits of AI with the duty to protect life and safety. Legal frameworks must evolve to address the unique capabilities and risks of these technologies. The goal is to prevent harm without stifling innovation or violating privacy rights. The path forward requires careful consideration of ethics, law, and technology. We must ensure that AI serves humanity safely and responsibly. The lessons from past tragedies must guide future regulations.