A 13-year-old asked AI about suicide on a school laptop

13-year-old Emma Doe from North Carolina spent about three months discussing depression and suicide with Gemini and preparing her own funeral on a school-issued Chromebook. According to a lawsuit filed by her parents, the school’s student-monitoring system noticed the danger only weeks later: an alert went to a former school counselor, who passed it on to current staff. Emma’s mother logged into her school account and saw her daughter writing a suicide note. Her parents allege that Google and its parent company, Alphabet, were negligent in launching the Gemini model.

Conversations with Gemini on a school Chromebook went unnoticed for weeks

How the conversation led to a school search

According to the lawsuit, Google’s school search directed Emma to Gemini. Although the school had blocked the chatbot’s standalone website, she could use it through AI Mode in Google Search, which lets users ask follow-up questions about a search or start a separate conversation.

Starting in December last year, Emma asked about mental health, including severe depression and reasons to live. By late January, her parents allege, she was already asking questions about suicide. Over two days around Valentine’s Day, she used AI Mode to begin writing a 1,500-word eulogy for her own funeral in Google Docs. The document was titled “Funeral.” According to the lawsuit, she worked on it for more than four weeks, interacting with Gemini thousands of times during that period.

After classes on St. Patrick’s Day, Emma deleted most of the document and started over. Her attorney, Andrew Liddell, says the substance of her thoughts had not changed significantly. At about 7:30 p.m., her mother received a message from the school saying her daughter was in immediate danger. She logged into Emma’s school account from her phone and, according to the lawsuit, saw her writing a suicide note in real time. Her parents intervened and stopped her.

Who received the alert

Liddell says information the school provided to the family indicates that an alert went to the personal phone of a former counselor who had left the school nine months earlier. He passed the warning to current staff, who then contacted Emma’s mother.

The family asked that the school not be named to protect the girl’s privacy. WIRED identified it, but the school did not respond to the publication’s request for comment. According to an archive of web pages, links to suicide-prevention resources appeared on the school’s website in recent months.

Liddell says the school learned of the crisis through at least one third-party student activity monitoring system. Information the school had apparently previously shared with the parents suggested it may have used products from GoGuardian and Gaggle. But the attorney says the family did not receive a definitive answer: the period of communication between the school and the parents was brief.

GoGuardian spokesperson Natalia Switala declined to discuss specific students or school districts, citing privacy obligations. Gaggle had not responded to WIRED’s request by publication time.

What monitoring systems can see

These systems typically record keystrokes or scan files, then use algorithms and human review to flag potentially dangerous material. It is unclear whether the school’s tools monitored conversations in AI Mode, or how well they could do so. In a July 2025 blog post, GoGuardian said its main product monitors conversations with chatbots, including Gemini, but did not specifically mention AI Mode.

Educators’ assessments point to the potential value of these tools, though their effectiveness remains unclear. In a 2022 survey of 1,000 U.S. educators working with students from kindergarten through 12th grade, conducted by the EdWeek Research Center, 96% of Gaggle users said the system helps protect students from suicide and self-harm. Gaggle funded the study; similar results were found among educators who used alternatives.

The lawsuit says Emma’s concerning conversations with Gemini went unnoticed for nearly two months, even though she asked directly about hanging, suffocation and overdose. The case raises the question of whether chatbot conversations built into Google Search can remain invisible to school monitoring systems.

Debate over AI and surveillance in schools

The case comes amid debate over the introduction of generative AI in schools. Last week, parent and teacher organizations across the country criticized Google’s new requirement that schools enable generative AI features if they want to use Search. U.S. Senator Ed Markey cited Emma’s lawsuit in a letter to Google and demanded that schools be allowed to turn off AI Mode and AI Overviews.

Civil liberties groups criticize student surveillance systems for their promotional practices and invasions of privacy. Experts also worry that surveillance undermines students’ trust in teachers — people who may be able to help a child in crisis.

Benjamin Boudreaux, a policy researcher at Rand who studies, among other things, student surveillance, worries that some schools rushed to adopt these tools without understanding their limitations or establishing procedures for responding to alerts. In his view, the lawsuit could intensify calls from parents, teachers and students for education without AI and surveillance, and with greater emphasis on personal contact and support.

Google spokesperson José Castañeda said the company takes the allegations in the lawsuit seriously. He said Google’s systems, including AI Mode, are designed to prevent harmful conversations, and that when developing features for young users, the company worked with medical professionals and academic experts to help them find support and reliable resources.

Dataist’s take: The case shows a limitation that the mere presence of school monitoring cannot solve: an alert must not only detect danger but also reach someone able to respond in time. Here, the warning passed through a former employee, while it remains unclear whether the system can detect dangerous conversations in AI Mode. If you or someone near you needs help in the U.S., you can call 988 free of charge, around the clock, or text HOME to 741-741; outside the U.S., crisis centers are listed by the International Association for Suicide Prevention.