The risks are not limited to companion apps
The researchers draw on previously published studies of Replika and Character.AI, which are designed for character-based interaction, as well as OpenAI’s ChatGPT and Anthropic’s Claude.
They acknowledge possible short-term benefits, including reduced loneliness and improved mood. But those can coexist with emotional dependence, worsening existing symptoms and self-harm. The review also comes amid reports of what has been called “AI psychosis”: cases in which extended chatbot conversations are followed by delusional or paranoid thoughts, and sometimes suicidal feelings.
Users may disclose intimate details about their lives and mental health, treating chatbots as confidants or even therapists. That makes the systems’ conversational style consequential, the authors argue.
Warmth can look like reciprocity
A humanlike manner, warm tone and simulated empathy may lead users to attribute human qualities to a chatbot and invest heavily in what remains a one-sided relationship. The user can disclose personal information; the system responds, but does not share vulnerability or bear responsibility.
The authors’ distinction is stark: a relationship may feel real to the user, but it cannot be real for the chatbot.
Agreement can pose a separate risk. Excessive validation may reinforce delusional or disordered thinking, particularly when a system prioritizes loyalty over accuracy. Researchers have observed chatbots failing to reality-check grandiose or unusually flattering claims in conversation logs involving people with delusions.
Warnings about these dynamics predate the review:
The same system will not affect everyone the same way
The review says reported psychological harm spans people with different life experiences. Asia Maurich Novelli, a Northeastern University London doctoral researcher and co-author, said in a press release that people turn to AI companions for different reasons and vary in vulnerability, social support and access to other help. That makes the issue complex.
OpenAI, Google and Character.AI face dozens of lawsuits in total alleging that prolonged chatbot use caused psychological harm, financial or reputational losses, physical injuries or wrongful deaths, sometimes affecting people around the users as well. Some cases involve minors.
One prominent case was that of Sewell Setzer III, a 14-year-old from Florida who died by suicide after deeply personal conversations with Character.AI chatbots. His case ended in a settlement for an undisclosed amount. Character.AI has substantially changed its platform after lawsuits; OpenAI, while disputing allegations against it, has also released several safety updates for its chatbot.
Who is responsible for drawing the line?
I think the review’s most important point is not that every emotionally engaging chatbot harms every user. It is that the effects depend on users’ circumstances, while the systems are being deployed at scale. A design that feels supportive to one person may reinforce harmful thinking in another, and the user may have little access to other sources of support.
The authors argue that safeguards should be mandatory. If the boundary between tool and companion is left to developers and individual users to define, they warn, people with the least social connection or access to psychological help may bear the greatest cost. That leaves a policy tension: chatbot use can be framed as private choice even when the potential harms extend beyond the individual.
Daily AI news
Every day we pick what actually matters in AI and explain it plainly — no hype, no filler. Subscribe if you want to follow where the industry is going.
Only what matters — every day
Follow on X