Case Study 2: AI in Mental Health — Chatbots, Crisis Lines, and Care

The Context

Mental health care faces a global crisis of access. According to the World Health Organization, approximately 970 million people worldwide live with a mental health condition, and in low-income countries, more than 75 percent of people with mental health disorders receive no treatment at all. Even in wealthy nations, wait times for therapy can stretch to months, and cost is a significant barrier for many.

AI-powered mental health tools have emerged as a potential response to this access gap. They range from chatbots that deliver structured therapeutic exercises to AI systems that monitor social media posts for signs of crisis, to tools that analyze speech patterns to detect depression. Proponents see them as a lifeline for people who cannot access traditional care. Critics see them as a dangerous substitution — cheap imitations of the deeply human work of therapy, deployed without adequate evidence or oversight.

Both sides have a point.

The Tools

Therapeutic Chatbots

Several companies have developed AI chatbots designed to deliver evidence-based therapeutic techniques, particularly cognitive behavioral therapy (CBT). These chatbots guide users through exercises such as identifying negative thought patterns, challenging cognitive distortions, and practicing relaxation techniques.

Woebot, one of the most studied therapeutic chatbots, was developed by a team at Stanford University. It uses a combination of natural language processing and scripted therapeutic content to engage users in daily check-ins. A 2017 randomized controlled trial published in JMIR Mental Health found that college students who used Woebot for two weeks reported significant reductions in depression symptoms compared to a control group. The study was small (70 participants) and short (two weeks), but it was one of the first to provide experimental evidence for a mental health chatbot.

Other chatbots — such as Wysa, Youper, and Replika — offer similar features with varying degrees of clinical grounding. Some are classified as "wellness apps" rather than medical devices, which means they face minimal regulatory scrutiny.

Crisis Detection and Response

In 2022, the National Eating Disorders Association (NEDA) replaced its human-staffed helpline with an AI chatbot called Tessa. The decision was controversial from the start, and the controversy deepened when users reported that Tessa gave advice that could be harmful to people with eating disorders — including suggesting calorie counting and dietary restriction to users who had identified themselves as struggling with disordered eating.

NEDA suspended Tessa after these reports. The incident became a cautionary tale about deploying AI in contexts where the wrong advice can cause direct, physical harm.

In a separate domain, the Crisis Text Line — a text-based crisis intervention service — came under scrutiny when it was revealed that the organization had shared anonymized conversation data with a for-profit spinoff company, Loris.ai, which used the data to develop customer service AI tools. The data came from people in crisis — some of whom were suicidal. The ethical implications of commercializing crisis data, even in anonymized form, sparked widespread concern.

Predictive and Monitoring Tools

Some AI systems aim to identify people at risk of mental health crises before they seek help. These include:

• Systems that analyze social media posts for linguistic markers associated with depression, anxiety, or suicidal ideation.
• Voice analysis tools that detect changes in speech patterns that may indicate depressive episodes.
• Smartphone apps that monitor behavioral patterns (sleep, activity, social interaction) as potential indicators of mental health changes.

These tools raise profound questions about consent, surveillance, and the boundary between care and intrusion.

The Promise

The strongest argument for AI in mental health is access. Traditional therapy requires a trained clinician, a scheduled appointment, transportation, and — in many systems — money. AI tools are available 24/7, cost little or nothing, and can reach people who would never walk into a therapist's office due to stigma, geography, or cost.

For some populations, the anonymity of interacting with a chatbot may actually be an advantage. Research suggests that some people disclose more to a computer than to a human, particularly around stigmatized topics like substance use, sexual health, and suicidal ideation. A chatbot does not judge. It does not look surprised. It does not cancel appointments.

There is also evidence that structured therapeutic techniques — particularly CBT exercises — can be effectively delivered through digital formats. The question is not whether digital delivery works in principle, but whether current AI tools deliver it safely and effectively in practice.

The Peril

Clinical Safety

Mental health care is not like troubleshooting a computer. A person in crisis may need immediate, nuanced, context-sensitive intervention. They may say one thing and mean another. They may be testing whether the listener cares. They may need to be talked through a dangerous moment with the kind of relational sensitivity that no current AI possesses.

The Tessa incident illustrates the danger concretely. An AI system giving generic dietary advice to a person with an eating disorder is not just unhelpful — it is actively harmful. The system had no understanding of the clinical context, no ability to recognize that its standard advice was dangerous for this specific user.

The Therapeutic Relationship

Decades of psychotherapy research have established that the therapeutic relationship — the bond between therapist and client — is one of the strongest predictors of treatment outcomes. This relationship involves trust, empathy, attunement, and the experience of being truly seen and understood by another human being.

An AI chatbot can simulate some surface features of empathy ("That sounds really difficult. I'm sorry you're going through that."), but it does not experience empathy. It does not remember you in the way a therapist does. It does not notice that your voice sounds different today, or that you are avoiding a topic you have discussed before. The question is whether the therapeutic techniques alone — divorced from the relationship — are sufficient.

Data Ethics

Mental health data is among the most sensitive data that exists. Users of mental health chatbots disclose information about their psychological state, their traumas, their relationships, and their most vulnerable moments. How this data is stored, who has access to it, and whether it could be used for purposes the user did not anticipate are critical concerns.

The Crisis Text Line data-sharing controversy highlights this tension. Even anonymized data from mental health interactions carries risks of re-identification and misuse. Users who reach out in moments of crisis are not typically thinking about data policies.

Equity

Do AI mental health tools reduce equity gaps or widen them? The answer may be both. On one hand, free chatbots are more accessible than $200-per-session therapy. On the other hand, if affluent patients have access to human therapists while low-income patients are directed to chatbots, AI could create a two-tiered mental health system: real therapy for those who can afford it, automated therapy for everyone else.

The Regulatory Landscape

Most mental health chatbots are not regulated as medical devices. By classifying themselves as "wellness" products rather than therapeutic tools, they avoid the clinical evidence and oversight requirements that apply to medical devices. This means that many of the chatbots currently available to consumers have never been rigorously tested in clinical trials, have no obligation to report adverse events, and face no regulatory consequences if they cause harm.

Some voices in the field are calling for stronger regulation. Others argue that over-regulation would stifle innovation and block access for people who need help now. The tension between access and safety is real and unresolved.

Discussion Questions

1. The Access Argument: The strongest case for AI mental health tools is that they can reach people who would otherwise receive no care at all. How much weight should this argument carry? Is a flawed tool better than no tool?

2. The Tessa Incident: NEDA replaced its human helpline with an AI chatbot that gave harmful advice. What went wrong? What safeguards could have prevented this? Should organizations be held liable for harm caused by AI chatbots they deploy?

3. Data Ethics: When a person texts a crisis line at 2 a.m. saying they want to die, they are not thinking about data policies. Should their conversation data ever be used for purposes beyond their immediate care? Under what conditions, if any?

4. The Two-Tiered System: If AI mental health tools become the default for low-income patients while affluent patients see human therapists, is this an equity improvement (some care vs. no care) or an equity failure (inferior care for those who already face disadvantages)?

5. The Therapeutic Relationship: Can an AI chatbot provide the therapeutic relationship that research identifies as central to mental health treatment? If not, does this make chatbots inappropriate for clinical use, or does it mean we need to rethink what matters in therapy?

6. Regulatory Position: Should mental health chatbots be regulated as medical devices? Write a 100-word argument for your position.

7. Personal Reflection: Would you use an AI chatbot for mental health support? Why or why not? What would make you more or less likely to trust such a tool?