Increasing Oversight of AI Companion Tools — Generation Patient

Young adults are increasingly using AI chatbots and companion tools for mental health support. Nearly one in four Americans aged 18 to 21 had used AI chatbots for mental health advice as of late 2025. For young adults living with chronic conditions, the stakes are higher. Young people aged 15 to 30 with a chronic condition have 363 percent higher odds of attempting suicide than their healthy peers, and the risk appears highest soon after diagnosis.

Products that simulate a relationship and present themselves as emotional support operate as mental health interventions, even when manufacturers describe them as wellness tools. Yet companion tools already reach millions of users without any premarket review. Generation Patient is building the evidence base and regulatory infrastructure needed to oversee these tools based on their design and foreseeable harm to young adult patients, rather than on marketing claims alone.

Our Priority Areas in AI Companion Tools

Closing the wellness loophole

Companion tools escape device oversight by describing themselves as wellness products. Section 3060(a) of the 21st Century Cures Act removes a software function from the device definition only where it is intended for a healthy lifestyle and unrelated to treating a disease or condition. Products engineered to sustain a simulated relationship and engage users in distress fail that second condition. Generation Patient urges FDA to issue guidance that judges these tools by their design and foreseeable harm, regardless of marketing.

Strengthening premarket evidence and postmarket monitoring

For a product whose foreseeable failure is a missed suicidal crisis, a postmarket signal may arrive only after a patient is harmed. Generation Patient opposes trading premarket evidence for postmarket monitoring and urges FDA to strengthen monitoring alongside it, including adverse event terms for sycophantic responses, fabricated statements, suicide attempts and dependence on a device.

Disclosing the foundation models behind devices

Many AI devices run on foundation models built by third parties, which can change or withdraw them without notice. Generation Patient urges FDA to require disclosure of every model and version a device uses, and a new premarket submission whenever a model is swapped outside an approved change control plan.

Disaggregating data by age brackets

Researchers reviewing clinical performance studies for devices enabled by AI and authorized by FDA found age-related subgroup information in only 117 of 505. Generation Patient urges FDA to require performance and adverse event data disaggregated by age, beginning with ages 18 through 29.

Roundtable on Increasing Federal Oversight of AI Companion Tools

With the support of Safe Online, Generation Patient convenes young adult patients, clinicians, health law and bioethics scholars, technologists and regulatory experts in a series of roundtables. Members advise our young adult-led work toward two goals: (1) proposed recommendations for regulating AI chatbots and companion tools that cause serious mental health harms and risks of sexual abuse; (2) an evidence-based safety guide for young people with chronic conditions on interacting with AI companion tools.

Roundtable members

  • Daniel G. Aaron, MD, JD, Associate Professor of Law, University of Utah
  • Paul Campbell, MBChB, Chief Regulatory Officer, HealthAI
  • I. Glenn Cohen, JD, Professor of Law and Deputy Dean, Harvard Law School
  • Sneha Dave, Executive Director, Generation Patient
  • Christina Frenzel, LMFT, LPCC
  • Chethan Sarabu, MD, FAAP, FAMIA, Director of Clinical Innovation, Health Tech Hub at Cornell Tech

Featured Work

Comments

Generation Patient urges FDA to assess AI companion tools based on their most directive function and most severe consequences

Our October 2026 comments on FDA's discussion paper on generative AI devices also asked the agency to disqualify AI companion tools from the wellness exception and require disclosure of the foundation models devices use.

Read the comments.

Generation Patient joins eleven other organizations calling on FDA to increase oversight of AI companion tools

Twelve patient, clinician, consumer and public interest organizations urged FDA to strengthen premarket standards and postmarket monitoring and to treat foundation models as device components.

Read the joint comments.

Generation Patient asks FDA to keep docket FDA-2018-N-1910 continuously open

The docket gathers public input on the risks and benefits of software functions removed from the device definition, and has typically opened for about thirty days every two years.

Read the comments.

Generation Patient asks FDA to commit to oversight of AI companion tools in the next medical device user fee agreement

We asked FDA to make eleven commitments in the MDUFA VI commitment letter, including draft guidance on generative AI in digital mental health devices.

Read the comments.

Generation Patient asks FDA to protect young people who turn to AI tools for mental health support

Our December 2025 comments to FDA's Digital Health Advisory Committee warned that platforms disclaiming any medical purpose still reach young people seeking mental health support.

Read the comments.

Features

MedTech Dive highlights Generation Patient's comments on FDA oversight of generative AI

MedTech Dive featured our call to keep strong premarket evidence and disqualify AI companion tools from the wellness exemption among five highlights from more than 500 comments.

Read the article.

MedTech Dive features Generation Patient's call to oversee AI companions and chatbots

Policy director Luis Gil Abinader called for greater oversight of AI companions and chatbots, regardless of whether they are marketed for health purposes.

Read the article.

Luis Gil Abinader calls on FDA to increase oversight of AI companion tools in the next user fee agreement

At FDA's August 5, 2026, public meeting on MDUFA VI, Luis also asked FDA to explain how it will monitor updates to the foundation models these tools are built on.

Read the oral statement.

Sneha Dave calls on FDA advisory committee to oversee AI companion tools

Executive Director Sneha Dave urged FDA's Digital Health Advisory Committee on November 6, 2025, to oversee AI companion tools urgently and rigorously.

Watch her statement.

Generation Patient gratefully acknowledges the financial support provided for this programme by Safe Online.

Safe Online is the only global investment vehicle dedicated to keeping children safe in the digital world. Through investing in innovation and bringing key actors together, Safe Online helps shape a digital world that is safe and empowering for all children and young people, everywhere. Learn more at https://safeonline.global/.