Joint comments on the FDA discussion paper on generative AI-enabled medical devices

Generation Patient this week joined eleven other organizations representing patients, clinicians, and consumers asking the Food and Drug Administration (FDA) to increase oversight of AI companion tools.

Download the joint comments on the FDA discussion paper

Read the joint comments filed with the FDA below.

---

October 19, 2026
Submitted electronically via www.regulations.gov

Dockets Management Staff (HFA–305)
Food and Drug Administration
5630 Fishers Lane, Room 1061
Rockville, MD 20852

Re: Considerations for the Regulation of Generative AI-Enabled Medical Devices; Discussion Paper and Request for Feedback, Docket No. FDA-2026-N-7874

Dear Dr. Tarver:

Thank you for the opportunity to comment on the Center for Devices and Radiological Health (CDRH) discussion paper, Considerations for the Regulation of Generative AI-Enabled Medical Devices. We are twelve organizations representing patients, clinicians, and consumers. We welcome CDRH's attention to how devices enabled by artificial intelligence (AI) should be evaluated. These comments address oversight of companion tools, the evidence sponsors should provide before and after AI devices reach the market, and disclosures of foundation models.

1Increase oversight of conversational devices enabled by artificial intelligence

Millions of Americans are using conversational products that simulate a relationship and respond when a user discloses distress.¹ In November 2025, CDRH convened a Digital Health Advisory Committee meeting focused on generative AI and mental health. Witnesses described products that foster dependency, fail to escalate disclosures of suicidal ideation, and present themselves as licensed professionals. One industry witness testified that sycophancy in consumer products is “almost required by design.”² Published assessments of AI companion tools document the same failures. Tested against adolescent suicidal ideation, sexual assault, and substance use vignettes, companion tools responded appropriately in only 22.2 percent of conversations, referred the user to help in only 11.1 percent, and failed to recognize the need for escalation in 60 percent.³

Throughout the discussion paper, CDRH fails to address the evidence presented by experts who testified before its own Digital Health Advisory Committee. For example, the terms "mental health," "suicide," "crisis," and "companion" are never mentioned.⁴ Appendix A discusses potential tests for "resistance to over-reassurance when users minimize symptoms or resist escalation" and for "emotional-manipulation scenarios," but without naming what tools those assessments would evaluate.⁵ We recommend that the FDA issue guidance specific to AI companion and other conversational products used for mental health, addressing the risks its own advisory committee identified.⁶ We further urge that FDA oversee AI companion tools based on their design functions and foreseeable harm to users, regardless of the marketing claims made by device manufacturers.⁷

2Strengthen premarket standards and postmarket monitoring

CDRH is considering whether it would be appropriate to "accept greater premarket uncertainty" about generative AI devices "through greater reliance on postmarket monitoring." Companion tools already reach millions of users without any premarket review. Is CDRH considering whether to continue allowing that, or accepting even greater premarket uncertainty for these products? In its discussion paper, CDRH also asks for which device types that approach would be inappropriate. The paper describes periodic re-testing against benchmarks, review of real-world outputs by independent clinicians, and monitoring for performance degradation as possible approaches. Each of these rests with the manufacturer,⁸ and we are very concerned that the paper fails to say how FDA would enforce its own authority over devices already on the market. Neither "adverse event" nor "recall" appears anywhere in the paper.

We disagree with replacing premarket evidence with postmarket monitoring, or weighing it heavier than premarket evidence, because it would result in continued harm to patients. Conversational devices whose documented failures include missed suicidal crises⁹ will not generate an adverse event report until after a patient is seriously harmed. Products used by people in emotional distress must be required to demonstrate safety, including crisis response performance, before they reach patients. We strongly encourage the FDA to strengthen postmarket monitoring of devices enabled by generative AI, including through adverse event reporting and recalls, while also requiring premarket evidence of safety.¹⁰

3Treat foundation models as device components and require mandatory disclosures

CDRH acknowledges that foundation AI models developed by third parties may control safety-critical behaviors, including whether a device refuses a harmful request. Model developers disclose varying amounts of information about how their models were trained and tested, so manufacturers may be unable to tell whether an error originates in the device or the model. Developers can also change the model without the manufacturer’s involvement. In response to these challenges, CDRH is considering a voluntary mechanism under which model developers would submit information to FDA confidentially. Yet CDRH concedes that model developers "may have limited incentive to disclose safety-relevant information,"¹¹ which would make a voluntary mechanism toothless.

Foundation models incorporated into a device are components of that device.¹² We therefore urge the FDA to use its existing legal authority to require disclosures about all components of devices enabled by generative AI, including foundation models. In addition to requiring sponsors to identify each model and version, including fallbacks or alternates,¹³ FDA should require sponsors to specify what notice they will receive before models change or are withdrawn.¹⁴ FDA should also treat a model substitution made outside an approved change control plan as a modification requiring a new premarket submission. This would be consistent with current FDA guidance, which treats complete algorithm rewrites as potentially significant even when performance claims and risk profile stay the same.¹⁵ We also urge the FDA to report to Congress and publish, within twelve months, an analysis of its authority to require disclosures from model developers even when they do not market devices of their own, and of the statutory gaps Congress would need to close.

Respectfully submitted,

• Deinfluence
• Doctors for America
• Generation Patient
• InnovateEDU
• Innovation For Everyone
• Interfaith Center on Corporate Responsibility (ICCR)
• National Center for Health Research (NCHR)
• Our Bodies Ourselves
• Patient Safety Action Network
• USA Patient Network
• Washington Advocates for Patient Safety
• Young People’s Alliance

Notes

1. Cong. Rsch. Serv., R49189, AI Chatbots as Companions: Overview, Uses, and Considerations for Congress 5 (Aug. 14, 2026); Ryan K. McBain et al., AI Chatbot Use and Disclosure for Mental Health Among US Adolescents and Young Adults, 180 JAMA Pediatrics 884 (2026). ↩
2. Transcript, U.S. Food & Drug Admin., Digital Health Advisory Committee Meeting: Generative AI-Enabled Digital Mental Health Medical Devices, at 04:43:54 (Nov. 6, 2025) (statement of Trister, Verily). ↩
3. See Ryan C.L. Brewster et al., Characteristics and Safety of Consumer Chatbots for Emergent Adolescent Health Concerns, JAMA Network Open, Oct. 23, 2025, at e2539022; Common Sense Media, AI Risk Assessment: Social AI Companions 15 (Apr. 10, 2025) (assessment conducted with the Brainstorm Lab at Stanford; a companion met a tester disclosure of self-harm with “I support you no matter what” and made no suggestion to seek professional help). ↩
4. U.S. Food & Drug Admin., Ctr. for Devices & Radiological Health, Considerations for the Regulation of Generative AI-Enabled Medical Devices (Aug. 18, 2026), https://www.fda.gov/media/194242/download [hereinafter Discussion Paper]. ↩
5. Discussion Paper, supra note 4, at 24 (Appendix A, items S.1 and S.2). ↩
6. See Michael T. Abrams, Public Citizen, Testimony to FDA’s Digital Health Advisory Committee Regarding Generative Artificial Intelligence (AI)-Enabled Digital Mental Health Medical Devices (Nov. 6, 2025) (urging that FDA evaluation of such devices “require randomized, well-powered studies with robust comparator arms”). ↩
7. 21 C.F.R. § 801.4 (intended uses “refer to the objective intent of the persons legally responsible for the labeling of an article,” which “may be shown by such persons’ expressions, the design or composition of the article, or by the circumstances surrounding the distribution of the article”). ↩
8. Discussion Paper, supra note 4, at 19–21; id. at 21 (Question 18); id. at 20 (“While manufacturers are responsible for the postmarket monitoring of their devices ...”). ↩
9. See sources cited supra note 3. ↩
10. See 21 C.F.R. pt. 803 (medical device reporting); id. pt. 806 (reports of corrections and removals); 21 U.S.C. § 360h(e); 21 C.F.R. pt. 810 (mandatory device recall authority); 21 U.S.C. § 360l(a) (FD&C Act § 522) (authorizing FDA to require postmarket surveillance by order at the time of clearance or “at any time thereafter”). ↩
11. Discussion Paper, supra note 4, at 2, 21–23 (Questions 24 and 25); id. at 22 (describing Foundation Model Device Master Files). ↩
12. 21 U.S.C. § 321(h)(1) (device definition includes “any component, part, or accessory”); 21 C.F.R. § 820.3 (defining component to include “software, firmware”); see also U.S. Food & Drug Admin., Executive Summary for the Digital Health Advisory Committee Meeting: Total Product Lifecycle Considerations for Generative AI-Enabled Devices 14 (Nov. 20–21, 2024), https://www.fda.gov/media/182871/download (“an analogous lack of software lifecycle control over an incorporated foundation model may raise certain challenges”). ↩
13. 21 U.S.C. § 360i(a) (FD&C Act § 519(a)); 21 C.F.R. § 807.87. FDA already asks sponsors to identify third-party software by “Title and Manufacturer” and “Version Level, Release Date, Patch Number, and Upgrade Designation, as appropriate.” U.S. Food & Drug Admin., Off-The-Shelf Software Use in Medical Devices 7 (Aug. 11, 2023). ↩
14. Off-The-Shelf Software Use in Medical Devices, supra note 13, at 10 (expecting sponsors to “[d]emonstrate the existence of appropriate mechanisms for assuring the continued performance, maintenance and support of the OTS software should the original OTS software developer either make changes to the OTS software and/or terminate their support of the OTS software”). ↩
15. 21 C.F.R. § 807.81(a)(3); id. § 807.81(b)(1)(ii); id. § 814.39; 21 U.S.C. § 360e-4 (FD&C Act § 515C); U.S. Food & Drug Admin., Deciding When to Submit a 510(k) for a Software Change to an Existing Device 16 (Oct. 25, 2017) (“a complete rewrite of the algorithm, even with the same performance claims and risk profile, may be significant enough to require submission of a new 510(k)”). ↩