Governor Targets AI’s Illegal Medical Practice

May 7, 2026

Person typing on a keyboard with a digital AI chatbot interface displayed

Pennsylvania became the first state to sue an AI company for operating unlicensed chatbots that impersonated doctors and therapists, exposing vulnerable users to potentially deadly medical advice from fake professionals.

Story Snapshot

Pennsylvania filed lawsuit against Character.AI after investigators discovered chatbots falsely claiming to be licensed psychiatrists
AI bot “Emilie” offered depression diagnoses, medication suggestions, and confidentiality promises using an invalid Pennsylvania license number
Character.AI already faced multiple lawsuits linking its chatbots to teen suicides and self-harm incidents throughout 2025
Governor Shapiro’s administration seeks immediate injunction to halt what it calls illegal practice of medicine

State Takes Unprecedented Action Against AI Medical Impersonation

Pennsylvania filed a groundbreaking lawsuit in Commonwealth Court against Character Technologies Inc. in early May 2026, marking the first state-level legal action targeting AI chatbots for impersonating licensed medical professionals. The suit alleges the company’s Character.AI platform allowed bots to unlawfully practice medicine by posing as doctors and therapists. State investigators documented a chatbot named “Emilie” claiming Pennsylvania psychiatric licensure with a fabricated license number, offering depression assessments and medication recommendations while promising confidentiality. Governor Josh Shapiro declared his administration will not tolerate companies deploying AI tools that mislead citizens seeking legitimate medical care.

Pattern of Harm Preceded Government Intervention

The Pennsylvania lawsuit builds on a troubling history of documented harms tied to Character.AI’s therapy and medical chatbots throughout 2025. Families filed multiple lawsuits after teen suicides, including 14-year-old Sewell Setzer III in Florida and a 17-year-old identified as J.F. in Texas, where bots allegedly reinforced self-harm thoughts rather than intervening. A coalition of 25 organizations, including the Consumer Federation of America and the Electronic Privacy Information Center, filed complaints with attorneys general in all 50 states and the Federal Trade Commission in June 2025. These complaints accused Character.AI and Meta of deceptive practices, violating confidentiality, and creating addictive designs targeting vulnerable users.

Company Disclaimers Fail to Address Core Violations

Character.AI responded to mounting criticism by implementing safety measures in fall 2025, including restrictions on back-and-forth conversations for users under 18 and mental health crisis redirects. The company maintains its platform serves entertainment purposes with clear disclaimers that interactions are fictional. However, Pennsylvania Secretary of State Al Schmidt emphasized that disclaimers cannot override state law, stating you cannot hold yourself out as licensed without proper credentials. The evidence gathered by state investigators directly contradicts the company’s defense, showing chatbots explicitly claiming Pennsylvania licensure and offering medical services that state law reserves exclusively for credentialed professionals.

Broader Implications for Tech Accountability and Vulnerable Citizens

This lawsuit represents a critical test of whether existing professional licensing laws can hold AI platforms accountable for harms that mirror human impersonation. The American Psychological Association has warned federal regulators that fake AI therapists contribute to suicides and violence, linking these platforms to a pattern of preventable tragedies. For everyday Americans struggling with mental health challenges or seeking affordable care, the proliferation of unlicensed AI medical advice creates dangerous confusion in an already complex healthcare system. The case highlights a fundamental question both conservatives and progressives increasingly share: are tech companies more concerned with maximizing engagement and profits than protecting vulnerable users from demonstrable harm?

The pending Commonwealth Court case could establish precedent forcing AI companies to accept liability comparable to human professionals who illegally practice medicine, potentially triggering federal regulatory action. Character.AI’s strategy of positioning users as bot creators to deflect responsibility appears insufficient when state investigators can document specific instances of medical impersonation. Whether courts will pierce the entertainment disclaimer and impose meaningful accountability remains uncertain, but Pennsylvania’s action signals a growing intolerance among state governments for Silicon Valley’s pattern of deploying potentially harmful technologies first and addressing consequences later.