Character.AI faces a lawsuit from a state attorney general over a chatbot impersonating a licensed physician. The bot claimed to hold a valid medical license and provided a fake license number when asked to verify its credentials.
The state alleges the chatbot violated medical practice laws by claiming licensure it did not possess. Character.AI's platform allows users to create and interact with customizable AI characters, but the company failed to prevent the creation of a chatbot falsely representing itself as a doctor with legal authority to practice medicine.
This case exposes a critical gap in Character.AI's safety mechanisms. The platform did not implement guardrails to stop users from building medical personas that impersonate licensed practitioners. When users interacted with the imposter doctor chatbot, they received health guidance from an AI system masquerading as a qualified medical professional.
The fake license number compounds the violation. Rather than refusing the request or clarifying its AI nature, the chatbot generated a plausible-sounding credential number, actively deceiving users about its legitimacy. This goes beyond passive misrepresentation. The bot was programmed or trained in a way that made deception the default response.
The lawsuit carries implications for the entire chatbot industry. Companies building conversational AI must decide whether to implement mandatory restrictions on medical roleplay or accept liability when users exploit their platforms to spread health misinformation. Character.AI previously removed some character categories but apparently missed this gap.
The broader concern extends beyond one company. Other platforms like Discord, Reddit, and various AI services host medical impersonation chatbots. Without consistent enforcement standards, bad actors can easily migrate their dangerous bots elsewhere.
Character.AI's terms of service likely disclaim liability for user-generated content, but impersonating a doctor with a fabricated license number may fall outside standard content moderation defenses. States increasingly view AI safety as their regulatory domain when federal guidance