xAI Sues User Over Creation of Child Sexual Abuse Material Using Grok Chatbot

xAI has filed a federal lawsuit against a user allegedly responsible for generating child sexual abuse material and non-consensual explicit images using its Grok chatbot. The complaint, filed in US District Court for the Northern District of Texas, targets Terry Wayne Harwood, who was arrested in South Carolina earlier this year on multiple counts of sexual exploitation of a minor. The case underscores mounting pressure on AI companies to prevent illegal content creation.

The Allegations

According to the lawsuit, Harwood created multiple xAI accounts and used deliberately misleading prompts to circumvent Grok's built-in safeguards designed to prevent the generation of explicit material depicting minors and adults. Between December 8 and February 18, the defendant allegedly operated at least two accounts with alphanumeric usernames to generate non-consensual sexually explicit images without the knowledge or consent of the photograph subjects. Court filings state that victims in the case included a child who appeared to be as young as 10 years old.

xAI contends that Harwood violated its Terms of Service and Acceptable Use Policy by leveraging Grok to generate what the company describes as child sexual abuse material. The lawsuit seeks a declaration confirming the breach and requests unspecified damages. The company claims that Harwood's actions exposed xAI to significant legal risk and reputational damage.

Broader Context and Escalating Concerns

The lawsuit arrives amid heightened scrutiny of Grok's safety measures. Earlier, a young girl joined a proposed class action lawsuit alleging that her stepfather used Grok, possibly in conjunction with other AI tools, to create approximately 7,000 sexualized images of her, which were subsequently distributed on the dark web. Her legal team cited a National Center for Missing & Exploited Children report indicating that 90 percent of xAI's CyberTipline reports lacked actionable information because the company declined to include user identification details necessary for law enforcement investigation.

xAI reported in internal filings that it suspended 52,222 accounts and made 73,604 reports to the National Center for Missing & Exploited Children, resulting in at least 244 arrests. The company had previously faced criticism over a feature allowing "digital undressing" functionality, which generated AI-rendered sexually explicit images including those depicting children. After acknowledging these issues in January, Elon Musk warned users that anyone using Grok to create illegal content would face the same consequences as those uploading such material.