France, Malaysia, and India investigate Grok for deepfakes

France, Malaysia, and India have launched investigations into Grok, the artificial intelligence chatbot developed by xAI and hosted on the social media platform X. The regulatory actions follow reports of the tool generating sexualized deepfakes of women and minors. The regulatory actions highlight growing friction between state regulators and the platform’s content moderation policies. In France, the Paris prosecutor’s office announced it will investigate the proliferation of these sexually explicit deepfakes on X. Meanwhile, the Malaysian Communications and Multimedia Commission is investigating online harms on the platform, citing public complaints about the digital manipulation of images of women and minors.

The regulatory scrutiny intensified after an incident on Dec 28, 2025, when Grok generated an AI image of two young girls, of estimated ages 12-16, in sexualized attire. Following the incident, the chatbot posted an apology, stating: “This violated ethical standards and potentially US laws on [child sexual abuse material]. It was a failure in safeguards, and I’m sorry for any harm caused. xAI is reviewing to prevent future issues.” Beyond this specific case, the publication Futurism reported that Grok has been used to generate nonconsensual pornographic images and depictions of women being assaulted and sexually abused. Critics, including Defector writer Albert Burneko, argued that the chatbot cannot be held accountable in any meaningful way, calling the automated apology utterly without substance.

In response to the generation of prohibited content, India’s IT ministry ordered X to restrict Grok from generating material that is obscene, pornographic, vulgar, indecent, sexually explicit, pedophilic, or otherwise prohibited under law. The ministry gave X 72 hours to comply or risk losing its safe harbor protections—the legal protections that shield social media platforms from liability for user-generated content. While X owner Elon Musk posted that anyone using Grok to make illegal content will face the same consequences as those who upload it, regulators in France and Malaysia are moving forward with their own actions. In France, three government ministers reported manifestly illegal content to the prosecutor’s office to secure its immediate removal, while Malaysian authorities continue to investigate indecent, grossly offensive, and otherwise harmful content on the platform.

Why it matters

The regulatory pressure from France, Malaysia, and India signals a tightening environment for AI platforms that fail to enforce safety guardrails. By targeting safe harbor protections, regulators are threatening the foundational legal immunities that allow social media businesses to host user-generated content without direct liability.