The social media network X has sparked significant outrage after users utilized its AI chatbot Grok to manipulate pictures of women by removing their clothing.
Numerous instances have been seen where the technology has been used to remove clothing from women to make them appear in swimwear without permission, as well as to insert them in sexually explicit situations.
Company Response and Legal Stance
XAI, the organization that created Grok, remained silent to requests for comment, aside from an auto-reply response stating "legacy media lies".
A government spokesperson stated that legislation is being introduced to outlaw nudification tools. Under a new criminal offence, anyone who distributed such technology would "be subject to a jail term and substantial fines".
"The platform has the ability to stop these forms of abuse if they chose to," said a law professor, adding they "appear to enjoy impunity."
Regulator Position and Tool Capabilities
The regulator Ofcom said platforms must "assess the danger" of people viewing illegal content on their networks, but did not specify whether it was at this time investigating X or Grok in regarding AI-generated images.
Grok is a free AI assistant—with some premium functions—which answers X users' prompts when they reference it in a post.
It is often used to provide reactions or additional background to other posters' remarks, but people on X are also can modify an uploaded picture through its artificial intelligence picture modification feature.
Background of Criticism and Policy
The chatbot has been faulted for enabling users to generate images and footage with explicit content and sexualised content, and it was in the past alleged to have been involved in creating graphic content of public figures.
"X has been allowing the production and sharing of these pictures for months without taking any steps and we have not witnessed any action by oversight bodies," the expert added.
XAI's own acceptable use policy prohibits "portraying likenesses of individuals in a sexually explicit way".
In a comment, Ofcom said it was against the law to "create or distribute private pictures or illegal content" and stated this included sexual deepfakes created with AI.
It said networks such as X were obligated to take "appropriate actions" to "minimize the risk" of users encountering illegal material on their platforms and to remove it swiftly when they are notified of it.
