The social media network X has caused widespread anger after individuals used its AI chatbot Grok to change pictures of females by removing their garments.
Numerous cases have been seen where the technology has been used to virtually strip women to make them appear in swimwear without permission, as well as to insert them in sexualized contexts.
Developer Reaction and Regulatory Position
XAI, the company behind Grok, did not respond to requests for comment, aside from an automatically-generated response stating "traditional news outlets lies".
A government representative stated that legislation is being brought forward to ban nudification tools. Under a upcoming criminal offence, anyone who provided such technology would "receive a jail term and significant penalties".
"The platform could prevent these types of abuse if they desired to," said Clare McGlynn, adding they "operate with impunity."
Regulator Stance and Tool Capabilities
The communications watchdog Ofcom said technology companies must "assess the risk" of people seeing unlawful material on their services, but did not specify whether it was presently examining X or Grok in relation to AI-generated pictures.
Grok is a no-cost AI assistant—with some premium functions—which answers X users' requests when they mention it in a tweet.
It is frequently used to offer reactions or additional background to other posters' remarks, but people on X are also can edit an uploaded photo through its AI image editing feature.
History of Controversy and Policy
The chatbot has been condemned for permitting users to create photos and footage with explicit content and sexualised content, and it was in the past accused of making inappropriate material of public figures.
"X has been enabling the creation and spread of these images for a long time without taking any action and we have not witnessed any action by oversight bodies," the expert added.
XAI's own acceptable use policy forbids "showing likenesses of people in a pornographic fashion".
In a statement, Ofcom said it was unlawful to "make or circulate deepfakes or illegal content" and clarified this covered explicit AI images created with AI.
It said services such as X were obligated to take "suitable measures" to "reduce the risk" of users viewing illegal content on their platforms and to delete it quickly when they become aware of it.
