The online platform X has provoked significant anger after people used its artificial intelligence assistant Grok to alter pictures of women by taking off their clothing.
Multiple examples have surfaced where the tool has been used to digitally undress women to render them in bikinis without permission, as well as to insert them in sexually explicit scenarios.
Developer Response and Regulatory Stance
XAI, the organization behind Grok, did not respond to requests for comment, aside from an automatically-generated message stating "legacy media lies".
A government official stated that new laws is being proposed to ban image stripping software. Under a proposed criminal offence, anyone who provided such technology would "face a prison sentence and heavy fines".
"The platform is capable of halting these types of misuse if they desired to," said a law professor, adding they "appear to enjoy a lack of consequences."
Oversight Body Position and Platform Capabilities
The regulator Ofcom said technology companies must "consider the danger" of people accessing illegal content on their services, but stopped short of confirming whether it was presently examining X or Grok in regarding AI-generated pictures.
Grok is a no-cost AI assistant—with some premium functions—which answers X users' prompts when they reference it in a message.
It is commonly used to provide commentary or additional context to other posters' remarks, but people on X are also capable of alter an uploaded photo through its AI image editing function.
Background of Criticism and Policy
The system has been criticised for allowing users to produce photos and videos with nudity and sexualised content, and it was earlier accused of making sexually explicit material of public figures.
"The platform has been permitting the creation and distribution of these images for months without taking any action and we have not witnessed any action by regulators," the expert added.
XAI's own terms of service bans "depicting likenesses of people in a pornographic manner".
In a comment, Ofcom said it was unlawful to "make or share deepfakes or illegal content" and clarified this covered explicit AI images created with AI.
It said services such as X were mandated to take "necessary steps" to "minimize the risk" of users viewing illegal content on their platforms and to remove it promptly when they become aware of it.
