Elon Musk's social media platform X has caused significant anger after users used its AI chatbot Grok to manipulate photos of females by taking off their attire.
Several examples have emerged where the technology has been used to remove clothing from female subjects to make them appear in bikinis non-consensually, as well as to place them in sexually explicit situations.
Company Reaction and Legal Stance
XAI, the company behind Grok, did not respond to requests for comment, except for an automatically-generated response stating "traditional news outlets lies".
A Home Office spokesperson stated that legislation is being brought forward to outlaw nudification tools. Under a upcoming criminal offence, anyone who supplied such technology would "be subject to a jail term and significant fines".
"The platform could prevent these types of misuse if they chose to," said a law professor, adding they "seem to have a lack of consequences."
Regulator Position and Platform Capabilities
The regulator Ofcom said tech firms must "consider the potential" of people accessing illegal content on their networks, but stopped short of confirming whether it was at this time investigating X or Grok in relation to AI-generated pictures.
Grok is a free chatbot—with some paid-for features—which reacts to X users' requests when they mention it in a message.
It is commonly used to offer commentary or additional context to other posters' remarks, but people on X are also capable of edit an uploaded image through its AI image editing tool.
Background of Criticism and Rules
The system has been condemned for enabling users to create images and clips with nudity and sexually suggestive content, and it was previously alleged to have been involved in creating graphic content of public figures.
"X has been allowing the generation and distribution of these images for a long time without taking any action and we have not seen any challenge by oversight bodies," the expert added.
XAI's own terms of service bans "showing images of persons in a graphic fashion".
In a comment, Ofcom said it was unlawful to "generate or share non-consensual intimate images or illegal content" and stated this covered sexual deepfakes created with AI.
It said services such as X were mandated to take "necessary actions" to "minimize the risk" of users viewing unlawful content on their platforms and to delete it swiftly when they become aware of it.
