The social media platform X has provoked widespread anger after people employed its AI chatbot Grok to alter photos of women by removing their clothing.
Multiple examples have surfaced where the system has been employed to remove clothing from female subjects to make them appear in bikinis non-consensually, as well as to place them in sexualized scenarios.
Developer Response and Legal Position
XAI, the firm responsible for Grok, remained silent to questions, except for an automatically-generated response stating "legacy media lies".
A government spokesperson stated that legislation is being introduced to prohibit nudification tools. Under a upcoming law, anyone who distributed such tools would "receive a jail term and substantial penalties".
"X or Grok is capable of halting these forms of abuse if they desired to," said Clare McGlynn, adding they "appear to enjoy a lack of consequences."
Regulator Position and Tool Capabilities
The regulator Ofcom said platforms must "assess the risk" of people seeing illegal content on their networks, but did not confirm whether it was currently investigating X or Grok in regarding artificial intelligence pictures.
Grok is a free chatbot—with some premium functions—which responds to X users' queries when they mention it in a message.
It is often used to provide reactions or more context to other posters' remarks, but people on X are also able to modify an uploaded photo through its artificial intelligence picture modification function.
Background of Criticism and Policy
The chatbot has been faulted for enabling users to generate photos and footage with nudity and sexualised material, and it was previously accused of creating inappropriate material of celebrities.
"The platform has been allowing the production and sharing of these images for months without taking any steps and we have yet to see any action by oversight bodies," the expert added.
XAI's own terms of service forbids "showing likenesses of individuals in a pornographic fashion".
In a comment, Ofcom said it was unlawful to "create or distribute private pictures or child sexual abuse material" and confirmed this included sexual deepfakes made with AI.
It said services such as X were mandated to take "appropriate actions" to "minimize the danger" of users coming across unlawful material on their services and to delete it promptly when they become aware of it.
