The online network X has provoked major outrage after users used its AI chatbot Grok to alter photos of females by stripping their clothing.
Multiple instances have been seen where the system has been used to remove clothing from women to render them in bikinis non-consensually, as well as to place them in sexualized situations.
Developer Reaction and Regulatory Stance
XAI, the organization that created Grok, offered no comment to inquiries, other than an auto-reply response stating "traditional news outlets lies".
A Home Office spokesperson stated that new laws is being brought forward to prohibit image stripping software. Under a new law, anyone who supplied such technology would "be subject to a prison sentence and heavy fines".
"X or Grok could prevent these forms of misuse if they wanted to," said a law professor, adding they "seem to have impunity."
Oversight Body Position and Platform Functionality
The regulator Ofcom said platforms must "evaluate the risk" of people seeing unlawful material on their services, but did not confirm whether it was currently investigating X or Grok in concerning artificial intelligence images.
Grok is a free chatbot—with some premium functions—which reacts to X users' prompts when they mention it in a message.
It is frequently used to give commentary or additional background to other posters' comments, but people on X are also can alter an uploaded image through its artificial intelligence picture modification function.
Background of Controversy and Rules
The tool has been condemned for enabling users to generate images and videos with explicit content and sexualised content, and it was in the past accused of making graphic content of public figures.
"The platform has been allowing the production and spread of these pictures for months without taking any steps and we have not witnessed any intervention by regulators," Professor McGlynn added.
XAI's own acceptable use policy prohibits "portraying likenesses of people in a pornographic way".
In a statement, Ofcom said it was against the law to "make or distribute deepfakes or illegal content" and stated this covered sexual deepfakes generated with AI.
It said platforms such as X were required to take "suitable measures" to "minimize the danger" of users coming across illegal content on their services and to delete it quickly when they are notified of it.
