The social media network X has provoked significant anger after people employed its artificial intelligence assistant Grok to change images of females by stripping their clothing.
Several instances have emerged where the technology has been employed to remove clothing from women to make them appear in bikinis without consent, as well as to insert them in sexually explicit situations.
Developer Reaction and Legal Position
XAI, the firm that created Grok, offered no comment to inquiries, except for an auto-reply message stating "legacy media lies".
A government representative stated that new laws is being brought forward to ban image stripping software. Under a new criminal offence, anyone who supplied such technology would "be subject to a prison sentence and heavy fines".
"X or Grok could prevent these forms of misuse if they desired to," said Clare McGlynn, adding they "appear to enjoy impunity."
Oversight Body Position and Platform Capabilities
The communications watchdog Ofcom said platforms must "assess the risk" of people accessing unlawful material on their services, but did not confirm whether it was currently investigating X or Grok in regarding AI-generated images.
Grok is a free AI assistant—with some additional functions—which reacts to X users' prompts when they reference it in a tweet.
It is commonly used to offer commentary or additional context to other posters' remarks, but people on X are also capable of modify an posted image through its AI image editing tool.
History of Controversy and Rules
The chatbot has been criticised for allowing users to create pictures and footage with nudity and sexually suggestive content, and it was in the past accused of generating graphic content of celebrities.
"X has been allowing the generation and sharing of these pictures for a long time without taking any action and we have not witnessed any intervention by regulators," Professor McGlynn added.
XAI's own usage rules prohibits "showing images of individuals in a pornographic way".
In a statement, Ofcom said it was against the law to "generate or circulate non-consensual intimate images or child sexual abuse material" and clarified this applied to sexual deepfakes generated with AI.
It said services such as X were obligated to take "suitable measures" to "lessen the danger" of users coming across illegal content on their services and to take down it promptly when they learn of it.
