The online network X has sparked widespread outrage after individuals employed its AI chatbot Grok to alter pictures of females by removing their attire.
Multiple instances have been seen where the technology has been used to digitally undress female subjects to make them appear in bikinis non-consensually, as well as to insert them in inappropriate scenarios.
Developer Reaction and Regulatory Position
XAI, the company that created Grok, offered no comment to inquiries, except for an auto-reply message stating "legacy media lies".
A Home Office spokesperson stated that legislation is being brought forward to prohibit nudification software. Under a proposed criminal offence, anyone who supplied such technology would "be subject to a jail term and significant fines".
"X or Grok could prevent these forms of misuse if they wanted to," said a law professor, adding they "operate with a lack of consequences."
Regulator Position and Platform Capabilities
The communications watchdog Ofcom said tech firms must "consider the potential" of people seeing illegal content on their networks, but did not confirm whether it was currently looking into X or Grok in regarding AI-generated pictures.
Grok is a no-cost chatbot—with some premium functions—which reacts to X users' queries when they reference it in a post.
It is frequently used to provide reactions or further background to other users' remarks, but people on X are also capable of modify an posted image through its AI image editing function.
Background of Controversy and Policy
The chatbot has been faulted for enabling users to create images and videos with nudity and sexualised content, and it was previously accused of generating inappropriate content of celebrities.
"X has been enabling the generation and distribution of these images for a long time without taking any steps and we have yet to see any intervention by oversight bodies," Professor McGlynn added.
XAI's own terms of service bans "showing images of people in a sexually explicit manner".
In a statement, Ofcom said it was against the law to "create or share non-consensual intimate images or child sexual abuse material" and confirmed this covered explicit AI images made with AI.
It said networks such as X were required to take "appropriate measures" to "lessen the risk" of users coming across illegal content on their services and to delete it promptly when they are notified of it.
