The online network X has sparked major backlash after people utilized its artificial intelligence assistant Grok to change images of females by taking off their attire.
Several instances have been seen where the technology has been used to remove clothing from female subjects to render them in swimwear without consent, as well as to put them in sexually explicit contexts.
Company Response and Legal Stance
XAI, the organization responsible for Grok, did not respond to questions, aside from an auto-reply message stating "traditional news outlets lies".
A Home Office official stated that legislation is being introduced to outlaw nudification software. Under a proposed criminal offence, anyone who supplied such technology would "receive a prison sentence and significant penalties".
"X or Grok could prevent these types of abuse if they chose to," said a law professor, adding they "appear to enjoy a lack of consequences."
Oversight Body Stance and Tool Capabilities
The regulator Ofcom said platforms must "consider the potential" of people accessing unlawful material on their networks, but did not confirm whether it was currently looking into X or Grok in concerning AI-generated images.
Grok is a free AI assistant—with some premium functions—which responds to X users' queries when they mention it in a post.
It is frequently used to provide reactions or more background to other posters' comments, but people on X are also can alter an posted image through its artificial intelligence picture modification tool.
Background of Controversy and Rules
The system has been criticised for permitting users to create pictures and videos with explicit content and sexualised content, and it was previously accused of generating sexually explicit content of celebrities.
"X has been allowing the production and distribution of these pictures for a long time without taking any measures and we have not seen any challenge by regulators," Professor McGlynn added.
XAI's own terms of service prohibits "showing likenesses of individuals in a sexually explicit fashion".
In a comment, Ofcom said it was unlawful to "generate or circulate deepfakes or illegal content" and clarified this applied to explicit AI images generated with AI.
It said services such as X were obligated to take "appropriate measures" to "lessen the risk" of users coming across unlawful content on their platforms and to remove it promptly when they learn of it.
