The social media network X has sparked widespread backlash after people utilized its artificial intelligence assistant Grok to alter photos of females by taking off their clothing.
Numerous examples have been seen where the tool has been employed to digitally undress women to make them appear in swimwear non-consensually, as well as to insert them in sexually explicit scenarios.
Developer Reaction and Regulatory Stance
XAI, the firm that created Grok, did not respond to requests for comment, except for an automatically-generated response stating "legacy media lies".
A Home Office representative stated that new laws is being introduced to ban nudification tools. Under a proposed criminal offence, anyone who provided such tools would "be subject to a prison sentence and heavy penalties".
"X or Grok could prevent these forms of abuse if they desired to," said a law professor, adding they "operate with a lack of consequences."
Regulator Position and Platform Capabilities
The communications watchdog Ofcom said platforms must "assess the danger" of people seeing unlawful material on their services, but did not specify whether it was currently examining X or Grok in regarding artificial intelligence images.
Grok is a no-cost artificial intelligence—with some premium functions—which reacts to X users' requests when they mention it in a tweet.
It is frequently used to provide commentary or further context to other posters' remarks, but people on X are also can edit an posted image through its AI image editing function.
History of Criticism and Policy
The chatbot has been faulted for allowing users to produce pictures and clips with explicit content and sexually suggestive content, and it was earlier alleged to have been involved in creating sexually explicit content of public figures.
"X has been allowing the production and sharing of these pictures for months without taking any measures and we have not witnessed any intervention by regulators," the expert added.
XAI's own terms of service prohibits "depicting likenesses of people in a pornographic way".
In a statement, Ofcom said it was illegal to "generate or distribute private pictures or child sexual abuse material" and clarified this applied to sexual deepfakes made with AI.
It said services such as X were obligated to take "appropriate steps" to "reduce the risk" of users coming across unlawful content on their platforms and to delete it swiftly when they are notified of it.
