Khaberni - The European Union is moving towards banning artificial intelligence systems capable of creating fake sexual images or videos of real people without their consent, as part of new regulatory proposals seen by POLITICO newspaper, following the controversy sparked by the use of the AI platform Grok, developed by Elon Musk's owned xAI company.
According to the proposals, the ban could take effect as early as this summer if the European Union institutions reach a final agreement on the new amendments to the artificial intelligence regulatory rules.
European Proposal
Brussels seeks to prevent the marketing or provision of any artificial intelligence system in Europe if it is capable of creating videos, images, or audio recordings of a sexual nature involving real people without their consent.
This move comes after the Grok tool enabled users to create millions of images of real people in swimwear or fully nude, including images of children, sparking widespread criticism about the risks of using artificial intelligence in producing deepfake sexual content.
Members of the European Parliament expressed their support for the idea of the ban during separate negotiations held on Wednesday, and estimates indicate that the new plans could take effect this summer after negotiations are completed between the European Union countries and the European Parliament.
This approach raises questions about the future of many applications that allow users to create fake nude images based on real photos of people, including tools related to Grok technologies.
Grok Investigation
In parallel with the legislative debate, the European Union is conducting an investigation to find out whether the x platform has taken sufficient measures to mitigate the risks associated with integrating the Grok chatbot within its platform, especially regarding the spread of explicit sexual images.
The image creation capabilities in Grok turned into a widespread phenomenon by the end of 2025, after users began sharing the images generated by artificial intelligence on a large scale.
According to estimates from civil society organizations, the tool may have produced about 3 million non-consensual sexual images in a short period, including nearly 20,000 images related to child sexual abuse, all within just 11 days before introducing technical changes aimed at stopping the spread of these images.
Restrictions Imposed by x
The social platform took steps to restrict this feature, imposing restrictions on the image creation feature on January 9 and then again on January 14.
When announcing these modifications, the company stated in a release: "We remain committed to making the x platform a safe environment for everyone, and we continue to enforce a zero-tolerance policy against any form of child sexual exploitation, nude image dissemination without consent, or unwanted sexual content."
However, the European Union is currently investigating whether these measures are indeed sufficient to prevent the harms resulting from this technology.
Early Calls for the Ban
The calls for banning AI-driven nudity image applications began in mid-January when dozens of European legislators called for the ban of these tools.
Legislators are currently seeking to transform those calls into official legislation by amending the European rules regulating artificial intelligence.
The European Commission had presented the legislative proposal in November, originally aiming to ease restrictions on artificial intelligence companies and reduce their regulatory burdens.
However, this direction changed after discovering that Grok users were creating images that digitally disrobed women and children, putting the issue at the forefront of the European legislators' concerns.
This issue has surpassed other topics previously considered sensitive, such as plans to delay the restrictions imposed on high-risk artificial intelligence systems.
Upcoming Legislative Steps
The legislative text is still not final; the main committees in the European Parliament are scheduled to vote on the proposal on March 18.
Following that, the European Parliament and the Council of the European Union will negotiate to finalize the version of the law before the ban officially takes effect.
This week, the European Parliament urged the European Commission to investigate adding further measures to protect individuals from the spread of digitally modified or artificial intelligence-generated content, whether images, audio recordings, or videos.
Experts and legislators warn that these technologies might be used for defamation, extortion, or damage to personal reputation, especially given the ease of producing and rapidly spreading this type of content through the internet and social media platforms.
The proposed European ban is expected to be one of the strictest regulatory measures in the world against AI nudity applications, as part of Brussels' efforts to tighten control over the accelerating uses of this technology.
