UK Children’s Commissioner Urges Ban on AI Nudification Apps

The UK Children’s Commissioner, Dame Rachel de Souza, has called on the government to ban AI-powered “nudification” apps. Artificial intelligence has been generating sexually explicit images by exploiting the original photos. The tools can digitally undress individuals, often target minors, and present severe risks to the childeren’s safety in the digital world.

A newly published report by the Children’s Commissioner, titled “Children Nudification Tools and Sexually Explicit Deepfakes,” has highlighted growing concerns over the availability of deepfake-generated apps, which are often found through online channels like social media and worldwide used search engines. However, children, particularly girls, are reportedly changing their behavior out of fear of being targeted digitally. Rachel has warned that the technology is being used to harass, shame, and exploit children, which can ultimately cause long-term psychological consequences.

These apps serve no constructive purpose and have no place in our society. Said Dame Rachel.

She emphasizes the stringent need for swift legislative action. The report recommends the establishment of new legal obligations for developers of artificial intelligence. Along with the development of frameworks to identify and remove explicit deepfake content online effectively.

The report also recommends that deepfake sexual abuse be considered a form of violence against women and girls. Paul Whiteman of the school leaders’ union, NAHT, supported the call. He noted the threat that such tools pose to both students and staff. The union will discuss this issue at its next conference.

Rachel De Souza also highlighted recent updates to the Online Safety Act and new laws against child sexual abuse content, which is primarily generated with the help of artificial intelligence.

AI nudification apps have enabled the non-consensual creation of explicit images of children and women. Dame Rachel de Souza’s call for a ban highlights the loopholes in regulation, which require immediate attention. Enforcing such a ban could restrain online exploitation. It also deters the use of AI for abusing minors and better protects vulnerable users.

Enforcing any ban on explicit AI-generated content requires robust deepfake detection tools for taking effective measures in a timely manner. Without them, the harmful images may continue to circulate unchecked, which limits the impact of the legal measures. 

Deepfake detection technology, such as FACIA, plays a pivotal role in identifying manipulated images in real-time. It helps platforms remove harmful content before it spreads. The AI-driven solutions offer the proactive defense that today’s online world urgently needs.