🔗 Share this article

Tech firms and child safety agencies will be granted authority to evaluate whether AI systems can generate child abuse material under new British laws.

Significant Increase in AI-Generated Harmful Content

The declaration came as revelations from a safety watchdog showing that cases of AI-generated CSAM have more than doubled in the last twelve months, rising from 199 in 2024 to 426 in 2025.

Updated Regulatory Structure

Under the changes, the authorities will permit designated AI companies and child protection organizations to examine AI systems – the underlying technology for chatbots and image generators – and ensure they have adequate safeguards to prevent them from creating depictions of child exploitation.

"Ultimately about preventing abuse before it happens," declared Kanishka Narayan, adding: "Specialists, under strict conditions, can now detect the risk in AI systems early."

Tackling Legal Challenges

The amendments have been implemented because it is illegal to produce and own CSAM, meaning that AI developers and others cannot create such images as part of a evaluation process. Previously, officials had to wait until AI-generated CSAM was published online before dealing with it.

This law is aimed at preventing that issue by enabling to stop the production of those images at source.

Legal Structure

The amendments are being introduced by the government as modifications to the criminal justice legislation, which is also establishing a prohibition on owning, creating or sharing AI systems designed to generate child sexual abuse material.

Real-World Impact

This week, the official toured the London base of Childline and heard a mock-up call to counsellors featuring a report of AI-based abuse. The call depicted a adolescent requesting help after being blackmailed using a sexualised AI-generated image of themselves, constructed using AI.

"When I hear about young people experiencing extortion online, it is a cause of intense frustration in me and rightful anger amongst parents," he said.

Concerning Data

A leading online safety organization reported that instances of AI-generated exploitation material – such as online pages that may contain multiple images – had significantly increased so far this year.

Cases of category A material – the gravest form of exploitation – rose from 2,621 images or videos to 3,086.

• Girls were predominantly victimized, making up 94% of prohibited AI images in 2025
• Portrayals of infants to toddlers increased from five in 2024 to 92 in 2025

Sector Reaction

The law change could "represent a vital step to ensure AI tools are safe before they are launched," stated the chief executive of the online safety foundation.

"Artificial intelligence systems have enabled so victims can be victimised repeatedly with just a few clicks, providing offenders the ability to make possibly endless amounts of sophisticated, lifelike exploitative content," she continued. "Content which further commodifies victims' suffering, and makes young people, especially female children, less safe on and off line."

Support Interaction Information

The children's helpline also released details of counselling sessions where AI has been referenced. AI-related risks discussed in the conversations include:

• Using AI to rate weight, physique and appearance
• Chatbots dissuading young people from consulting trusted adults about harm
• Facing harassment online with AI-generated content
• Digital blackmail using AI-manipulated pictures

During April and September this year, Childline delivered 367 counselling sessions where AI, chatbots and associated terms were mentioned, significantly more as many as in the same period last year.

Half of the references of AI in the 2025 sessions were related to mental health and wellness, encompassing utilizing AI assistants for assistance and AI therapeutic apps.