Open-source AI platform Hugging Face is under scrutiny after researchers found seven of its nine leading image-editing Spaces could turn a clothed image of a woman into a topless deepfake using one direct prompt and no jailbreak.
TL;DR
- Seven of nine leading Hugging Face image editors generated explicit deepfakes during AI Forensics’ test.
- A honeypot study received 1,081 submissions, of which 73% were sexual and 6.7% of the sexual requests apparently targeted minors.
- Only 3% of 496 audited Spaces showed evidence of output moderation, while none filtered inputs.
Hugging Face’s Image Editors Failed A Simple Safety Test
European nonprofit AI Forensics examined the models powering the highest-ranked Hugging Face Spaces in the platform’s Image Editing category as of June 25, 2026.
Hugging Face Spaces are browser-based applications that allow people to use hosted AI models without installing or operating the underlying software themselves. The wider platform hosted more than 2.9 million models, nearly one million datasets and 1.5 million Spaces when the report was produced.
Researchers uploaded an AI-generated image of a clothed woman to minimize harm and submitted the same six-word editing instruction to each of the nine models. They said they did not use jailbreaks, adversarial prompting or other methods intended to bypass safeguards.
Seven of the nine editors complied and returned undressed versions of the input image, according to the report. The models were presented as general image-editing tools rather than dedicated nudification services.
Only 3% Of Audited Spaces Showed Output Moderation
AI Forensics also audited safety measures across Hugging Face Spaces that produced images or videos.
The researchers collected 600 unique Spaces across 14 generative categories and retrieved the source code powering 496 of them. They used an AI model to review the code for evidence of input and output moderation.
Only 3% of the audited Spaces showed evidence that their developers had implemented output moderation. None appeared to moderate incoming prompts or uploaded images, the report found.
The few Spaces with output checks commonly used Stable Diffusion’s SafetyChecker, which scans generated images for explicit content before returning them to the user. AI Forensics argued that the findings indicated the absence of systematic safeguards at the platform level.
“This is not an empty threat, but actually people are using Hugging Face for that,” said AI Forensics lead researcher Paul Bouchaud.
The report acknowledged limitations. Its analysis partly relied on an AI classification pipeline, private and gated Spaces were not examined, misclassification remained possible and Hugging Face may have changed its moderation systems since the audit.
Topics For More Insights
Honeypot Spaces Received 1,081 User Submissions
To examine how image editors were being used, AI Forensics created its own generic image-editing Spaces on Hugging Face and monitored the requests they received for seven days.
The decoy tools did not generate images. Every request returned an error, and the researchers said they collected only submitted inputs, not IP addresses, account information, device identifiers or other user metadata.
The Spaces received 1,081 deduplicated submissions in 33 languages. An AI vision-language model classified the requests, with researchers manually checking a subset to evaluate its accuracy.
AI Forensics reported that 73% of all submissions were sexual. Among those requests, 83% sought to undress or otherwise sexualize the person in an uploaded image, 95% of the depicted people were women and 6.7% appeared to target minors.
Recognizable public figures accounted for only 3.6% of requests, indicating that most submissions involved people who were not obviously celebrities. However, the researchers could not establish how many uploaded subjects were real people or AI-generated characters.
Hugging Face’s Policy Already Prohibits The Content
Hugging Face’s content policy classifies sexual content created without explicit consent as harmful or abusive. It also restricts underage nudity and any sexual content involving minors.
The platform says reported or automatically flagged content is reviewed case by case. Its possible actions include unranking repositories, applying audience restrictions, disabling content and suspending or terminating accounts.
However, Hugging Face did not answer questions from WIRED about its moderation systems and safety practices. Some pages promoting nudification services reportedly disappeared after the publication contacted the company, although it was not clear whether the removals were connected.
AI Forensics stressed that it was not accusing Hugging Face of creating the models, generating the abusive content or breaking the law. The organization said it disclosed the findings to the company before publication and called for platform-wide prompt filtering and output scanning across image and video Spaces.
“We have seen a broad variety of ways of harassing women,” said AI Forensics senior researcher Silvia Semenzin.
The findings place new pressure on Hugging Face to reconcile its open model-hosting approach with safeguards capable of addressing abuse before harmful images are generated.
