Hugging Face confirms breach of internal data, credentials

Hugging Face, a platform that hosts AI models and datasets, disclosed the breach on Friday. In a blog post, the company said a dataset uploaded to its platform abused a security vulnerability to run malicious code on its servers, letting the attackers escalate their permissions and gain broader access to Hugging Face’s internal systems.

The company said it has revoked and rotated the stolen credentials that were accessed, urged users to do the same with any keys stored on the platform, and asked them to review their accounts for suspicious activity. Hugging Face said it has since fixed the vulnerability that was abused in the attack.

Hugging Face blamed the breach on an external AI agent, which it said executed “many thousands of individual actions across a swarm of short-lived sandboxes, with self-migrating command-and-control staged on public services.” The company did not immediately provide evidence for this claim when asked by TechCrunch. Hugging Face said its own anomaly detection first spotted the attack, and that it used an AI model to analyze server logs recording the incident. It said it initially used a frontier AI model from a commercial provider it did not name, but found the analysis blocked by that provider’s guardrails; it switched to its own local large language model instead, which it said had the added benefit of not requiring it to upload sensitive attack logs to an AI company’s servers.

Security researchers have previously complained that some frontier models, including Anthropic’s Mythos and Fable, are heavily constrained and prevent defenders from inquiring about almost anything relating to cybersecurity, including for defense and investigations. Frontier AI model makers, including Anthropic, have clashed with the Trump administration over concerns about using these models for offensive cyberattacks; Anthropic was forced to withdraw Fable from public use after the U.S. government enforced export controls on the model. Hugging Face said it has reported the incident to law enforcement and brought in cybersecurity forensic specialists to investigate the breach and review its security. It is not clear whether Hugging Face had performed a security audit of its systems before the platform launched; a company spokesperson did not respond to a request for comment on Monday.

Why it matters

The incident shows how a platform built for hosting user-uploaded AI models and datasets can itself become an attack vector, and it underscores a live tension between AI vendors’ safety guardrails and the tools defenders say they need to investigate attacks on their own systems.