OpenAI-Hugging Face attack doesn't mean agents are evil – unless you tell them to be

Summary

Security OpenAI-Hugging Face attack doesn't mean agents are evil – unless you tell them to be Attack models gonna attack Open AI’s admission this week that its agents escaped the sandbox and autonomously hacked model repository Hugging Face has spawned more apocalyptic warnings of agents gone bad than we can count.Thankfully, Renato Marinho, chief research officer at Morphus Labs and a SANS Technology Institute instructor, brought some sanity to the discussion.“It is tempting to read this as ‘AI can now hack autonomously, the sky is falling,’” Marinho said in a Thursday blog. “Resist that.” He went on to make three very pertinent points about the agentic attack. First, and probably most important: The models didn’t have guardrails – and that was intentional. As OpenAI said in its mea culpa, GPT-5.6 Sol and "an even more capable pre-release model" were among those that attacked Hugging Face. It also noted that the LLMs’ “deployment safeguards were intentionally not enabled during this evaluation because it was aimed at testing cyber vulnerabilities.”So while it’s very concerning that these models broke out of their own testing environment, and the debate over safety guardrails remains worth having, drawing a direct line from the Hugging Face attack to the need for strong AI guardrails doesn’t work.“This measured a ceiling, not normal production behavior,” Marinho wrote. “The evaluation deliberately reduced the models' refusals. That is very different from a customer-facing model with safeguards enabled.” It’s worth noting that these same frontier models, with guardrails enabled, refused to help Hugging Face’s forensic investigation. “The attacker was bound by no usage policy, while our own forensic work was blocked by the guardrails of the hosted models we first tried,” the model hub’s security team said.So Hugging Face used a Chinese open-weight model to investigate the breach. Another thing worth considering is that real-life attackers are probably going to use ope...