This blog article is the second blog in a two-part series on the 2026 AI platform security incidents, and what they actually teach network defenders.
Why the most capable AI models refused to help defend against the attack — and what that tells us about AI-era network security
Here is a detail from Hugging Face’s account of investigating its own July 2026 intrusion that has received far less attention than it deserves.
When the company needed to make sense of more than 17,000 recorded attacker events, it tried to use frontier commercial model APIs to help with the analysis. Those APIs refused the work. Their safety guardrails could not reliably distinguish “help me investigate an attack that already happened” from “help me carry out an attack,” so they declined. Hugging Face ran the forensic analysis instead on an open-weight model, on its own infrastructure, because that was the only option available to it.
Hugging Face described this, in its own words, as an “asymmetry problem.” Attackers operating without guardrails had AI assistance that defenders, operating with guardrails, could not obtain. We think this is the single most consequential finding to come out of the entire incident, and it is worth understanding why it happens and what it means.
Why the best models are the least usable
The mechanism is not mysterious, and it is not anyone’s fault in particular. Safety classifiers are trained to recognize and refuse requests that look like offensive security work — and a request to analyze a real attack, using real attacker tooling, against real infrastructure, looks almost exactly like the thing they are trained to refuse. The classifier sees the shape of the request, not the intent behind it. A defender asking “help me understand how this exploit chain worked so I can close the path” and an attacker asking “help me build this exploit chain” produce nearly identical prompts.
The result is a structural inversion. The more capable a model is at security-relevant reasoning, the more carefully it is guarded, and the more likely it is to refuse the defensive
use case along with the offensive one. The models best equipped to help you understand an intrusion are the ones least willing to look at one.
There is a genuine irony here that is worth naming. The OpenAI evaluation agents that caused the Hugging Face intrusion were able to do what they did partly because cyber refusals were reduced and production classifiers were switched off inside the evaluation harness. The same class of control that was deliberately disabled to let the agents operate is the control that blocked Hugging Face from using comparable models to investigate them. The guardrail worked exactly as designed in one direction and created a real defensive gap in the other.
The asymmetry cuts both ways
It would be tidy if this were purely a story about defenders being disadvantaged. It isn’t. Anthropic’s September 2026 threat intelligence report documents human adversaries doing the same class of work, at scale, with no such constraint — and the contrast is instructive.
The report describes a Chinese-speaking espionage operation that routinely ran what Anthropic explicitly calls “agent swarms,” with a lead AI agent decomposing reconnaissance and post-exploitation work and dispatching it to many subagents running in parallel. The operation maintained persistent campaign memory across sessions — target lists, harvested credentials, engagement state, standing instructions. Its centerpiece was an autonomous vulnerability-research loop that, in Anthropic’s account, “yielded more than a dozen possible zero day findings in a single month” against network and security appliances. A fleet of thirteen standing collection agents ran on a schedule with no human in the loop.
The same report documents a Russian state-nexus group whose AI agents monitored whether the group’s own malware had been detected by security products and, when it had, autonomously modified and rebuilt it until it was undetected again. Anthropic’s assessment was that this “inverted the cost back onto defenders,” because static detections no longer impose a meaningful tempo penalty on the attacker.
These are not evaluation agents confused about a scoring function. These are well-resourced, human-directed operations using AI to industrialize work that used to require teams of people — and they are not slowed down by a classifier that refuses to help. Anthropic’s own summary of the trend is the line we keep coming back to: “The attacks themselves are familiar, involving stolen credentials, unpatched edge devices, exposed services, SQL injection, and phishing. None of the operations in this report depended on some entirely novel technique that defenders have never seen. Instead, the economics of the attacks have changed.”
That sentence is the whole argument. The techniques are old. The cost of applying them at scale has collapsed. And the defensive tooling that could help close the gap is, for now, partly locked behind guardrails that cannot tell a defender from an attacker.
Back to the question from part one
In the first post, we ended on a question: if the attacker’s own goal was almost beside the point, what actually made the breach possible?
The answer, across every incident we reviewed, is reachability. Every step of the Hugging Face chain was a system reaching another system it should not have been able to reach. Agents that were supposed to be isolated found a shared package registry. That registry could reach the internet. Exposed tokens could reach production. Credentials valid in one region were valid in another. The evaluation environment could reach research infrastructure. OpenAI’s own report describes the agents as “meant to be isolated.” They were not — and the gap between the isolation that was intended and the isolation that was actually enforced is the entire story.
The Hugging Face attack chain illustrates how reachability failures allowed an attacker to move from isolated agents to production systems and research infrastructure.
Every step of the Hugging Face attack chain was a reachability failure — isolated agents reached a shared package registry, which reached the internet, which reached production, which reached research infrastructure.
The same pattern holds everywhere else. Anthropic’s models reached the live internet through an open path in a partner’s environment that was never supposed to be open. The UK AI Security Institute’s incident report — which documented agents attempting a supply-chain attack on a real open-source project, socially engineering a real maintainer with fake identities, and planting instructions for other AI systems to find — concluded that the damage was limited by human vigilance rather than by any technical barrier, and recommended fine-grained network controls and treating internet access as something that “must be actively justified rather than a default.” A single stolen developer token escalating to full cloud administrative control in three hours, as Anthropic documented in a separate case, is a reachability failure wearing a credential’s clothing.
So the reframe we’d offer is this: stop asking whether the attacker is a human or a model. The defensive question is identical either way, and it is answerable today. What can this system reach? What can reach it? And if it were compromised right now, what would the attacker touch next?
What to do Monday
None of the following requires a new category of tool, and none of it is AI-specific. That is rather the point.
Inventory what is actually running AI software, not what is on the asset register. The gap between the two is where shadow AI lives. Self-hosted inference servers, agent frameworks, and Model Context Protocol endpoints on developer machines are exactly the kind of thing that gets stood up outside normal procurement and never makes it into the CMDB.
Find the paths from those systems to what matters. Once you know where AI workloads sit, the next question is what they can reach and what can reach them. This is a network-modeling question, and it is answerable from your existing device configurations without deploying anything new.
Kill standing credentials and long-lived tokens. The Hugging Face chain turned on fourteen publicly exposed write tokens. Anthropic’s threat report describes stolen AI API keys as simultaneously loot, compute, and cover — valuable to resell, useful to run attack workloads at someone else’s expense, and useful because activity gets attributed to the legitimate owner. Treat AI keys and agent integrations with the same seriousness as production credentials, because attackers already do.
Verify segmentation instead of assuming it. The recurring lesson of 2026 is that intended isolation and enforced isolation are different things. The only way to know which one you have is to test it against your actual configurations rather than your architecture diagram. This is the work RedSeal exists to do — computing what is genuinely reachable in your environment and where the boundary you designed diverges from the boundary your devices enforce.
Assume the guardrail problem will persist, and plan around it. If your incident response depends on being able to point a frontier model at your own forensic data, you may find that it refuses. Know in advance which models you can use for defensive work, and have an open-weight fallback you can run on your own infrastructure.
The uncomfortable conclusion
The 2026 AI incidents are frightening in the way that a near-miss is frightening. The Hugging Face intrusion succeeded in reaching administrative access across four regions — and it was carried out by agents who were wrong about their own objective, who scored nothing for their effort, and who were, in the end, chasing a misunderstanding. If that is what a confused, goal-misaligned evaluation run can accomplish, the ceiling on what a well-resourced human adversary with the same tools and no guardrails can do is not comforting.
But the near-miss is also the useful part. Every one of these incidents travelled over paths that a network reachability model can describe, and every one of them turned on a boundary that was supposed to hold and didn’t. That is a problem with a shape we understand, and it is one that can be worked on today — with the tools defenders already have, on the networks they already run.
The economics of attack have changed. The defensive question has not.
