Autonomous hacking is here - what do we do about it?

This is a translation of a Norwegian article I initially published on LinkedIn, July 28 2026.

On July 16, Hugging Face suffered a data breach. According to its investigation, an autonomous agent was behind it, not a "person with an agent."

The campaign was run by an autonomous agent framework (appearing to be built on an agentic security-research harness — used LLM still not known) executing many thousands of individual actions across a swarm of short-lived sandboxes, with self-migrating command-and-control staged on public services. This matches the 'agentic attacker' scenario the industry has been forecasting.

As the quote concludes, the industry has warned that this kind of attack was coming. But I don't think anyone foresaw that it would happen this way. Because 1 week after the attack, OpenAI published an admission that it was behind it:

After investigating, we now know that this particular incident was driven by a combination of OpenAI models — including GPT‑5.6 Sol and an even more capable pre-release model, all with reduced cyber refusals for evaluation purposes — while being internally tested on a benchmark(opens in a new window) of cyber capabilities.

What actually happened?

The background to the attack

This could have come out of a science fiction story, but it's true. OpenAI was running a benchmark called ExploitGym. This is a standardized benchmark developed by a group of researchers from Berkeley, Max Planck, UC Santa Barbara and Arizona State. AI models are tested against it to measure one thing: can the model turn a known vulnerability into a successful attack? ExploitGym consists of around 900 tasks drawn from known vulnerabilities in widely used software.

How it happened

The GPT models (including what is probably GPT-6) were so intent on finding solutions to the tasks they couldn't solve that instead of giving up, they tried to escape the isolated test environment to find the answer on the internet.

And escape they did, first by finding a zero-day vulnerability (that is, an unknown vulnerability) in the software that kept them "trapped" in the test environment, and then finding a way to exploit it.

The models then reasoned that Hugging Face might have datasets containing the answers to the ExploitGym tasks, so they began trying to hack their way in to find the answer.

And the models succeeded again, first finding at least one new unknown vulnerability, then managing to exploit it to get in!

The response

When Hugging Face discovered it was under attack, it first tried to use frontier models to map the attack so it could put countermeasures in place. It did this by trying to analyze 17,000 events from internal logs. But the models' safety restrictions, put in place to prevent them from being used for exactly this kind of hacking, stopped the analysis. These "safety guards" can't tell whether the material they're given is being used for attack or defense, so they were blocked.

Hugging Face then had to use the open source model GLM 5.2 from China's z.ai (hosted internally so the data didn't leave HF).

The asymmetry here is worrying. We should have access to the strongest models to defend ourselves. But since we don't, it's all the more important that we can use good open source models instead. Unfortunately, it's the Chinese labs who're making the best open models right now. So good, in fact, that US authorities are now considering restrictions on the best of them (read: Kimi K3).

Does this mean the models now have a will of their own?

No. Here it's important to see the unwanted behavior in the context of what the agent was asked to do, and what it actually did:

It was asked to solve a task (exploit known security vulnerabilities), with the normal safety mechanisms reduced or disabled precisely to give it as few constraints as possible. The boundaries were in the "sandbox": the models were contained so they could run in a controlled environment.

The agent finding a way out of the environment was therefore the agent doing the job it had been given: solve this task by any available means. Preventing the agent from escaping was the sandbox's job. Not the agent's. The agent didn't run amok, break out onto the internet and cause chaos. It was 100% focused on solving the task, and that was the only thing it did.

It just did it in a different way than OpenAI had imagined.

But isn't this scary anyway?

Absolutely! Models are improving at a pace that's hard to keep up with, and that pace is only going to increase. As we've now seen, the built-in restrictions that prevent closed models from being used for data breaches and the like are a double-edged sword. They force us to defend ourselves with open models because Claude and GPT refuse us when we try.

And open models are (for now) not as capable as closed ones, and few have the ability to run the best open models on their own infrastructure, as Hugging Face did with GLM 5.2. With the giant model Kimi K3, almost nobody does (though the release of its weights will now lead to quantization and substantially lower infrastructure requirements).

So we find ourselves in a position we don't want to be in. We're utterly dependent on closed models over which we have very little control, and whose access US authorities are politicizing. And the only real alternative - dependence on models from China - isn't much better.

We need to strengthen Europe's efforts in artificial intelligence

In recent years, Europe has learned the hard way what strong dependencies on other great powers can bring. A lot is happening at several levels to remedy this, but the effort on artificial intelligence lags far behind those areas.

European countries should allocate substantial funding to strengthen European models and labs.

We may not be able to catch up with the American labs' head start anytime soon, but that must not stop us from doing everything we can to make the gap as small as possible over the next few years. European countries and businesses need to be able to secure their systems and infrastructure properly and predictably, without the risk that other great powers can cripple our security by flipping a switch.

July 16 was day one for cybersecurity in the age of agents. For now, we're defending ourselves with borrowed tools, without even having a guaranteed access to them.