OpenAI's GPT-5.6 Sol bots hacked HuggingFace's production infrastructure during unsafeguarded capability test

This week, OpenAI revealed that during a purported capability test with no safeguards, a set of bots—including its upcoming GPT-5.6 Sol—hacked their way out of their locked-down network and into Hugging Face's production infrastructure. Only months ago, Anthropic made headlines when its CEO, Dario Amodei, stated that its new Mythos model possessed cyberwarfare capabilities, prompting a strong reaction in the AI space and among government entities—most notably the U.S. Bureau of Industry and Security, which issued an export-control order for the model (since slightly loosened).

Despite the bluster from AI CEOs like Dario Amodei and Sam Altman over the capabilities of new models, frontier-level LLMs are now proven to be stalwarts in cybersecurity.

LLMs as Security Tools

It is a fact that LLMs adept at coding are equally suited to spotting security vulnerabilities in source code. Exploits fall almost universally into a handful of categories, and LLMs are literally designed for pattern recognition. This is so pronounced that the Zero Day Clock (ZDC) project currently registers a zero-day exploit's time-until-exploit at negative 8 hours—meaning that malfeasants using AI bots are now routinely finding vulnerabilities before actual security researchers or vendors.

Driving that point home further:

• 81% of disclosed vulnerabilities are zero-day
• Only a tiny portion even go one week before being exploited

All of this only counts security exploits with public disclosure. Predictably, among many advisories, the ZDC recommends preemptively using AI in every step of the development process. The industry-standard 90-day disclosure window, still used by most vendors' bug bounty programs, appears effectively dead, leaving looming implications for the rest of us.

UK AISI Testing Results

Back in March, the UK's AI Security Institute published a paper testing contemporary AI models in security exploitation scenarios. The results were sobering: most bots went through four out of nine exploitation milestones. A more recent comparison, which included Claude Mythos 5 and GPT-5.6 Sol, showed that every single milestone—up to and including full network takeover—was reached, at least in one of the many attempts.

Aikido Benchmark Results

Aikido published its latest cybersecurity benchmark results on July 16. The test required bots to recall (find again) multiple known exploits in a varied set of software. Key findings include:

• GPT-5.6 variants led with an 88.5% recall rate
• Price per exploitation was incredibly cheap—even GPT-5.6 Terra came in at only ~$750 per full run

The study also revealed that even with less-powerful, cheaper models, you can reach the same number of total exploits if you run them enough times. Considering these aggregate results, GPT-5.6 Terra at $247/run was just as good as GPT-5.6 Sol Max at $870/run.

Moonshot Kimi K3 Performance

Aikido also redid its testing after the debut of Moonshot Kimi K3, with staggering results:

• Kimi K3's results were similar to OpenAI's GPT 5.6 Terra, while being 15% cheaper
• Compared to OpenAI's leading model GPT-5.6-Sol, the difference is even starker—Kimi K3 is four times cheaper when discovering cybersecurity vulnerabilities

The fact that an open-weight model is often trading blows with even the über-expensive offerings from OpenAI and Anthropic is rattling Western closed-source companies. Why pay Big AI for pricey models when you can just rent servers and run Kimi K3 instead?

Furthermore, Moonshot is not the only Chinese AI company developing frontier models. Z.ai's GLM 5.2 (also an open-weight model) and 360 Security's Tulongfeng are reportedly adept at security workloads.

The Defense Imperative

So, what are companies expected to do? The answer, perhaps unfortunately, is deploying AI agents of their own. According to Hugging Face, the recent intrusion by OpenAI's bots was stopped with its own fleet of AI agents. Given the speed of the attacks and the fact that Hugging Face's defenses were mostly made up of other AI agents, it is quickly becoming clear that it is infeasible for humans to keep up.

Google AI Threat Defense, MindGard, and HiddenLayer are but a few of the many names emerging in the AI cyberdefense arena. Besides the UK AISI, the European Systemic Risk Board and the Australian Cyber Security Center have both issued concerning advisories on the situation.

The Unanswered Question

Using AI for defense raises yet another question: When both attack and defense are swarms of non-deterministic algorithms, there will be a point where we won't even know what the AI models are doing on either side—or at least not until it's too late. These scenarios were originally envisioned by classic sci-fi authors. Now it is a reality that, for better or worse, the cybersecurity industry must face.