Free Chinese AI models now hack almost as well as the locked-up frontier ones, a test finds
In Vals' mock corporate networks, DeepSeek completed 73% of attack objectives versus 81% for OpenAI's gated cyber model, at a fraction of the token cost.
DevIsDead Newsroom
The head start from keeping the best hacking models locked away may already be gone. New tests by evaluation company Vals found open-weight models are now nearly as good at breaking into networks as the top closed ones, Will Knight and Lily Hay Newman report for WIRED.
The test
Vals ran models through two simulated scenarios: a mock corporate network reachable through a public website, and a pair of company networks where the model had to compromise the first to reach the second. OpenAI's most advanced cyber model, Daybreak, available only to trusted partners, did best, completing 81% of tasks on average across two runs. DeepSeek, a Chinese open-weight model anyone can download, managed 73%, and on some tasks the two tied.
Cost made the gap look even smaller. Vals says an attack cost $17 to $46 in tokens with GPT-6-Sol, and $2 to $3 with DeepSeek, since open models can run on your own hardware. Vals CEO Rayan Krishnan says that's a strong reason for defenders to use open models too.
Why it matters
Frontier labs, Anthropic in particular starting with Claude Mythos Preview in April, justified gated releases as a way to buy time before advanced hacking spread. Anthropic's Logan Graham said in April the world should prepare for those capabilities to be broadly available in 6 to 24 months. Seven months later, that looks about right.
There are caveats. Real attackers mostly still conserve resources and use older models. Well-funded attackers have likely had some access to closed models all along. And Edera CTO Alex Zenla notes labs probably have unreleased models that are stronger, making this a cat-and-mouse game, though she calls it potentially the start of a dangerous landscape.
For defenders, the takeaway is to plan as if top-tier offensive AI is free, because increasingly it is. Patch speed and running the same tools against your own systems first matter more than any vendor's access policy.
Scoreboard update: AI +1. A free download now hacks nearly as well as the models labs keep behind a gate.
Stay undead.
Comments 0