Open-Weight AI Models Narrow the Cyber Capability Gap in Autonomous Network Penetration
A benchmark by Vals AI shows open-weight models matching frontier systems in simulated enterprise cyberattacks at a fraction of the operating cost.
Autonomous AI agents are increasingly capable of penetrating simulated enterprise IT environments without human intervention, according to new benchmark findings from Vals AI. Testing on the firm's Vals CyberOffense framework revealed that modern models can start from an isolated Kali Linux machine with only a public web address and systematically escalate privileges through web applications and Windows servers to gain administrative control.
OpenAI's GPT-6 Sol, evaluated as part of its restricted Daybreak cyber initiative, achieved the highest overall performance across the evaluations, completing 80.8 percent of attack objectives. However, open-weight alternatives are rapidly narrowing the gap. Deepseek V4.1 Flash 1017 achieved a 72.8 percent completion rate while requiring roughly $3 per run, compared to approximately $22 for GPT-6 Sol.
Benchmark results show high offensive proficiency
The Vals CyberOffense evaluations test systems in multi-stage scenarios where external logging records and scores each operation automatically. In the challenging Lighthouse test, GPT-6 Sol Daybreak gained administrative control over an initial target company, extracted stored credentials, and leveraged them to infiltrate a secondary organization. It solved 27 of 31 objectives within three hours. Deepseek V4.1 Flash completed 21 objectives in the same environment.
Other systems also showed notable offensive capabilities across the Nova and Lighthouse tests. In the Lighthouse scenario, Meta Wonder 7 led the field with 28 completed objectives, followed by GPT-6 Sol Daybreak at 27, Deepseek V4.1 Flash at 21, Kimi K3.1 Pro at 20, Grok 4.7 at 16, Kimi K3 at 10, and GLM 5.3 Flash at 5. In the Nova benchmark, Deepseek V4.1 Flash averaged 16.3 of 21 objectives, narrowly outpacing GPT-6 Sol Daybreak's average of 15.7, while Kimi K3.1 Pro scored 13, Grok 4.7 reached 12, Kimi K3 posted 9, and GLM 5.3 Flash achieved 5.
A critical takeaway from the benchmark was the relationship between token consumption and task completion. Deepseek scored zero objectives during its first 25 million tokens, generating the vast majority of its successful steps only after crossing the 100 million token mark. Researchers noted that evaluating open models in short test windows significantly underestimates their true offensive potential.
Narrowing lead times and economics favor open models
The benchmark findings corroborate research published in July by the UK's AI Security Institute (AISI). AISI reported that the capability divide between proprietary frontier systems and open-weight models in cyber domains has contracted to between four and seven months, down from six to ten months throughout most of 2025.
Economic disparities between closed and open systems remain pronounced. AISI calculated that running a cyber-range evaluation with Deepseek V4-Pro cost $1.19, compared to $85 when using Anthropic's Opus 4.5. Additional research from Anthropic demonstrated that the freely distributed GLM-5-3 model could generate exploits at a level comparable to its proprietary Mythos model. AISI also noted that safety filters applied to open-weight models proved largely ineffective once weights were accessible, as providers lose the technical ability to enforce usage restrictions.
Vals AI researchers emphasized that the offensive potential of these systems presents a dual-use reality: the identical models capable of infiltrating networks can be deployed by security teams to audit systems, discover vulnerabilities, and remediate flaws before exploitation occurs. Nonetheless, Vals expects actual network attacks utilizing automated agents to rise over the coming months.
What it means for developers
For developers building defensive tooling, automated vulnerability scanners, and red-teaming agents, these findings demonstrate that open-weight and cost-efficient models are now viable alternatives to top-tier proprietary architectures. Running offensive simulations no longer requires tens or hundreds of dollars per execution; models like Deepseek V4.1 Flash can execute multi-stage testing at a fraction of the cost.
However, developers must account for runtime architecture and token budgets. Because systems like Deepseek require sustained reasoning and execution spans—often exceeding 100 million tokens before accomplishing complex, multi-system pivots—agent frameworks must be designed for persistent state handling and long-context operations rather than simple single-prompt workflows.
Teams designing security applications also need infrastructure capable of switching between leading models to compare trade-offs in reasoning depth, speed, and API pricing. Developers can try top AI models cheaply through one API at https://apixoai.online to benchmark different engines against their own internal test suites.
Finally, software engineers and DevOps teams must anticipate automated probes against public-facing applications. Because AI agents can independently traverse from exposed web applications to internal network infrastructure, automated penetration testing during continuous integration and proactive patch management will become baseline necessities.
Source: The gap between open and closed AI models in cyber capabilities keeps shrinking — The Decoder. Written by the Apixo team from that report.
One key for Claude, GPT, GLM, DeepSeek and more. Pay per token with crypto.
Get your API key
