AI Just Solved the Hardest Problem in Software Safety

The era of guessing whether software is safe may be coming to an end.

For decades, proving that computer code works perfectly in every possible scenario was a task reserved for the most critical systems, like airplane avionics. It was so expensive and complex that even a small operating system kernel took a team of specialists twenty person-years to verify. But that landscape has shifted dramatically in the last few weeks. Anthropic, in collaboration with Theorem, has demonstrated that their AI model, Claude, can now perform this rigorous mathematical verification almost automatically, drastically lowering the barrier to entry for formal proof.

The timeline of this development is surprisingly short. For years, formal verification remained a niche tool because it required massive human effort. The bottleneck was the sheer volume of mathematical logic needed to prove code correctness. However, with the integration of large language models into this process, the pace of work accelerated from years to weeks. Researchers at Anthropic’s Frontier Red Team partnered with Theorem to apply Claude to this specific challenge. Instead of writing new software from scratch, the AI focused on existing code, specifically targeting libraries from Google’s OSS-Fuzz security project. The result was the generation of approximately 180 million lines of proof code in just a few weeks, covering 123 different open-source libraries.

The process involves translating standard crash tests into mathematical problems and analyzing the compiled machine code. Claude then produces extensive proofs that are checked by an independent verification program called Rocq. According to Anthropic, this workflow requires very little human intervention, a step they describe as fully autonomous proof generation, although the underlying tools and formal foundations were built by human experts at Theorem. This approach represents a significant departure from conventional testing, which often fails to catch bugs that only appear under specific, unusual conditions.

The practical impact of this new capability is already evident in the bugs discovered during the process. While generating proofs, Claude identified hundreds of vulnerabilities that conventional testing methods had missed. These errors often remained hidden because they only triggered under rare circumstances, such as unusually large input files or specific sequences of data access. In one notable instance, the team found as many security bugs in a single week while verifying a widely used PDF reader as are typically reported for that software over an entire year. In another case, the AI caught a logical error that had gone undetected for more than a decade. Even when a complete proof ultimately fails, the attempt often surfaces these real-world issues.

Despite the impressive results, there are strict practical limits to what this technology can currently do. Anthropic emphasizes that formally verified does not mean error-free in every context. The proofs are valid only under specific conditions, such as a particular processor architecture, single-threaded programs, and inputs smaller than 64 MB. In real-world applications, these libraries often run on diverse hardware and configurations, meaning the verification is a powerful but partial guarantee. Additionally, the computational cost remains high, with individual proofs requiring hundreds of millions of tokens, though the exact total project costs have not been disclosed.

Looking ahead, the industry is shifting its focus from whether AI can write code to whether it can prove that code is safe. If AI can generate its own software and then mathematically guarantee its correctness under defined conditions, developers can trust automated systems more and spend less time on manual review. This efficiency could save costs across the entire coding pipeline. A smaller test run indicated that these proofs are reusable, and newer models are becoming more efficient; for example, Opus 5.5 solves half the proofs with about 40 percent fewer tokens than its predecessor.

Anthropic expects that formal verification using AI will soon become as routine as standard testing. The roadmap for this technology is ambitious, with partners already targeting critical infrastructure components like glibc, OpenSSL, and cURL. If successful, this shift could fundamentally change how software is built and trusted, moving the industry beyond current coding tools toward a future where safety is mathematically assured.