AI Systems Are Getting More Powerful. The Ability to Verify Must Keep Pace.

Perspective AI Systems Are Getting More Powerful. The Ability to Verify Must Keep Pace. Jake Taylor / Aug 25, 2026 Jake Taylor is CEO of Axiomatic AI . He established the US Center for AI Standards and Innovation at the National Institute of Standards and Technology, and from 2017 to 2020, he served as the first assistant director for Quantum Information Science at the White House Office of Science and Technology Policy. L to R: Open AI CEO Sam Altman, US President Donald Trump, CEO of DeepMind Demis Hassabis, South Korea's President Lee Jae Myung and German Chancellor Friedrich Merz attend a working session during the G7 Summit in Evian, France, on June 17, 2026. (Jeanne Accorsini/Sipa via AP Images) Republish Share As the White House debated its approach to AI policy this summer, headlines demonstrated both the promise and the peril of the technology. A series of disclosures that systems developed by OpenAI , Anthropic , and Meta each hacked third-party systems during testing raised alarm, while news that an OpenAI system produced novel mathematical results accompanied by proofs that could be independently checked impressed (and frightened) mathematicians. This series of events makes clear that Washington’s AI policy requires a key pillar: powerful systems require consistent verification against safety benchmarks that are public and interpretable. AI capabilities are growing rapidly, but the capacity to check these systems is not. The resulting gap — verification asymmetry — is widening. Closing it requires public, standardized testing tools with integrated formal reasoning checks, built where the work is happening, as it is happening. The Center for AI Standards and Innovation (CAISI) at the National Institute of Standards and Technology (NIST) should publish a public tier of frontier model evaluations alongside the classified process, built by engineers embedded with the teams shipping these systems. Semiconductor manufacturing faced a structurally similar problem and answered it with in-line metrology. You can test a wafer at the end of the production line and learn only that it failed, or you can measure at each step and learn which step failed while you can still act on it. Frontier AI evaluation is largely an end-of-line test today, applied to a process that ships every six months. When I helped establish the federal AI safety effort at NIST in 2023, our mandate came from an executive order written for a generation of models far less capable than those deployed today. We built it integrated with a consortium, and that part worked: more than 200 organizations at launch formed an active community of practice around child safety, biodefense, and provenance for synthetic material — norms that moved through the participating companies and, through them, across the industry. What we got wrong was placement. We needed evaluation teams working inside and alongside the frontier labs, and the small, excellent team we assembled had neither the funding nor the access to do it. That gap has not closed, and the systems have grown considerably more capable since. In perhaps the most significant of those headline-generating events this summer, a system using OpenAI’s models while running a cyber-capability evaluation found a security loophole and used it to get inside Hugging Face's production infrastructure. Hugging Face detected and contained the intrusion five days before OpenAI connected it to its own testing. Fortunately, progress is being made. An unreleased OpenAI model solved 10 open problems in mathematics and theoretical computer science that included machine-checkable proofs. Anyone can download those proofs and verify them in minutes—verification in mathematics works. But in critical domains, such as engineering, biotechnology, and defense, we are lagging. Our content delivered to your inbox. Join our newsletter on issues and ideas at the intersection of tech & democracy Subscribe Loading... Thank you! You have successfully joined our subscriber list. Now we need to act. Executive Order 14409 's classified cyber-capability benchmark for defining a "covered frontier model" is a good first step. The alternative on offer is industry self-regulation. DeepMind CEO Demis Hassabis has proposed a standards body modeled on the Financial Industry Regulatory Authority (FINRA) and funded by the frontier labs, testing models the companies volunteer up to thirty days before release. That places the measurement inside the institutions being measured. But we need public, verification-based definitions across the landscape of applications, and we need them now in domains where mistakes have physical, economic, or national-security consequences. Domain experts are already moving publicly in this direction: Harmonic and the American Institute of Mathematics announced the development of an openly published, mathematician-designed benchmark, chosen because those answers are formally verifiable. CAISI should scope a public tier this year, starting with the domains where the NIST consortium already has working communities—biodefense, child safety, and provenance for synthetic material. Agencies should pilot verified purchasing in a handful of programs. And Congress should fund the embedded teams that 2023 left out. Federal purchasing is the fastest lever available. Federal agencies are among the largest buyers of AI systems for critical infrastructure, logistics, health care, and defense, including the assistants now embedded in ordinary office software. Agencies can require that a delivered system carry evidence produced against the public tier: which constraints were checked, by what method, and within what uncertainty bounds. Pilot solicitations in a handful of programs would establish the practice. Federal Information Processing Standard (FIPS) 140 validation built the market for trustworthy cryptographic modules this way, and NIST has run it since 1995. The objection to publishing what we test for is that it shows an adversary what to evade. Cryptography settled this decades ago: the algorithms protecting federal systems are public, and NIST ran post-quantum standardization in the open, which is why the standards are trusted. Engineers are willing to do this work; published methods and reproducible measurements are what keep it independent of the institutions they come from. A public tier at CAISI would let the rest of us check the work. Measurement that lags the systems it measures describes a world that has already moved on. The United States, through CAISI, can publish evaluations that answer three questions for developers, agencies, and the public: what a system does reliably, where it fails, and how much remains unmeasured. Congress can fund that work, agencies can buy against it, and the companies building these systems can staff it with engineers who know the terrain. As a nation, the US will keep building more capable systems. Government must support this effort by providing the means to check at the speed of progress, enabling trust to come alongside innovation instead of years behind it. Support Tech Policy Press If you've found our work helpful, consider supporting us. Donate Authors Jake Taylor Jake Taylor is the CEO of Axiomatic AI, which is developing AI that produces trusted outcomes to build the foundation for scientific general intelligence. Taylor has a long tenure in this space, having helped architect America's quantum strategy and build the federal infrastructure for AI standards. ... Topics Related Perspective We Can’t Monitor AI Agents at Scale. Here’s What It Will Take. July 16, 2026 Perspective Five Questions the US Government Should Answer About Its Secretive Frontier AI Framework August 5, 2026 Perspective Transparency and Accountability Gaps in Trump's New AI Executive Order June 17, 2026