What Happens When the World is Run on Code No One Understands?
· Time
The International Congress of Mathematicians met in Philadelphia last month, its first U.S. meeting since 1986. Minutes after accepting the Fields Medal, mathematics’s highest honor, Jacob Tsimerman told reporters he was leaving academia to work on AI safety. “I think the world is changing,” he said, adding that he did not expect the mathematical career to survive in its present form.
Two months earlier, he had been one of nine mathematicians asked to review whether an AI model had truly refuted an 80-year-old central pillar of modern geometry, known as the Erdős unit distance conjecture. Tsimerman and his peers determined that it had, and their commentary alongside the 18-page proof helped translate the machine's argument into mathematics a human could understand.
Visit rouesnews.click for more information.
The breakthrough, and subsequent report, reflected how the bottleneck that holds us back from forging new discoveries in mathematics, and, increasingly, in every other field, is changing. For most of history, the scarce resource was discovery. With AI, discovery is nonstop, and human confirmation is now what is scarce.
To be clear, we are AI optimists. We believe AI tools will complement human ingenuity and expand what we can know and build, but our infrastructure for vetting and certifying discoveries was built for human throughput, and that is now the binding constraint. That is what holds innovation back. The answer to this problem is formalization: translating AI’s outputs into precise forms whose correctness can be checked automatically. Building the infrastructure to make verification routine is now a national-scale engineering problem.
Machines are outrunning us in more than math. The same is happening to the code that runs hospitals, banks, and power grids. In April, for example, Anthropic disclosed that its Mythos model could find unknown vulnerabilities in major operating systems and browsers, and restricted access to fifty organizations racing to patch them. Soon after, Microsoft engineers found 90 critical flaws in a widely used product; and in June, Sen. Mark Warner told a Senate hearing, citing the NSA director, that the tool "broke into almost all of our classified systems, not in weeks but in hours."
All of this arrives in a world that already runs on code we assume is correct. In July 2024, a single faulty software update, not a cybersecurity attack, grounded flights and disrupted hospitals worldwide. Nowadays, developers can “vibe code,” or use generative AI to write code with natural language prompts. This lets teams build quickly, but at the same time can produce vast quantities of code nobody fully understands.
The same AI systems accelerating discovery can also solve the resulting information overload — if we pair their generative power with the rigor of mathematics.
For decades, researchers in defense, intelligence, academia, and industry have built an alternative to blind trust: formal methods, systems for proving that software mathematically will do exactly what it was designed to do. Like the way a software compiler translates a program into instructions a computer can execute, formal verification takes code, a specification of what it should do, and a proof connecting the two, and accepts the program only if every logical step checks out. This makes it possible to automatically validate AI output, and more broadly rigorize software and other security guarantees. And generative AI tools have made this sort of formalization broadly accessible in a way it never was before.
Once proof checking is cheap, the next bottleneck becomes writing the right specification. Deciding precisely what a system should do and capturing the world around it remains a matter of human judgment. A flawless proof of the wrong specification is still an answer to the wrong question. Formal verification cannot stop phishing, prevent every failure in a complex human system, or certify an AI model's overall safety—but it can eliminate whole classes of errors in critical software. For systems where failure can mean harm to millions, this should be a crucial step in the adoption of AI-generated code.
The Leiden Declaration, endorsed in June by the International Mathematical Union, warns that AI threatens the verifiability of proof. We would put it the other way around: mathematics offers the infrastructure to make generative AI more trustworthy. For this reason, the United States should treat that mathematical rigor as a national mission. We need public infrastructure for verified software: open libraries of verified components and specifications, standards and benchmarks, better tools for checking updates, and training programs that connect mathematics, computer science, engineering, and national security—coordinated across agencies rather than confined to one program. And because formal methods rest on deep mathematical foundations, this requires sustained investment in mathematics research and education, not only in the applications built on top.
Companies operating critical infrastructure should not rely solely on reactive patching and bug bounties where failure could hurt the public. Instead, they should aim to build in auditing and formal guarantees from the start. The goal is not to formally verify every line of every program, but to identify the logic of critical systems, specify what that logic must and must not do, and achieve mathematical assurance in that logic.
Mathematics has weathered crises of trust before. Geometry, infinity, logic, and the nature of proof each threw the field into foundational doubt, and each time the response was more precise languages, stricter standards, and better verification. That rigor is what made mathematics scalable, and software now needs the same transformation. As AI writes more of the code the world runs on, blind trust is no longer good enough. The question is whether the United States will lead the shift to mathematical trust, or wait for the failures that force it.