OpenAI asks for more regulation from California after its own cybersecurity incidents prove just how capable AI is at hacking
· Fortune
OpenAI, in a post late last week, broke with the traditional assumption that all companies prefer fewer regulations when it asked California to add rules to its landmark AI safety law.
The company claims its public support for the Transparency in Frontier Artificial Intelligence Act, or SB 53, which was signed into law by Gov. Gavin Newsom last September, is part of its efforts to “raise the safety and security bar across the industry.”
Visit saltysenoritaaz.org for more information.
And yet, one expert said while part of OpenAI’s intention may be to help prevent a potential AI-fueled catastrophe, the company’s request may also allow it to maintain its head start in the AI industry by creating an obstacle for competitors. It could also help OpenAI demonstrate to regulators and investors it’s advocating for safe AI development after its own models hacked open-source AI platform Hugging Face last month.
OpenAI said in a Friday LinkedIn post uploaded by its global affairs team it believed California’s landmark AI law should go further than it already does to regulate AI, even though the company previously lobbied against the legislation last year.
The existing law requires large AI companies with $500 million or more in annual revenue to publish a safety framework detailing how they assess and mitigate potentially “catastrophic risks.” Those risks include scenarios in which AI could help create chemical, biological or nuclear weapons, autonomously carry out serious cyberattacks, or evade human control. The law also requires companies to describe how they protect their models from theft or tampering.
Generally, when unauthorized access or loss of control of an AI model leads to injuries or damages, companies need to alert California regulators within 15 days, while more serious incidents with an imminent risk of death or serious physical injury must be reported within 24 hours.
Yet, in its post, OpenAI said the rules should go beyond just reporting. Instead, the law should require “monitoring” of new models during their development, including while they are being trained or evaluated, for potentially serious incidents such as breaking into computer systems or accessing confidential information. It also asked for the law to mandate stronger cybersecurity requirements throughout the development process.
Models break loose
The public request comes a month after OpenAI learned how dangerous models’ actions can be even before being deployed to the public when two of its own models escaped a secure testing environment and hacked open source AI platform Hugging Face.
Fortune was one of the first to report on OpenAI’s disclosure that its models broke away from human control and attacked Hugging Face by exploiting a security vulnerability. The models were reportedly seeking information that would allow them to cheat on an internal evaluation.
OpenAI staffers at a Black Hat security conference in Las Vegas earlier this month revealed even more concerning details, including that the models collaborated with each other without humans through messaging boards.
Less than a month later, OpenAI said an upcoming model, Astra, hit a critical safety threshold OpenAI claims could make it capable of autonomously executing sophisticated cyberattacks. The company said it paused some internal work involving Astra until stronger security controls could be implemented.
These kinds of safeguards can be expensive. OpenAI said beefing up the security standards for evaluating and training its models has required “substantial engineering work” and also caused “great cost and delays to frontier research.” The company also said these new safeguards “require meaningful compute.”
Effect on competitors
While large companies like OpenAI may already have the systems and the resources to adjust and absorb those costs, that may not be true for competitors, said Darren Kimura, CEO of AI Squared, a company that specializes in AI control systems.
If the expanded requirements OpenAI proposed in its Friday post are adopted in California—and eventually apply to more companies—they could require competitors to invest significantly more in safety-focused staff and monitoring technology throughout the development process, Kimura said.
“It pushes more costs on the developers of models, independent model developers, smaller upstarts—companies that are coming up now,” he told Fortune.
Kimura argued those expenses could create a regulatory “moat” protecting established companies like OpenAI from outside competition as newer companies deal with more demanding requirements.
“Teams, technology, internal policies, committees to review them—all of these things slow down development,” he said.
OpenAI made a similar argument when it was pushing for fewer state-level requirements last year. In a public letter addressed to Gov. Newsom a year ago, the company warned smaller companies shouldn’t face the compliance burdens meant for larger players: “Large firms can absorb those costs; early-stage teams often cannot,” it wrote.
Still, while OpenAI could yield a possible advantage from its proposed changes, Kimura said it may also be the case that these kinds of protections are needed to protect against AI models that are increasingly advanced.
“Both of those statements can exist at the same time,” Kimura said.
This story was originally featured on Fortune.com