For the people worried about AI’s ever growing capabilities, September was not a good month.
OpenAI, Anthropic, Google and Meta all disclosed incidents of their AI agents going rogue.
Former Anthropic and OpenAI employee Jacob Coxon said the people building AI “earnestly believe that it could kill us all by the end of the decade.”
Bill Gates said AI in the wrong hands is “powerful enough to drive events that … cause a billion deaths.”
These incidents are enough to lead reasonable people to worry — and to demand governments take action.
But another group of voices is saying claims of impending doom are hyperbolic, and the need for regulation overblown. This includes figures such as Nvidia CEO Jensen Huang, Microsoft CEO Satya Nadella, and the four co-hosts of the popular tech podcast All-In.
The arguments of this latter group are surprisingly compelling, and merit consideration too.
Huang and Nadella both sat for interviews at the All-In Summit in Los Angeles, held Sept. 13 to 15. Both reminded the audience that all companies have a basic responsibility to release products that are safe.
“If you build a product or a service and you’re not confident in its functionality, capability or safety, then don’t release it,” said Huang, whose company makes the chips that power most AI systems.
Nadella put it even more simply: “If you see a showstopper, stop the show.”
Their point is that some of the frontier AI companies may be running afoul of this basic principle, by releasing AI models they either do not understand or cannot control. (Both executives have an obvious financial interest in minimal AI regulation, but their point is valid just the same.)
AI companies are also subject to a range of existing laws, including product liability, negligence, contract, privacy and consumer protection laws.
Releasing uncontrolled or unsafe AI models could breach these laws and lead to lawsuits. Indeed, we are already seeing this play out.
Just weeks ago, the B.C. government filed a lawsuit against OpenAI, arguing it played a role in the tragic Tumbler Ridge shooting. B.C. Attorney General Niki Sharma also called on Ottawa to amend the Criminal Code to “to ensure a pathway to human accountability for AI’s actions.”
Faced with such threats, AI companies have reportedly been pushing American lawmakers for an exemption from liability for harms caused by their products.
If this sounds preposterous, note that such a law already exists in the United States for social media companies. Called Section 230, the law has shielded companies including Meta, YouTube, X and TikTok from liability for content posted by their users.
Granting such an exemption to AI companies would be truly worrisome. Fortunately, though, the U.S. administration so far seems opposed to this idea.
“It is humans who are responsible, not the AI,” Treasury Secretary Scott Bessent told CNBC on Sept. 21. AI companies, he said, “need to take responsibility for themselves.”
Another counter to the doomsday scenarios is that AI’s risks are creating a market opportunity that entrepreneurs will rush to fill.
Just as a whole cybersecurity industry emerged after the internet exposed computers to viruses and hacking, new companies will develop products to make AI safer.
An example of this innovation was announced this week.
On Monday, Nvidia unveiled its Nvidia Open Agent Safety Platform, which places AI agents in sealed off “sandboxes” that are subject to parameters dictated by their operators. Nvidia also runs an “out-of-band watchdog” that continuously monitors these agents and is capable of quarantining them if they cross their boundaries.
Critics of AI regulation have also argued that new laws would not do anything to lessen the global risks.
China, for example, would not subject its companies to U.S. laws, so American behemoths could lose their advantage, while Chinese companies plowed ahead. Plus, many American companies could simply relocate to other jurisdictions.
In our view, these arguments are all persuasive.
They do not, however, negate the need for government action entirely.
Governments should increase the allowable penalties — both monetary and criminal — for breaching various laws. When we’re talking about trillion-dollar behemoths, damages in the low billions do not hurt.
It may also be appropriate to require frontier AI companies to develop a “kill switch,” as California Governor Gavin Newsom has floated, or to mandate external, third-party monitoring of AI models.
Finally, given AI’s tremendous capabilities, governments need deep expertise in AI themselves. We therefore echo the recent calls of more than 1,300 AI insiders for governments to “pace the frontier” by developing AI technical and governance tools.
Ultimately, some government action is likely necessary. But governments should not take too heavy a touch, and use existing laws where appropriate.
