Anthropic wants you to know that artificial intelligence poses an existential threat to humanity.
It also wants you to invest in it.
On Monday, the company behind the Claude AI chatbot filed for an initial public offering (IPO) that could value the company at an astonishing $2 trillion, potentially making it one of the largest stock market debuts in history.
The filing makes for some interesting reading.
In nearly 100 pages devoted to potential risks, Anthropic warns that increasingly powerful AI systems could facilitate devastating cyberattacks, enable the development of biological weapons, undermine human autonomy, or even escape human control altogether.
These are risks that any potential investor should be aware of, especially given Anthropic’s own acknowledgment that a catastrophic AI-related incident could expose the company to liabilities exceeding its insurance coverage. A very real possibility, especially in light of recent lawsuits against Meta, Google, Open AI, and other big tech companies.
But the timing of Anthropic’s IPO is curious, to say the least.
Earlier this month, Politico reported that Anthropic representatives had been quietly circulating draft legislation among congressional offices. The proposed AEGIS Act would establish federal AI safety requirements while preempting certain state regulations and limiting the circumstances under which authorities could intervene to stop the deployment of dangerous AI models.
In other words, just as Anthropic is preparing to ask investors to assume the substantial risks associated with its technology, it is also attempting to influence the laws governing how that technology will be regulated.
This raises questions about Anthropic’s true intentions. Is it truly seeking to put fail-safes in place to prevent rogue agents from causing catastrophic harm, or is it seeking regulatory capture—rules that protect its interests rather than the public interest and favor Anthropic over current or would-be competitors?
Regulatory capture does not always mean corruption or backroom deals. It often happens in plain sight, with companies positioning themselves as responsible corporate citizens, offering their technical expertise to lawmakers. But in so doing, they also have a hand in developing the very standards by which their own conduct will be judged.
Anthropic has built its reputation, in part, on being the responsible AI company. Its leadership has repeatedly warned of the potentially catastrophic consequences of the technology it is developing. Yet Anthropic continues to pursue increasingly autonomous AI systems, in which AI could accelerate its own development beyond human comprehension or ability to control.
The proposed AEGIS Act would require companies to document and report certain dangers but gives little authority to the federal government to take meaningful action. And by preempting state laws, it could eliminate some of the very mechanisms through which companies might otherwise be held accountable.
For a company preparing to enter the public markets at a potentially historic valuation, the benefits of such an arrangement are obvious.
Maybe Anthropic’s concerns about AI safety are sincere. And maybe they really are offering what they truly believe to be the best regulatory framework to ensure the continued safe development of AI tools. But it is fair to ask whether the companies with the greatest financial stake in AI’s continued expansion should be writing the rules about how they will be governed.
