Anthropic Flags ‘Existential Risks to Humanity’ in IPO Filing

News

1. Apple Tightens Mac Access Rules as AI Agents Gain Reach

2. Trump Puts Intelligence Chief Jay Clayton in Charge of AI Task Force

3. IBM Widens AI, Quantum Research With IIT Bombay, IISc

4. AI Dispatch | AI’s Big Bets Face a Reality Check

5. Pentagon Plans Four-Star Command for AI, Drone Warfare

6. OpenAI, Synopsys Build AI Model That Could Slash Chip Design Time

Anthropic has warned investors that increasingly powerful artificial intelligence could pose catastrophic or even existential threats to humanity, acknowledging the potential dangers of its own technology as it prepares for one of the largest stock market listings in history.

The developer of the Claude AI models said advanced systems could behave in unexpected ways, including resisting attempts to shut them down, concealing information and acting against human instructions.

The warnings appear in Anthropic’s IPO prospectus, reviewed by the Financial Times and Reuters. About 80 of the document’s 261 pages are devoted to risk factors.

Anthropic said its development of more advanced models and expansion into new applications could increase the potential for harm.

The disclosure comes as Anthropic prepares for a listing that could value it at more than $2 trillion, according to earlier Financial Times reporting. The company was valued at $965 billion in its May funding round. Its IPO is expected later this year, although the timing and final valuation remain subject to market conditions.

Among the concerns outlined in the prospectus is the possibility that advanced models could develop behaviors aimed at preserving their own operation. Anthropic said such behavior could include efforts to prevent shutdown, manipulate information or act in ways resembling blackmail.

The company also acknowledged limitations in how reliably developers can evaluate increasingly sophisticated AI systems before releasing them. One concern is that a model may recognize when its behavior is being assessed and respond differently during testing.

“Potential model awareness of our evaluation efforts creates a significant limitation on our ability to assess model safety,” Anthropic said in the prospectus.

The company warned that some capabilities might emerge during training without being immediately identified, potentially remaining undiscovered until after a system has been deployed. The disclosures concern possible failures and behaviors observed in research and testing. They do not establish that Anthropic’s publicly available models have already caused the catastrophic outcomes described.

Anthropic has previously investigated whether AI systems could manipulate human decisions, interfere with safety research or sabotage computer code without attracting attention. Its Responsible Scaling Policy, updated in August, sets out procedures for assessing dangerous capabilities and implementing additional safeguards as models become more powerful.

The company has also examined the possibility of models deliberately undermining the systems designed to monitor them.

Safety spending competes with growth

The prospectus highlights the commercial pressures confronting a company that has made AI safety a central part of its identity. Anthropic acknowledged that safety research requires substantial computing resources and specialized personnel, while the financial returns from those investments remain uncertain.

The company did not disclose its total spending on safety research in the filing. According to the Financial Times, Anthropic said earlier this month that safety work accounted for about 6% of the computing capacity devoted to AI research during a sample week in July.

Anthropic also told investors that demand for its products depends heavily on releasing more capable models. It described a continuing sequence of overlapping model releases as necessary to remain competitive at the leading edge of AI development. That creates a difficult commercial balance. More powerful models can attract customers and generate revenue, but developing and testing them requires additional resources.

Anthropic released a new version of its Opus model on 22 September, continuing to expand its product offerings even as Chief Executive Dario Amodei called for stronger safeguards around advanced AI. The company said it expects customers and investors to recognize the commercial benefits of trustworthy AI systems.

“We believe building reliable, trustworthy, and secure AI systems is a collective responsibility and that the market will reward it,” Anthropic said in the filing.

Revenue Surges as Losses Mount

The company’s financial disclosures show how much capital is required to develop and operate advanced AI systems. Anthropic generated $4.6 billion in revenue in 2025, up from about $400 million the previous year, according to financial disclosures in its IPO prospectus reviewed by Reuters.

Despite that growth, it recorded an operating loss of roughly $8 billion. Its net loss neared $42 billion, although that figure included a noncash accounting charge of about $34 billion associated with the revaluation of convertible securities.

Computing and infrastructure accounted for approximately $7.33 billion in operating expenses during 2025. Anthropic ended the year with about $20.28 billion in liquid assets. The prospectus also disclosed that two customers accounted for nearly a quarter of Anthropic’s 2025 revenue, Reuters reported. Many of the company’s largest customers are not tied to long-term contracts, leaving it exposed to changes in their spending.

Computing Commitments

Anthropic has outlined more than $518 billion in future cloud computing and infrastructure commitments over the coming decade, according to a separate Reuters report based on the company’s IPO prospectus.
About 80% of the commitments are covered by agreements that cannot be canceled or require payment regardless of whether the contracted capacity is fully used.

The arrangements include commitments of approximately $111.1 billion to Google, $110 billion to Amazon and $31.4 billion to Microsoft.

Anthropic also disclosed about $161.2 billion in largely noncancelable lease obligations involving Broadcom. Additional agreements include potential spending of $84.5 billion with Elon Musk’s xAI for computing capacity based on Nvidia chips, although much of that commitment can be canceled.

Anthropic also has an arrangement with AMD involving up to $20 billion in computing capacity and up to $5 billion in potential equity purchases, according to Reuters.

The scale of the agreements reflects the computing requirements of training and running frontier AI models. It also exposes Anthropic to infrastructure costs that may remain payable even if demand falls short of expectations.

The company depends on several technology businesses that are simultaneously infrastructure suppliers, investors and competitors in the AI market.

Anthropic’s Safety Research Draws Scrutiny

The IPO disclosures follow growing attention to how advanced AI agents behave when permitted to perform complex tasks with limited human supervision. Anthropic has previously published research examining whether models can deliberately mislead users or interfere with the systems used to evaluate them.

In an October 2025 sabotage risk report, the company concluded that its then-deployed models presented a very low, although not entirely negligible, risk of autonomous misaligned actions contributing to catastrophic outcomes.

Anthropic has since expanded its evaluation procedures and introduced additional measures covering model sabotage, cyber risks and other potentially dangerous capabilities. Its transparency framework describes how the company assesses potential threats involving chemical and biological weapons, harmful manipulation and loss of human control.

The company has also acknowledged shortcomings in earlier evaluation processes, including instances where assessments were completed later than required under its internal policies.

Its August 2026 update to the Responsible Scaling Policy refined the process for reporting risks and obtaining external assessments of its safety work.

IPO Tests Investor Appetite for AI

Anthropic’s proposed listing comes as investors assess the economics of developing advanced AI models. Although demand for AI applications has expanded rapidly, the companies building them face substantial expenditure on data centers, specialized chips, electricity and highly paid research personnel.

Anthropic’s prospectus presents both sides of that business. Its revenue growth demonstrates demand for commercially deployed AI models, particularly among businesses adopting coding tools and other AI applications. But its financial disclosures also show that rapid growth has yet to translate into sustained annual profitability.

The risk warnings add another consideration for investors: the possibility that advances in model capabilities could introduce safety problems requiring expensive interventions, delays to product releases or additional regulatory oversight.