Can Anthropic’s IPO Survive Its Dystopian AI Warnings?

On September 8, Anthropic researcher Jacob Coxon announced he was leaving the company and accused it and OpenAI of gambling with human lives in the race toward self-improving AI (recursive self-improvement). Then Evan Hubinger, who leads Anthropic’s alignment science work, gave his own estimate that there’s more than a 10% chance AI could kill everyone within the next decade. 

Oh, and in case you’ve been on vacation since June 1, Anthropic is also preparing to sell shares to the public. Makes those statements tough to shrug off.

Before anyone accuses me of throwing Hubinger under the bus with Coxon, it’s important to note that he later said the risk from current models was low. In fairness, he was talking about future superintelligence (RSI), and that 10% figure was his personal estimate, not an official Anthropic forecast. 

So, if I understand Hubinger correctly, it would be misleading to suggest Claude has a 10% chance of killing its users. Phew. That was a close one.

In June, Anthropic announced that it had confidentially submitted a draft S-1 in preparation for its IPO. And part of that job is telling prospective investors what could hurt the business. Management, the board, lawyers, and bankers all have a hand in putting that story together. 

So where do these warnings about human extinction fit in, and how does the company explain them to someone considering buying the stock?

Every S-1 has a risk section. Customers could leave, a competitor could undercut prices, or a chip shortage could slow growth. Fine. But when someone leading your alignment research says future AI might end humanity, I think prospective shareholders deserve a heck of a lot more explanation than they’d get from the usual run-of-the-mill paragraph about competition.

The SEC requires companies to discuss material risks, but an individual employee’s opinion doesn’t automatically meet that test. 

The SEC also doesn’t endorse an IPO just because a registration statement becomes effective. 

But an IPO is also a sales process, and perception matters. Hubinger isn’t a random person commenting online; he’s been with the company for between three and four years and leads its alignment science work. When he puts a number on a potential catastrophe, investors will ask whether Anthropic’s leadership agrees with him.

And if it does, what does that belief mean for the business? Could a model release be delayed? Might customers hesitate to use the product, or might insurers and lawyers see liability that hasn’t been priced in? Those aren’t abstract questions when buyers are being asked to value a company on years of future growth.

We could stop right there and probably agree that Anthropic has some “splainin” to do! But where’s the fun in that? 

Let’s pivot for a moment to Anthropic CEO Dario Amodei’s new essay. He calls for slowing the pace of advances in AI capabilities and proposes outside evaluators, coordination among leading labs, and a role for governments.

I understand why he wants safeguards to catch up. But if the CEO believes development needs to slow, investors should know how that could affect releases, future revenue growth, and overall projections behind an IPO price. How does he decide when to put the brakes on a product his shareholders expect him to sell? 

The tobacco industry’s 1994 congressional hearing offers a lesson in how closely people examine what companies say about their products. Executives denied believing nicotine was addictive. I’m not saying AI’s speculative risks are equivalent to smoking’s documented outcomes, and I’m not accusing Anthropic of deception. But I am saying investors should be able to compare its public warnings, any internal assessments, and the eventual IPO filing without feeling that they’re reading three different stories.

None of this means an Anthropic IPO can’t happen. 

People invest in risky businesses all the time when they understand the risk and see enough potential reward. What bothers me is the possibility of selling investors a fast-growth story while the people closest to the technology are publicly questioning whether it should move that fast.

I also wouldn’t assume these warnings have delayed Anthropic’s or OpenAI’s offering without evidence. If an IPO’s timing does slip, some investors might postpone selling existing AI stocks they’d planned to sell to raise cash to invest in Anthropic or OpenAI’s offering. Less selling would likely be a net benefit for the industry in the immediate term. But a delay driven by unresolved safety or business questions might make investors nervous about the entire industry. I’d want to know why the offering moved before calling it good or bad news for the rest of AI.

Look, I’m not trying to take the fun out of AI. I’m optimistic about it, and I plan to keep learning, designing agents, and building apps. I think the investment opportunities ahead are enormous.

But when a company asks the public to finance its growth, investors deserve a straight answer: Does its leadership believe it can safely build and sell the next generation of products on the timetable behind its IPO valuation?

At the time of publication, Byrne had no positions in any securities mentioned.