Jacob Coxon, a researcher who spent three years working on pretraining at both Anthropic and OpenAI, resigned on September 8, publicly accusing the two companies of racing toward self-improving AI faster than they can control. His departure has cast a spotlight on whether Anthropic's voluntary safety controls can withstand commercial pressures as the company scales its operations.
The resignation, reported by the Associated Press on September 9, is a concrete event, but Coxon's forecast of catastrophic harm remains an opinion rather than an observed capability of current models. The 2026 International AI Safety Report, which involved input from over 100 independent specialists, found early signs of relevant capabilities but not at levels that would enable a loss of control. The report described the probability, timing, and nature of such risks as unusually uncertain, underscoring the need for investors and policymakers to separate speculation from evidence.
Governance and Policy Framework
Anthropic, structured as a public benefit corporation, has a governance framework that includes stockholders and a Long-Term Benefit Trust in oversight roles. A resignation tied directly to the company's core mission is more significant than a typical personnel change, even if it does not prove a policy breach. The company's Responsible Scaling Policy, last updated on August 14, and version 3.4 effective July 8, require unredacted risk reports to be shared with at least 200 employees and allow different external reviewers to examine separate sections. The company also maintains a noncompliance-reporting and anti-retaliation policy.
These procedures address compliance and review, but Coxon's concerns center on incentives before formal thresholds are crossed. The critical question is whether Anthropic publishes enough information for outsiders to evaluate its decisions, rather than whether the public accepts a single prediction about extinction.
Commercial Stakes and Investor Implications
The stakes are high as Anthropic's safety policy sits alongside a rapidly expanding commercial operation. In a May 28 funding announcement, Anthropic revealed it raised $65 billion at a $965 billion post-money valuation and had passed $47 billion in run-rate revenue. The round included $15 billion of previously committed hyperscaler investment, with $5 billion from Amazon. Anthropic also disclosed agreements for up to five gigawatts of new Amazon capacity and five gigawatts of next-generation Google and Broadcom TPU capacity.
For Amazon and Alphabet shareholders, Coxon's resignation is not a direct hit to revenue or earnings, but the connection is indirect. Both companies supply capital and computing infrastructure to Anthropic, so tighter model-development rules or external-evaluation requirements could alter the timing and utilization of large data-center commitments. Conversely, stronger and more credible controls could enhance enterprise adoption by reducing operational and regulatory risks.
Counterarguments and Next Steps
There is a counterargument to viewing the departure as evidence of institutional failure. Anthropic publishes risk reports, records changes to its scaling policy, and has a formal route for employees to flag noncompliance. Coxon did not provide evidence that Anthropic violated these rules. A safety researcher may reasonably find the industry's direction unacceptable even when a company complies with its governance framework.
The next measurable checkpoint is imminent. Anthropic's Frontier Safety Roadmap sets September 30 targets for an initial assessment of extreme security practices and a prototype method for cryptographically attributing outputs to specific model weights. The roadmap says the company must decide next steps for the security assessment within two weeks of completion.
Meeting these dated goals would not resolve the broader debate Coxon raised, but missing them, weakening them without explanation, or narrowing external review would be more actionable evidence. Anthropic's September 30 disclosures, the scope of external review, and any formal response to Coxon's governance criticism are the facts to watch.
