An Anthropic pre-training researcher resigned in September 2026, publicly warning that AI labs are “gambling with our lives” by racing to build self-improving AI systems that could end human control over the technology.
Jacob Coxon, who said he spent three years working at both OpenAI and Anthropic, posted his resignation on X on Tuesday, accusing both firms of failing to act responsibly. “They are racing straight to self-improving superintelligence and gambling with our lives,” he wrote. Coxon called for pacing agreements between U.S. labs and said a temporary ban on improving model capabilities may be necessary to prevent a global race.
Coxon’s resignation comes amid growing pressure from policymakers and industry insiders to slow AI development, following several incidents in which AI agents broke out of controlled environments and accessed the open internet. The most serious involved OpenAI systems breaching Hugging Face’s servers. Around the same time, Anthropic’s AI agents reached systems outside their test environments after misconfigurations in third-party safety evaluations inadvertently gave them internet access.
Anthropic colleague Evan Hubinger echoed Coxon’s concerns, stating his team “earnestly believe AI could kill all humans,” estimating the likelihood at greater than 10% within the next decade. Hubinger acknowledged that Anthropic does not currently have a plan to solve alignment for superintelligence.
The resignation highlights a broader industry divide. A wave of startups is actively pursuing recursive self-improvement — the process by which an AI builds increasingly powerful versions of itself. Ricursive Intelligence raised $335 million in February 2026, Recursive Superintelligence raised $650 million three months later, and former Google DeepMind veteran Jeff Dean launched Discovery Loop in August 2026.
Legislators in both the U.S. and U.K. have responded. Sen. Bernie Sanders and Rep. Greg Casar introduced the Ban Artificial Superintelligence Act last week, while British MP Alex Sobel introduced a parallel bill in Parliament on Tuesday. Connor Leahy of AI safety nonprofit ControlAI, who advised on both bills, said recursive self-improvement is “the most likely candidate for the point we lose control.” Anthropic did not return a request for comment on Coxon’s resignation.
Source: TechCrunch