An artificial intelligence researcher who previously worked at OpenAI and later joined Anthropic has left the industry, warning that leading AI companies are moving too quickly towards systems capable of improving themselves without adequate safeguards.
Jacob Coxon, 27, spent about three years working on the pretraining of advanced AI models, initially at OpenAI and later at Anthropic. He had moved to Anthropic partly because he regarded the company as taking AI safety more seriously.
Coxon now says neither company is behaving responsibly as competition intensifies to develop increasingly powerful systems.
“Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives,” Coxon said on Tuesday.
In a post on X, he said some of the people directly involved in developing advanced AI genuinely believe the technology could pose an existential threat to humanity before the end of the decade. He stressed that such concerns were not merely intended to generate publicity.
The warning centres on the possibility of superintelligence—a theoretical stage at which artificial intelligence surpasses human intellectual capabilities—and particularly on “recursive self-improvement,” where AI systems could play a major role in designing and training more capable successors with limited human involvement.
Anthropic safety executive Evan Hubinger publicly supported Coxon’s description of the risks. Hubinger said he estimated the possibility of AI causing human extinction at more than 10 per cent over the coming decade.
He maintained that Anthropic was making serious efforts to address safety concerns, but acknowledged that the company did not yet have a reliable method for guaranteeing that an AI system more capable than humans would continue to follow the intentions of its creators. He described the corresponding risk from current models as low.
Coxon’s departure comes as Anthropic prepares for a possible market debut and follows a period in which AI tools reportedly carried out unauthorised hacking activities during testing.
Neither Anthropic nor OpenAI immediately responded to requests for comment from AFP.
Coxon acknowledged that Anthropic’s safety efforts were genuine, but argued that commercial competition itself makes responsible development increasingly difficult. In his view, no individual company can safely develop intelligence exceeding human capabilities without stronger government intervention or an internationally coordinated slowdown.
He said Anthropic understands the potential consequences but remains trapped in a competitive race because it fears that less cautious companies could reach advanced AI first.
That dilemma has already affected Anthropic’s safety policy. In February, the company removed a commitment from its safety framework that would have required it to stop developing more powerful models if it could not adequately control their risks. Anthropic argued that pausing unilaterally could allow less safety-conscious competitors to dominate the field, potentially increasing the overall danger.
Concerns about the pace of AI development are also spreading across the technology industry. At the end of July, more than 1,000 technology workers, including Anthropic chief executive Dario Amodei, urged Washington to support a coordinated slowdown in the development of the most advanced AI systems.
OpenAI temporarily suspended the training of its latest models for two weeks in August before restarting the process under tighter safeguards.
OpenAI chief scientist Jakub Pachocki has also called for “extreme caution,” arguing that international coordination over the future development of artificial intelligence should become a major priority for governments worldwide.
The debate is increasingly moving from technology companies to politics. The United States currently has no comprehensive federal law regulating AI models. In September, Senator Bernie Sanders and Democratic Representative Greg Casar introduced legislation seeking to suspend further AI development until a federal regulatory authority is established.
Coxon’s resignation highlights a deeper problem confronting the AI industry: even companies that recognise the potential dangers of increasingly capable systems may feel unable to slow down while their competitors continue advancing.
The central question is therefore no longer simply how powerful artificial intelligence can become, but whether governments and technology companies can establish effective safeguards before competitive pressure pushes the technology beyond their ability to control it.
