A former researcher at artificial intelligence company Anthropic has issued a stark warning about the future of advanced AI, arguing that the industry’s relentless push towards more powerful systems could have catastrophic consequences if safety measures fail to keep pace.
Jacob Coxon, who recently left the company, has publicly said leading AI developers are prioritising rapid technological advancement over the rigorous safeguards needed to keep future systems under human control. His comments have added fresh momentum to an ongoing debate within the technology sector over the risks associated with increasingly capable AI models.
I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives. More thoughts below.
— Jacob Coxon (@hilbertspaess) September 9, 2026
The warning comes as competition among major AI laboratories has intensified, with companies investing billions of pounds in next-generation systems capable of performing complex tasks once considered uniquely human.
Coxon’s concerns centre on what AI researchers describe as the “alignment problem” — the challenge of ensuring that highly advanced AI systems consistently act in accordance with human values and intentions. While current AI models remain tools controlled by human operators, some experts fear future systems could become far more autonomous and harder to manage.
The former researcher argued that the pace of development may be exceeding the industry’s ability to fully understand and mitigate emerging risks. He suggested that companies and policymakers should place greater emphasis on safety research before deploying increasingly powerful technologies.
His remarks echo concerns raised by several prominent AI safety researchers in recent years. Some experts have warned that if future AI systems become substantially more intelligent than humans, unforeseen behaviour could pose serious risks to society. These concerns range from economic disruption and misinformation to more extreme scenarios involving a loss of human oversight over critical systems.
Anthropic has positioned itself as one of the industry’s leading advocates for AI safety, frequently emphasising the importance of responsible development and risk assessment. The company has invested heavily in research designed to make advanced AI systems more transparent, predictable and aligned with human objectives.
However, critics argue that even organisations committed to safety face commercial pressures in an increasingly competitive market. The race among technology firms to develop more capable AI models has accelerated over the past two years, with new releases arriving at a pace that some researchers believe leaves insufficient time for thorough testing.
Related
- Advanced AI could pose ‘existential risk’ to humanity, UN rights chief warns
- Musk Names Google, China, SpaceX as Likely Winners in Global AI Race
- UBA Executive Delivers Warning on AI Operational Risks
- AI Researcher Job Description: Key Duties, Qualifications, and Career Outlook in Nigeria
- 5 Key Facts About Project Maven and AI in Warfare
Supporters of continued AI development maintain that advanced systems could deliver significant benefits across healthcare, scientific research, education and economic productivity. They argue that slowing innovation could delay breakthroughs that could improve millions of lives.
Nevertheless, Coxon’s warning highlights a growing divide within the AI community between those who see rapid progress as an opportunity and those who believe stronger safeguards are urgently required before the technology becomes more powerful.
As governments around the world consider new regulations for artificial intelligence, the debate over balancing innovation with safety is expected to intensify. For now, the comments from a former insider have once again placed the spotlight on one of the most consequential questions facing the technology industry: whether humanity can maintain control over the systems it is creating.
While experts remain divided on the likelihood of worst-case scenarios, they broadly agree on one point — the decisions made today on AI governance and safety could shape society for decades to come.
Senior AI Writer
Bio: Okikiola is a writer and AI enthusiast with a background in Office Technology and Management from the Federal Polytechnic Offa. She went further to study an MSc in International Business at De Montfort University (DMU). With extensive work experience across administrative and business roles, she now focuses on exploring how artificial intelligence can transform work, innovation, and everyday life.