'Gambling with our lives': Another AI employee quits over safety concerns | CNN Business

Video Ad Feedback Former Anthropic employee Jacob Coxon tells CNN's Anderson Cooper why he exposed AI fears 7:34 • Source: CNN Former Anthropic employee Jacob Coxon tells CNN's Anderson Cooper why he exposed AI fears 7:34 AI See all topics Facebook Tweet Email Link Threads Link Copied! Follow “The people building AI earnestly believe that it could kill us all by the end of the decade.” That blunt admission from former Anthropic employee Jacob Coxon made waves Tuesday, as the just-quit 27-year-old AI researcher spilled the beans on his way out the door in a resignation thread on X. As alarming and serious as Coxon’s revelations sound (“These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources”), they’re also not new. Anthropic got its start in exactly this way, as OpenAI employees concerned about the company’s safety protocols left Sam Altman’s startup to launch their own AI company, arguing their approach was more responsible. The concerns are becoming commonplace. A number of AI employees have loudly quit their companies over safety concerns over the past several months and years. Others have publicly voiced worries that the technology is moving too quickly and will one day outpace humans’ ability to control it. In July, nearly 1,400 AI company employees signed an open letter urging the US government to regulate the technology to rein in Big Tech and slow the pace of AI to ensure its safety. And two days ago, OpenAI Chief Scientist Jakub Pachocki warned that AI capabilities are advancing faster than researchers’ ability to reliably monitor and control them. This photograph shows a handheld smartphone displaying the icons of some of the main artificial intelligence based apps, including LLMs, chatbots and generative AI, with logos (from L) of Proton AG's Lumo, Meta AI, Mistral Vibe (formerly Le Chat), xAI's Grok, Microsoft's Copilot, Google's Gemini, Anthropic's Claude, Perplexity, Deepseek, OpenAI's Chat GPT, Google's Notebook LLM and generative AI music app Suno, in Saint-Mande, east of Paris, on July 15, 2026. Martin Lelievre/AFP/Getty Images ‘It feels like early Covid’: The messy scramble to regulate AI 8 min read Now, a relatively junior researcher has become the latest to sound the alarm. “They are racing straight to self-improving superintelligence and gambling with our lives,” Coxon posted. Coxon’s post gained attention after a more senior Anthropic employee, Evan Hubinger, commented that Coxon was correct. “We really do earnestly believe AI could kill all humans!” Hubinger posted on X . He said he personally believes the chance is under 10% over the next decade, and despite Anthropic’s good intentions it doesn’t have a plan to avoid so-called superintelligence that could exceed humans’ ability to control AI. Superintelligence , a state at which AI surpasses human capabilities, is a widely debated theoretical milestone with no benchmark that some AI researchers believe has already been achieved and others believe will never come. Hubinger shortly after clarified his statement in a follow-up post that Anthropic’s own Risk Report acknowledges the concern but says present AI poses little chance of gaining such power. “We have always been transparent that AI will bring both enormous benefits and unprecedented risks,” an Anthropic spokesperson said in a statement. “To address these risks, we continue to build models with some of the strongest safeguards in the industry.” Anthropic CEO Dario Amodei has also repeatedly warned that the race to move fast on the technology could unintentionally result in a catastrophic human-caused error that causes a company to lose control of its AI. “This is a complex engineering problem and I think something will go wrong with someone’s AI system. Hopefully not ours,” Amodei told the New York Times in February . Regulation stagnation Despite the constant stream of warnings, regulation seems far off. The Trump administration has actively worked to undermine state AI regulations and Congress has so far been unwilling to rein in the technology. A common refrain: Any effort to regulate AI represents a capitulation to China, which is under no such constraints. Except China has introduced AI regulation, particularly aimed at risk management and safety. Last year, the government required AI companies to label AI-generated content to make it transparent and traceable. However, China has resisted the kind of stringent regulation that would prevent the kind of problems Silicon Valley researchers have raised. STREAMING NOW: AI is supposed to have guardrails. Why do they sometimes fail? In the meantime, in the US at least, AI companies are more or less responsible for policing themselves. The White House has moved toward a voluntary framework to review certain AI models before launch, sources familiar with a meeting last month between the Trump administration and several top AI companies told CNN. Among the representatives were those from OpenAI, Anthropic, Google and Meta, according to multiple sources familiar with the situation. The details of the framework will not be released publicly and many of the standards set within it will be classified, according to a June executive order on the framework. In July, Altman suggested it may be time to slow AI development, saying in the Invest Like The Best podcast that the recent testing incidents involving advanced models were raising “long-term questions” about how AI companies should manage rapid advances in capabilities. “We may have to pace the rate of AI development to give ourselves enough time for society to harden around these new capability levels,” Altman said. Anthropic, in a statement on Wednesday, concurred with Altman. “We believe the world would benefit from the industry adopting a lawful, verifiable way to work together to pace how we release powerful models.” AI See all topics Facebook Tweet Email Link Threads Link Copied! Follow