AI staff ‘genuinely frightened’ for humanity’s future, ex-Anthropic researcher tells BBC
A chilling revelation has emerged from the heart of the artificial intelligence industry, sending ripples of concern through scientific and political circles alike. Jacob Coxon, a former researcher at leading AI firm Anthropic, has candidly disclosed that many within the sector are “genuinely frightened” by the breakneck pace of AI development and the profoundly uncertain implications it holds for humanity. His stark warning, delivered to the BBC, centers on the palpable fear of an unforeseen future, with a real possibility of an AI extinction risk looming.
Coxon, a mere 27 years old, recently resigned from Anthropic, a company at the forefront of AI research. His departure was marked by a viral social media post articulating his deep-seated anxieties about AI’s escalating dangers. He minced no words in his interview, stating, “I believe that if we don’t slow down at the current rate of progress, there is a strong chance that we could all die in the immediate future.” This isn’t merely a speculative academic debate; it’s a grim assessment from someone immersed in the technology’s creation, reflecting a growing internal disquiet that has now spilled into the public consciousness.
The Industry’s Divided Stance on AI Extinction Risk
The call for a significant deceleration in AI development isn’t unique to Coxon. His former boss, Dario Amodei, Anthropic’s head, has similarly advocated for a slowdown. While Amodei’s motivations have faced scrutiny from some corners of Silicon Valley, his proposal has found powerful allies in other tech titans. Sam Altman, CEO of rival OpenAI, and Elon Musk, head of xAI, have both publicly supported Amodei’s call for industry-wide deceleration, stringent regulation, and independent oversight of AI model development. Amodei, in a recent essay, stressed that while the pursuit of this transformative technology is undeniable, the associated risks are undeniably “serious.” He urged companies and governments to allocate ample time to understand and effectively address these multifaceted dangers.
Coxon, who also worked at OpenAI before joining Anthropic, wholeheartedly welcomed the notion of a slowdown. However, he emphasized a crucial caveat: such a pause must be meticulously coordinated internationally, particularly with China, to avert a perilous “race at an international scale.” This geopolitical dimension underscores a profound challenge; the fear among developers, Coxon revealed, is that they are trapped in a competitive sprint, one whose ultimate outcomes fill them with dread. This sentiment highlights the complex interplay between technological advancement, national interests, and the collective safety of humankind. Indeed, the shadows of geopolitical tensions often loom large over such critical technological races.
When pressed on the specifics of an “AI apocalypse,” Coxon admitted it was the most difficult question to answer. Yet, he presented a terrifyingly plausible scenario, one alluded to by Amodei: a swarm of highly advanced bots, functioning as a distributed supercomputer, capable of seizing control of the internet. Coxon chillingly suggested this could become a reality within six months to a year, a timeline that injects an alarming sense of urgency into the discourse.
Anthropic’s Defense and the Broader Chorus of Concern
In response to Coxon’s departure and his public comments, an Anthropic spokesperson affirmed the company’s long-standing transparency regarding AI’s dual nature – immense benefits coupled with unprecedented risks. They stated their commitment to building models with “some of the strongest safeguards in the industry,” highlighting their pioneering work in studying AI model behavior, publishing risk mitigation frameworks, and aggressively testing models to prevent “AI misalignment.” The company maintains that industry-wide collaboration on a “lawful, verifiable way to pace how we release powerful models” is essential.
The anxieties articulated by Coxon are far from isolated. Other prominent voices within the AI community have echoed similar sentiments. Anthropic scientist Evan Hubinger has publicly declared his personal belief that there is a “>10%” chance AI could lead to human extinction within the next decade. This unsettling statistic gained further validation from none other than Geoffrey Hinton, the Nobel Prize-winning computer scientist widely recognized as the “Godfather of AI.” Hinton, in his own recent interview, deemed a 10% chance of AI causing human demise to be “not unreasonable.” Marc Warner, CEO of AI safety firm Faculty, while finding it “extremely hard to place a probability,” stressed the absolute sincerity of these warnings from industry insiders. Even former UK Prime Minister Rishi Sunak, an advisor to Anthropic, has expressed his worries about AI’s potential risks to humanity, despite his overall optimism regarding the technology’s promise, as detailed in his Sunday Times column.
Skepticism, Economic Motives, and the Race for Dominance
However, the narrative surrounding AI’s existential threat is not universally accepted. A notable segment of industry figures dismiss these dramatic warnings as either overstated or strategically designed to generate hype and investment. Clement Delangue, CEO of the AI platform Hugging Face, famously likened asking Coxon about extinction risk to consulting an AC technician about climate change, implying a potential lack of broad perspective. Yet, even Delangue later offered to collaborate on solutions proposed by Amodei, suggesting a nuanced position.
More pointed criticism has come from figures like Jensen Huang, CEO of chip-making giant Nvidia, who reportedly dismissed Coxon’s concerns as untrue during a conference. George Arison, CEO of Grindr, posited that such alarming statements might serve to inflate company valuations, justifying astronomical figures by promising AI’s ubiquitous takeover of industries and jobs. This cynical perspective introduces a crucial economic layer to the debate: is the AI extinction risk truly existential, or is it, in part, a calculated PR strategy in a high-stakes investment game?
Further fueling this skepticism are accusations that companies like Anthropic might be attempting to trigger regulatory action designed to stifle competition, thereby entrenching a lucrative duopoly with firms like OpenAI. With Anthropic reportedly gearing up for a potentially record-setting initial public offering and OpenAI valued at an astonishing $852 billion, the financial stakes are undeniably colossal. Meanwhile, geopolitical leaders like former US President Donald Trump have framed AI as a new frontier for international dominance, stating, “whoever wins AI wins.”
A Future Haunted by Unseen Possibilities
Beyond the boardrooms and political arenas, the personal toll on those developing this revolutionary technology is stark. Coxon painted a vivid picture of colleagues “planning what to do with their lives” and even “considering buying land somewhere because they’re so scared of the instability as a result of rapid AI progress.” This chilling detail underscores a profound psychological burden carried by those directly shaping the future. They live with the existential weight of their creations daily, grappling with both the immense potential for good – solving diseases, improving lives – and the terrifying specter of an unprecedented AI extinction risk.
The debate surrounding AI’s trajectory represents a critical juncture for humanity. The collective apprehension among AI researchers, the calls for regulatory frameworks, and the stark warnings from luminaries like Hinton cannot be easily dismissed. As the industry races forward, balancing innovation with prudent caution, and navigating the complex landscape of economic ambition and geopolitical competition, will be paramount to ensuring that AI becomes a tool for progress, not an architect of our undoing.