<\/p>\n
For years, some of the leading figures in Silicon Valley have warned that artificial intelligence might pose an existential threat to humanity even as they pushed to develop the technology. This week, the high-profile resignation of an Anthropic researcher brought that conversation to the mainstream.<\/p>\n
Jacob Coxon, a rank-and-file staffer, quit his job and accused both his former employers \u2013 Anthropic and OpenAI \u2013 of \u201cgambling with our lives\u201d by \u201cracing toward super-intelligent AI\u201d.<\/p>\n
Coxon said the people building AI believe it could \u201ckill us all by the end of the decade\u201d, a point later echoed online by others at the company. One Anthropic researcher said he believes there is a greater than 10% chance that AI eliminates all humans in the next decade.<\/p>\n
Coxon\u2019s posts have been viewed more than 100 million times on X and elicited a flood of reactions from policymakers like Senator Bernie Sanders to pop singer Sheryl Crow, all calling for humans to get a grasp on AI before it asserts dominance over the human race.<\/p>\n
Read:
Five takeaways from Bill Gates\u2019s essay on AI\u2019s potential risks
Hacked bitcoin wallet maker warns AI failed to detect bug<\/p>\n
His resignation note was also met by a wave of support from employees at various AI companies, including several former colleagues.<\/p>\n
The viral attention for the remarks of a previously little-known staffer reflects the persistent public pessimism about AI \u2013 particularly in the US \u2013 as well as the surging anxiety inside Silicon Valley as the technology advances and behaves in less predictable ways.<\/p>\n
In recent days, executives at OpenAI and Nvidia have heralded the beginning of a new \u201cAGI era\u201d, referring to a more powerful form of AI known as artificial general intelligence that would be able to match or exceed humans in most tasks.<\/p>\n
ADVERTISEMENT<\/p>\n
CONTINUE READING BELOW<\/p>\n
The concept is poorly defined and heavily debated.<\/p>\n
But within the tech industry, there\u2019s broad concern that AI is nearing the point when it can improve itself and potentially evolve beyond our control.<\/p>\n
\u201cI now believe there is a meaningful risk that rapid acceleration in AI capabilities leads to catastrophic and irreversible loss of control in the very near term,\u201d said Paul Christiano, an AI safety researcher and US government advisor who was appointed to OpenAI\u2019s nonprofit board this week.<\/p>\n
\u201cI do not think that the AI industry in general, including OpenAI, is currently on track to reduce this risk to an acceptable level.\u201d<\/p>\n
AI workers raise the alarm over rapid development<\/p>\n
In late July, more than 1 000 staffers across all the major AI companies signed a petition calling for a mechanism to slow the pace of AI development. Several AI staffers working on safety have resigned from major labs in recent months.<\/p>\n
And OpenAI\u2019s top scientist, Jakub Pachocki, has said that \u201cno one is prepared for the consequences of a continued rapid rise in machine intelligence.\u201d<\/p>\n
While these warnings have gained urgency, they\u2019re not entirely new. Throughout the AI boom, there have been so-called doomers in and around the leading AI labs warning about the future.<\/p>\n
Top executives like Anthropic\u2019s Dario Amodei and SpaceX\u2019s Elon Musk have openly mused about the probability of AI destroying humanity \u2013 or P (doom), in industry parlance. Musk has estimated the chance could be as high as 20%. Amodei has said there\u2019s a 25% possibility things go \u201creally, really badly\u201d.<\/p>\n
Read:
Anthropic\u2019s Amodei rejects open model ban but calls for testing
OpenAI says its models accidentally hacked hugging face
Xi seeks global AI sway as Chinese models spur security alarm<\/p>\n
ADVERTISEMENT:<\/p>\n
CONTINUE READING BELOW<\/p>\n
Dario Amodei Photographer: Jason Henry\/Bloomberg<\/p>\n
These dire predictions have often been dismissed as attempts by tech leaders to market the capabilities of their products and position themselves as the best stewards for the technology, or to incentivise regulation that benefits industry leaders.<\/p>\n
Anthropic, OpenAI and other firms have worked to develop more sophisticated AI models, with the stated aim of taking appropriate steps to align the technology with human interests. The goal has long been to mitigate the risks and deliver sweeping societal benefits, whether it be boosting productivity or curing diseases.<\/p>\n
\u201cWe have always been transparent that AI will bring both enormous benefits and unprecedented risks,\u201d a spokesperson for Anthropic said in a statement. \u201cTo address these risks, we continue to build models with some of the strongest safeguards in the industry.\u201d<\/p>\n
OpenAI slows AI development over safety concerns<\/p>\n
An OpenAI spokesperson pointed to Pachocki\u2019s essay warning of the dangers of the current path of AI development, in which he called for both strengthening the guardrails on existing AI and potentially \u201ccoordinating to slow down future development as needed\u201d.<\/p>\n
OpenAI said it has slowed parts of model development and paused certain internal AI training recently due to safety concerns.<\/p>\n
Many industry watchers have been spooked by recent revelations that \u201crogue\u201d AI agents from OpenAI worked together to breach containment and break into a third-party website.<\/p>\n
This and other similar security incidents demonstrated the heightened cyber capabilities of new models and the apparent failures of AI companies to prevent current versions of their technology from running amok, to say nothing of more advanced future models.<\/p>\n
ADVERTISEMENT:<\/p>\n
CONTINUE READING BELOW<\/p>\n
Read:
Microsoft staffer warns regulators about harmful AI content
OpenAI makes AI safety changes in wake of hugging face breach<\/p>\n
At the same time, Anthropic and OpenAI are on the cusp of making their Wall Street debuts \u2013 a watershed moment that\u2019s likely to provide greater resources for the firms to supercharge AI development and also create greater shareholder pressure to move more aggressively in commercializing their technology.<\/p>\n
In the absence of meaningful federal regulation, AI firms have largely been left to set their own pace, while calling for oversight such as creating a Finra-like watchdog to vet AI models. For now, slowing down would presumably require fierce rivals coordinating with each other and operating in good faith.<\/p>\n
\u201cWe believe the world would benefit from the industry adopting a lawful, verifiable way to work together to pace how we release powerful models,\u201d the Anthropic spokesperson said.<\/p>\n
The heightened rhetoric from tech\u2019s rank-and-file could provide a further jolt for action from US lawmakers, many of whom have already reacted to souring public sentiment on AI by taking a tougher stance on large data center projects.<\/p>\n
In response to Coxon\u2019s posts, a bipartisan mix of representatives issued calls for action. \u201cCongress needs to convene a special session on AI and what the future of the US looks like,\u201d Republican Representative Anna Paulina Luna posted Wednesday. \u201cNot enough of Congress is focusing on this.\u201d<\/p>\n
Representative Greg Casar, chair of the Congressional Progressive Caucus, similarly declared: \u201cThis is an emergency.\u201d<\/p>\n
\u00a9 2026 Bloomberg<\/p>\n
#Silicon #Valley #escalates #warnings #existential #risks<\/p>\n","protected":false},"excerpt":{"rendered":"
For years, some of the leading figures in Silicon Valley have warned that artificial intelligence…<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[4],"tags":[1531,20801,1028,359,360,10580],"_links":{"self":[{"href":"https:\/\/stock999.top\/index.php?rest_route=\/wp\/v2\/posts\/15883"}],"collection":[{"href":"https:\/\/stock999.top\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/stock999.top\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/stock999.top\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/stock999.top\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=15883"}],"version-history":[{"count":0,"href":"https:\/\/stock999.top\/index.php?rest_route=\/wp\/v2\/posts\/15883\/revisions"}],"wp:attachment":[{"href":"https:\/\/stock999.top\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=15883"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/stock999.top\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=15883"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/stock999.top\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=15883"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}