An Anthropic safety researcher — the company behind Claude — posted on X a warning that generated over 10 million views, stating there is more than a 10% chance that artificial intelligence will kill all of humanity before 2036.
Evan Hubinger, who works on AI alignment at Anthropic, acknowledges that the risk from current models is low, but says he is concerned about AI’s ability to improve itself until it reaches a level of existential threat.
Anthropic researcher admits the company does not yet have a plan to ensure the safety of a AI superintelligence
Hubinger was direct in stating that Anthropic “genuinely believes” AI poses an extinction risk to the human species.
He added that Anthropic is doing its best, but that the company “does not yet have a plan to solve alignment for superintelligence and is not clearly on track to do so.”
AI alignment is the field of research that aims to ensure increasingly capable systems behave in accordance with human values and goals.
The comments came in response to Jacob Coxon, a researcher who recently left Anthropic after also working at OpenAI, who wrote that “none of the companies are acting responsibly” and that the systems being developed will “soon be superhuman, capable of hacking anything, revolutionising any field overnight, and acquiring real power and resources.”
Reactions were mixed. David Sherman of io.net said the warnings should not be dismissed and pointed to the concentration of AI development in a handful of companies as the main obstacle to dissenting voices being able to influence the direction of the technology.
Dame Wendy Hall, a scientist who advises the United Nations on artificial intelligence, said she was “shocked” by the posts, but did not rule out that some of the comments could be a marketing strategy by companies seeking investment and a potential stock market listing.
Anthropic’s own August safety report admits to being “less confident” in its risk assessments than previously, saying it is seeing “early signs of potential acceleration.”
The company also withheld its most recent model from the UK’s AI Safety Institute — a government body that evaluates the risks of advanced artificial intelligence systems.
More than 1,300 AI industry employees have signed an open letter calling on the US government to support an international effort to develop technical and governance tools that would allow for “deliberately controlling the pace of automated frontier AI development.”
Sobre o autor
Aproveite a campanha:
