AI Could “Kill All Humans” In Next Decade, Anthropic Lead Says
Overview & Key Takeaways
Anthropic alignment science lead Evan Hubinger has said the AI company behind Claude and other AI products "earnestly" believes that AI "could kill all humans," and it could happen in the next decade.
Posting in response to Anthropic AI researcher Jacob Coxon's own resignation from the company due to safety concerns, Hubinger said, "We really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade."
He added: "I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to."
Hubinger, who still works at Anthropic, went on to cite Anthropic's own latest risk report that says the risk of AI wiping out humanity from present models is "low." However, Hubinger said he is worried that superintelligence could arise from "recursive self-improvement," which is the term for AI rewriting its own code and constantly improving itself.
For his part, Coxon said the AI market leaders Anthropic and OpenAI are not "acting responsibly" with regards to safety. "Do not underestimate the power of this technology. These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources. We have all witnessed the progress in each of these domains, and progress is not slowing," Coxon wrote.
He said Anthropic is pushing ahead despite these concerns because the company is "locked in a race to get there first." He said a company like OpenAI isn't taking the issue seriously enough but Anthropic understands the "civilizational stakes," and, if "no one else will act responsibly ... they must do it themselves, despite the risk."
Unsurprisingly, these posts have generated a lot of debate and discussion. Senator Bernie Sanders of Vermont said, "AI threatens our economy, privacy, democracy and the well-being of our kids." He is lobbying Congress to "stand up to the big tech oligarchs and protect the American people."
Senator Chris Murphy of Connecticut, meanwhile, advocated for "a regulatory system that allows the research to continue, but in a way that doesn't destroy us."
https://twitter.com/ChrisMurphyCT/status/2097669346814304677
Veteran media and technology reporter Taylor Lorenz, meanwhile, said Hubinger's comments are "sanctimonious doomer" statements that may lead to "passing the worst laws imaginable." She called on Hubinger, if he truly believes AI systems could endanger society, to provide "actual proof and receipts."
"Otherwise you're just vagueposting and fomenting fear which will result in terrible policy," she said.
Another matter at play here is that Anthropic is expect to go public this year at an expected valuation of north of $2 trillion. Some have theorized that the "kill all humans" comments from Anthropic itself could be strategic move to create panic in the marketplace in a bid to follow the business strategy of "create the problem and then sell the solution." If Anthropic expects there to be AI regulation, the thinking goes that they could be the ones to help write the rules in ways that would benefit them. Anthropic boss Dario Amodei previous said AI could eliminate half of all entry-level white collar jobs in the coming years, comments that many took to be aimed at fundraising.
Whatever the case, AI business today is helping fuel the growing US economy, and many are expecting the bubble to burst, leading to big pain in the market. AI is a big topic in video games, too, and it was just recently revealed that EA Sports used generative AI to clone a commentator's voice. The use of AI and generative AI in video games has been controversial, with many expressing fears of numerous negative effects like job losses and heavier workloads.
Anthropic alignment science lead Evan Hubinger has said the AI company behind Claude and other AI products “earnestly” believes that AI “could kill all humans,” and it could happen in the next decade.
Posting in response to Anthropic AI researcher Jacob Coxon’s own resignation from the company due to safety concerns, Hubinger said, “We really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade.”
He added: “I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.”
Hubinger, who still works at Anthropic, went on to cite Anthropic’s own latest risk report that says the risk of AI wiping out humanity from present models is “low.” However, Hubinger said he is worried that superintelligence could arise from “recursive self-improvement,” which is the term for AI rewriting its own code and constantly improving itself.
For his part, Coxon said the AI market leaders Anthropic and OpenAI are not “acting responsibly” with regards to safety. “Do not underestimate the power of this technology. These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources. We have all witnessed the progress in each of these domains, and progress is not slowing,” Coxon wrote.
He said Anthropic is pushing ahead despite these concerns because the company is “locked in a race to get there first.” He said a company like OpenAI isn’t taking the issue seriously enough but Anthropic understands the “civilizational stakes,” and, if “no one else will act responsibly … they must do it themselves, despite the risk.”
Unsurprisingly, these posts have generated a lot of debate and discussion. Senator Bernie Sanders of Vermont said, “AI threatens our economy, privacy, democracy and the well-being of our kids.” He is lobbying Congress to “stand up to the big tech oligarchs and protect the American people.”
Senator Chris Murphy of Connecticut, meanwhile, advocated for “a regulatory system that allows the research to continue, but in a way that doesn’t destroy us.”
Veteran media and technology reporter Taylor Lorenz, meanwhile, said Hubinger’s comments are “sanctimonious doomer” statements that may lead to “passing the worst laws imaginable.” She called on Hubinger, if he truly believes AI systems could endanger society, to provide “actual proof and receipts.”
“Otherwise you’re just vagueposting and fomenting fear which will result in terrible policy,” she said.
Another matter at play here is that Anthropic is expect to go public this year at an expected valuation of north of $2 trillion. Some have theorized that the “kill all humans” comments from Anthropic itself could be strategic move to create panic in the marketplace in a bid to follow the business strategy of “create the problem and then sell the solution.” If Anthropic expects there to be AI regulation, the thinking goes that they could be the ones to help write the rules in ways that would benefit them. Anthropic boss Dario Amodei previous said AI could eliminate half of all entry-level white collar jobs in the coming years, comments that many took to be aimed at fundraising.
Whatever the case, AI business today is helping fuel the growing US economy, and many are expecting the bubble to burst, leading to big pain in the market. AI is a big topic in video games, too, and it was just recently revealed that EA Sports used generative AI to clone a commentator’s voice. The use of AI and generative AI in video games has been controversial, with many expressing fears of numerous negative effects like job losses and heavier workloads.
