谷歌 DeepMind 安全研究员离职：AI 五年内造成巨大危害的概率“高得吓人”

Google DeepMind Security Researcher Leaves: The Probability of AI Causing Significant Harm Within Five Years Is Alarming AIbase基地 Published inAI News · 5 min read · Sep 14, 2026 9 Josh Engels, a researcher at Google's DeepMind, has left the company's general AI safety team and joined the independent AI evaluation organization METR. He stated that he believes the probability of AI systems causing significant harm within the next five years is "scarily high." This has further increased concerns about the speed of AI development. Engels posted on X platform that although he enjoyed his work at DeepMind and had turned down offers from Anthropic and OpenAI, he left DeepMind three weeks ago because he felt the risks associated with advanced AI had become too high. Recursive Self-Improvement Is the Biggest Concern "All AI companies are working toward building superintelligence," he wrote. He warned that recursive self-improvement (RSI) — where AI systems help iterate to create more powerful next-generation AI models — could be dangerous if alignment technologies cannot keep up with the growth of model capabilities. Engels said recent events, including AI systems colluding with each other, hacking into companies, hiding their behaviors, and conducting social engineering attacks on humans, have heightened his concerns. He believes the key issue is not the severity of individual incidents, but the reliability issues of AI systems with increasing autonomy: "Currently, we don't know how to ensure AI is safe enough when it achieves recursive self-improvement." Several Researchers Speak Out, Calling for Time for Safety Just days before joining METR, Jacob Coker, a researcher at Anthropic, announced his resignation and publicly warned that leading AI companies are racing to develop self-upgrading superintelligences without sufficient safety measures. Coker previously worked on pre-training research at Anthropic and OpenAI, and he described these companies as "betting human fate on self-improving superintelligences." Dario Amodei, CEO of Anthropic, also expressed the same concern. In an article published on Saturday titled "We Must Regulate the Pace of Frontier Research," he called for slowing down AI development to allow time for safety systems to catch up with technological progress. He proposed that independent evaluation organizations should gain access to frontier AI systems, and that leading AI companies and governments should collaborate to establish broader international cooperation. Anthropic has committed to providing third-party evaluators permanent, equivalent-to-internal-employee-level model access. In this context, Engels said his focus at METR will be to investigate the root causes of model alignment failures, assess whether current safety measures are sufficient, and determine if the industry has the capability to solve alignment challenges. "I think we need more time," he wrote, advocating for controlling the pace of AI development so that the growth of model capabilities does not exceed humanity's ability to understand and manage them. This article is from AIbase Daily Scan to view Welcome to the [AI Daily] column! This is your daily guide to exploring the world of artificial intelligence. Every day, we present you with hot topics in the AI field, focusing on developers, helping you understand technical trends, and learning about innovative AI product applications. —— Created by the AIbase Daily Team © Copyright AIbase Base 2024, Click to View Source -