Anthropic 称已阻止疑似利用 Claude 进行的生物武器研究

A new report from Anthropic, the AI firm that’s perhaps most positioned itself as the industry’s conscience, claims that the company had to thwart multiple potential schemes over the past year by scientists intending to use its Claude AI to engage in biological weapons research. The company’s Threat Intelligence team detailed five case studies on “biological misuse” of Claude in the report—including gain-of-function research that would make the mosquito-borne illness chikungunya more dangerous and a scheme to catalogue animal venom compounds”. Anthropic’s team noted that many of these cases were ambiguous, plagued (as so much biowarfare policy has been) by “dual use” problems in which projects meant to safeguard the public against biological weapons can often easily be inverted into making biological weapons themselves. The company said it hopes its report will “spark a conversation within the AI industry, and with governments, about emerging biological risks and how best to counter them.” Former Pentagon official Andrew Weber—who served for over five years as the assistant secretary of defense for nuclear, chemical, and biological defense programs—reviewed the report ahead of its publication on Thursday for The New York Times. Weber, now with the Council on Strategic Risks, told the Times that the company’s findings were “chilling examples of state-sponsored biological weapons developers tapping into the rapidly advancing capabilities” of the AI sector’s most state-of-the-art models. A cautionary fable Anthropic’s team noted that the risks posed by biowarfare research conducted with Claude have mostly emerged with recent releases, “most notably Claude Fable 5.” To that end, the company has worked to strengthen Fable 5’s ability to recognize when bad-faith actors are attempting to copy its capabilities via “distillation,” a similarly thorny “dual-use” technique whereby the work product of a more capable “teacher” AI is extracted to train a smaller “student” AI. Anthropic said in the report that it has foiled illicit distillation attacks targeting Claude from seven labs based in China since first making the issue public this February. It’s unclear if any of these distillation attacks overlapped with the bioweapons case studies, because the firm chose not to name the parties responsible. And, in some cases, many of these biological researchers were working hard to evade identification. Anthropic’s report noted that the parties attempting to create a more powerful and deadly strain of the chikungunya virus had “tunneled traffic through U.S. infrastructure to evade our regional blocks, and used a zero data retention (ZDR) service to hide content.” But what most alarmed the firm about this chikungunya case was that the queries asked of Claude were ostensibly seeking help in drafting a grant application for research the user evidently intended to perform at a military institute. The “combination of content and institutional association” was so troubling that Anthropic said it conducted another internal investigation, even after finding that Claude had rejected the user or users’ requests. Venomous schemes Anthropic’s case study involving the venom peptides came from a user who was apparently attempting to construct an atlas of these signaling molecules as they appear in venom toxins produced by animals in nature. The Threat Intelligence team noted that plenty of venom research has benign applications—including treatments for migraines and cancer pain relief. What alarmed them about this Claude user’s research, however, was that despite its supposed purpose of identifying potentially new painkillers and antidepressants, the researcher had asked for help building an atlas that also catalogued venom peptides capable of inducing paralysis. “[It] could, therefore, be used to generate both novel therapeutic or harmful compounds,” the team noted. Anthropic’s head of threat intelligence Jacob Klein told the Times that the subtlety of these risky queries was one of the challenges inherent to ensuring that the company’s AI tools are used responsibly: “You are not seeing someone in a comic book kind of way say, ‘Hey, I want to build a biological weapon to kill everybody,’” he said.