Anthropic said it had disrupted several potential plots this year by scientists who used its leading artificial intelligence models to conduct research that could have helped develop biological weapons.
In a report describing misuses of its A.I. models, Anthropic said it could not determine whether the research served a legitimate or nefarious purpose because valid biological inquiry — the kind that can lead to breakthroughs like vaccines — can also help engineer dangerous pathogens. In the face of that uncertainty, Anthropic said it erred on the side of caution because the consequences of missing malicious activity could be severe.
“You are not seeing someone in a comic book kind of way say, ‘Hey, I want to build a biological weapon to kill everybody,’” Jacob Klein, the head of threat intelligence at Anthropic, said in an interview. “It’s an incredibly nuanced situation.”
The potential for cutting-edge A.I. models to facilitate the development of known or entirely new biological pathogens is among the gravest concerns experts have about a technology that is developing so rapidly that even its leading architects doubt whether humans will be able to fully control it.
Compared with the threat of catastrophic cyberattacks or A.I. agents that fail to align with human intentions, biological misuse has gained less attention as an existential risk of A.I., in part because past examples have generally been shown only in research settings rather than in the real world.
Andrew Weber, a senior fellow on the Council on Strategic Risks who reviewed Anthropic’s report before its release, said the findings were “chilling examples of state-sponsored biological weapons developers tapping into the rapidly advancing capabilities” of leading A.I. models.
The lengthy report that Anthropic published on Thursday cataloged a litany of misuses on various models of its Claude A.I. chatbot over the past eight months. Some examples were similar to past disclosures from Anthropic and other A.I. labs, including suspected Chinese and Iranian government-linked actors targeting dissident and diaspora communities for surveillance.
The report also highlighted cases of Russian state media using Claude to generate online propaganda masquerading as independent reporting, including fabricated claims about an election in Moldova.
Anthropic documented another genre of abuse it said was new: attempts to use Claude to develop software for conventional weapons design and development, including firearms, missiles, armed drones and bombs. It detailed three cases in China, two in Russia and one in Yemen.
The report does not identify by name which parties were involved in the Yemeni case, but the context makes clear that it is referring to the Iran-backed Houthi militia. A.I. use by terrorist networks is a growing concern among U.S. security officials.
But among all the categories of threats shared in the report, none may be as worrisome as the biological research cases. Anthropic did not disclose the names of researchers or institutions that it blocked, their countries of affiliation or the specific biological agents at issue, in part because of its uncertainty about their aims.
Still, Anthropic said the scientists circumvented its controls intended to prevent users from blocked regions from gaining access to its A.I. models and worked to “obfuscate the purpose of their research to evade our safeguards.” The company banned accounts associated with the research.
In one example from May, a scientist sought Claude’s help with writing a grant application for funding to conduct research intended to experiment on the chikungunya virus, a mosquito-borne malady that can lead to months of severe pain and other symptoms. Such research, known as gain of function, can be legitimate and lead to vaccine development, but it can also create superbug versions of viruses. In this instance, the scientist wanted to engineer mutations to the virus that would make it more harmful as it repeatedly infected live animals.
Anthropic said it believed the particular research was worrisome in part because it could discern that the work was intended to be performed at a military research institute.
“What we don’t know is if the research was meant to be weaponized,” Mr. Klein said. “But a military institution doing gain-of-function research is concerning.”
Anthropic said evaluations from last year of its older models demonstrated that they were not yet able to meaningfully assist in conducting dangerous biological research. Its current models are more able to complete complex scientific research, the company said, which spurred tighter safeguards intended to restrict access to a wide range of dual-use biological research queries.
Susan Monarez, a microbiologist and public health expert who oversaw a review of national biosecurity preparedness in the Obama administration, also reviewed the Anthropic report before its publication. She said the findings provided real-world evidence to support concerns that bad actors were trying to covertly use advanced A.I. “to improve their chances of building biological pathogens that could cause significant harm.”
Dr. Monarez, who briefly served as the director of the Centers for Disease Control and Prevention last year, said A.I. held enormous promise to usher in an era of medical breakthroughs that save and extend lives. But the same abilities that make that possible, she added, could also “let bad actors hide in plain sight, using seemingly legitimate research to create pathogens we may not see coming and may not be able to stop once released.”
Mr. Weber of the Council on Strategic Risks, who also served as the assistant secretary of defense for nuclear, chemical and biological defense programs during the Obama administration, called for limiting access to A.I. tools capable of this level of biological research to trusted researchers only.
“The fact that Russia, China and North Korea continue to develop prohibited biological weapons makes it imperative that we deny their researchers access to these extraordinarily capable models,” he said.
- Credits: The New York Times
- Author: Dustin Volz
- Photo: Marissa Leshnov
