OpenAI’s artificial intelligence went rogue this year in at least four additional incidents, hacking and trying to break into government and university websites without being instructed to do so, according to researchers and government officials.
The attacks took place in May and June, before OpenAI’s technology breached the A.I. start-up Hugging Face in July and set off a global debate about A.I. safety.
Unlike the Hugging Face attack and other incidents in which A.I. systems were told to complete cybersecurity tests that effectively invited the models to demonstrate their hacking skills, the new incidents occurred when A.I. systems were directed to perform relatively mundane data collection, researchers said. When OpenAI’s systems struggled to gather data from websites, they resorted to hacking techniques to get the information.
Three of the incidents were identified by Transluce, a research lab focused on A.I. oversight, and all were confirmed by OpenAI. Here is how they happened:
OpenAI’s systems tried hacking a digital library at the University of New Mexico on May 25 and 26. The A.I. did not appear to succeed.
The technology targeted Data USA, a repository of public data about American employment and education, on May 28. This attempt also appeared to be unsuccessful, researchers said.
On June 18, OpenAI’s A.I. hacked an Australian government website, the Medicare Statistics Reporting Service, and acquired health data. Australia’s prime minister, Anthony Albanese, disclosed the episode on Wednesday.
On June 20 and 21, OpenAI’s technology tried breaching the website of the Australian Institute of Health and Welfare. No private information was obtained, Australian officials said.
The incidents added to a spate of breaches in which A.I. from OpenAI, Anthropic, Meta and Google has broken into other systems without human knowledge. The events have intensified a debate over whether A.I. development needs to be slowed to address the technology’s potential dangers.
Dario Amodei, the chief executive of Anthropic, has called for A.I. companies and governments to work together before the technology becomes too powerful for human control. But other executives, such as Jensen Huang, chief executive of the chipmaker Nvidia, have said such doomsday scenarios are overwrought. President Trump has said he does not believe A.I. needs to be heavily regulated.
The disclosure of the four additional incidents “adds further evidence to the idea that agents need to be dealt with carefully,” said Conrad Stosz, the head of governance at Transluce, which used public web traffic data to analyze the activity of OpenAI’s agents. Agents are autonomous programs that work to execute tasks for a user.
Mr. Stosz added that the Australian episodes were probably “the first instance of an agent autonomously choosing to hack into a government.”
An OpenAI spokeswoman said on Wednesday that the company had reached out to the University of New Mexico and DataUSA and had been in communication with the Australian government about the incidents.
“In our broader review, we’re continuing to prioritize the most serious incidents while expanding our work to lower-severity activity, including agents spamming websites,” she said.
She separately added that the San Francisco company had uncovered the Australia incidents during an “extensive review” of its A.I. models and found that “our models took actions we did not intend.” OpenAI’s review will take months, she said.
Sam Altman, the chief executive of OpenAI, said on social media this month that safety should be more important than enhancing A.I.’s abilities and that, without guardrails, society could “lose control of the future to A.I.”
Mr. Albanese said he spoke to Mr. Altman on Wednesday and expressed “extreme concern” about the hack. He said that “nonsensitive” data such as spending had been breached, but that no personal medical information had been involved.
(The New York Times has sued OpenAI and Microsoft, claiming copyright infringement of news content related to A.I. systems. The two companies have denied those claims.)
The additional incidents suggest that OpenAI’s systems have been trying to hack websites, databases and corporate systems for longer than was previously known. Transluce found web traffic from the agents as early as March and as recently as last Wednesday, indicating that the behavior started months ago and persisted after OpenAI began investigating the Hugging Face episode and other misbehavior.
In the incidents in May and June, the company’s A.I. systems appeared to be involved in data retrieval trainings, the researchers said.
For the attempt on the University of New Mexico library, the A.I. tried to gain access to photos of a historic tuberculosis treatment center. When it could not get them, it began probing the site for vulnerabilities that would allow it to break in. After not finding any holes, the A.I. sent what it described as a “flood” of 80 requests to the university’s server.
In its targeting of Data USA, the A.I. sent a jumbled query to the site for data. When that failed, the A.I. sent 12 probes for various vulnerabilities, but failed to find one.
“If you were to train a swarm of agents to accomplish some generic task and those agents are willing to resort to hacking, anyone who happens to have that information might be at risk,” said Mr. Stosz of Transluce.
The Australian government website that was hacked is a statistics reporting portal containing data on Medicare, the country’s universal health care system, which covers 27.5 million enrollees in addition to international visitors. The health system is often referred to as a “third rail” in Australian politics because of its wide support.
In the incidents in May and June, the company’s A.I. systems appeared to be involved in data retrieval trainings, the researchers said.
For the attempt on the University of New Mexico library, the A.I. tried to gain access to photos of a historic tuberculosis treatment center. When it could not get them, it began probing the site for vulnerabilities that would allow it to break in. After not finding any holes, the A.I. sent what it described as a “flood” of 80 requests to the university’s server.
In its targeting of Data USA, the A.I. sent a jumbled query to the site for data. When that failed, the A.I. sent 12 probes for various vulnerabilities, but failed to find one.
“If you were to train a swarm of agents to accomplish some generic task and those agents are willing to resort to hacking, anyone who happens to have that information might be at risk,” said Mr. Stosz of Transluce.
The Australian government website that was hacked is a statistics reporting portal containing data on Medicare, the country’s universal health care system, which covers 27.5 million enrollees in addition to international visitors. The health system is often referred to as a “third rail” in Australian politics because of its wide support.
- Credits: The New York Times
- Authors: Kate Conger and Victoria Kim
- Photo: Lucas Foglia
