FILE: The OpenAI logo is displayed on a cell phone in front of an image generated by ChatGPT’s Dall-E text-to-image model, Dec. 8, 2023, in Boston.
Michael Dwyer/AP
hide title
toggle title
Michael Dwyer/AP
SAN FRANCISCO – OpenAI revealed on Friday that its artificial intelligence agents had interacted with several US government websites in unexpected ways, discovered as part of an ongoing review of unexpected behavior of the company’s models.
The AI giant’s models accessed publicly available information on two websites operated by the Securities and Exchange Commission, as well as data from the U.S. Census Bureau, the company revealed Friday. OpenAI found no use of SEC credentials, access to non-public accounts or information, changes to SEC data or systems, or evidence of a compromise or vulnerability, the company said.
The disclosure comes at a time of growing global concern about AI systems escaping human control and hacking external websites, as well as industry calls for a slowdown in AI development, which OpenAI has said it supports.
OpenAI spokesperson Liz Bourgeois said in a statement that the lab continues to conduct a review of “misaligned model activity” — when AI systems behave in undesirable ways — and is notifying organizations when it identifies potential impacts to their systems.
OpenAI CEO Sam Altman said on social media Friday that there is an “extensive and ongoing review related to our agents’ use of internet access during training and testing.”
AI evaluator and research lab Transluce said Friday that through an independent investigation it also found that agents who appeared to come from OpenAI attempted a rudimentary hack into a Department of Education website for the department’s civil rights office, which was unsuccessful.
The Department of Education’s “system operations reviews” found “no evidence of any impact on our website or databases,” a department spokesperson said Friday.
A Transluce spokesperson said that as part of its investigation, it found data on the open web that revealed new details about the activities of some OpenAI agents previously identified on US government websites and brought it to the attention of OpenAI.
Transluce found “additional rogue activity, some of which is not clearly attributable to OpenAI,” targeting other government agencies, including the Department of Justice and the Department of Commerce, as well as some state government websites in California, Maryland, Illinois, Texas and New York. The models “used sites in unwanted ways and sometimes violated explicit usage policies,” Transluce said in a statement.
OpenAI said it is reviewing Transluce’s report.
If OpenAI notifies organizations it identifies as affected by unexpected model behavior, that does not mean there was a security incident, the company said, and could identify a design issue or security weakness that affected organizations want to address.
Most of the activity OpenAI said it has reviewed so far has involved routine investigative tasks in which agents accessed public web content to answer questions, including government websites considered authoritative sources of public information.
Several companies have revealed incidents in recent months when they say their models have behaved unpredictably or hacked into other organizations’ websites or systems. OpenAI revealed in July that two of its most capable AI models were responsible for the cyberattack targeting AI startup Hugging Face.
Altman said in his social media post Friday that the Hugging Face incident “remains the most serious event we have seen.”
That incident sparked widespread panic across the industry and beyond about AI models going rogue, with several competing AI labs making similar revelations in the days and weeks that followed. OpenAI recently shared six reports of “unexpected or concerning” behavior in AI models and introduced a framework to track, probe and reveal cases of what it called misalignment.
