OpenAI says Russian agents used its AI chatbot to update their bosses, shedding light on influence campaigns not previously attributed to Russia.
Jeff Chiu/AP
hide title
Jeff Chiu/AP
OpenAI said in a blog post on Thursday that it banned Russian and Iranian accounts that used the company’s AI tools to help with influence operations in multiple countries. The agents said they managed to get their false narratives picked up by the media and, in one case, provoked a response from a member of the European Parliament, according to internal reports the operations produced using OpenAI’s chatbot ChatGPT.
OpenAI investigated the impact of different campaigns that mentioned the accounts and found that those that focused on placing false narratives in established media outlets appeared to have reached a broader audience than campaigns that focused on using fake social media accounts to spread their message.
The Russian operations targeted Latin American countries and appeared aimed at undermining support for Ukraine and influencing local politics, OpenAI said. The accounts appeared to primarily be using ChatGPT to write reports detailing their work to unspecified superiors, OpenAI said. The company said one of these reports appeared to describe the most far-reaching influence operation OpenAI has encountered so far, because it generated a public response from a political figure.
In this campaign, Russian agents said they posed as a regional educational authority in Peru and tricked some schools into organizing activities about Ukraine in which Stepan Bandera appeared. Bandera is a controversial figure, admired by nationalists for leading a campaign for Ukrainian independence before and during World War II. He is vilified by others, even in Poland, because his movement sided with the Nazis and killed Poles and Jews. The Peruvian media Extra reported that one of those events took place at a Peruvian school.
A Polish news report on the school event led a far-right Polish member of the European Parliament to suggest banning the entry of people who professed support for Bandera.
“Russian counterfeiting therefore fueled and fueled historical tensions between Ukrainians and Poles,” OpenAI wrote in its report.
“The Russians are constantly engaging in these kinds of active measures,” said Darren Linvill, co-director of the Watt Family Innovation Center Media Forensics Hub at Clemson University, which studies online influence operations. “Attempting to engage in narrative laundering, creating false stories and false characters to create or spread propaganda that aligns with Russia’s worldview and political agenda.”
Internal Russian memos created with ChatGPT also claimed that agents ran a fake think tank in Latin America and recruited unwitting researchers to interview experts and write reports. OpenAI said that effort was similar to a 2020 Russian operation posing as an independent news outlet linked to late oligarch Yevgeny Prigozhin’s Internet Research Agency, which was involved in efforts to influence the 2016 US election.
In other cases, OpenAI said, Russian agents took credit for events for which they were not responsible. The Russians used AI to generate fake material, OpenAI said, but it was a minority of the workload.
“These Russian subordinates were trying to use AI to create a shortcut and do less work in the same way that many of us use AI, but in this case it helped get them caught,” Linvill said.
While the Russian agents mainly used AI to update their superiors, the Iranian agents used AI to help them publish content in Persian and English. The campaign focused primarily on the US-Iran war and, to a lesser extent, American politics. OpenAI said the operation’s activity “appeared consistent with a commercial actor running a paid influence campaign” but that it was unable to identify who it was.
According to OpenAI, the Iranian accounts created seven fake authors to submit and polish long-form articles for small media outlets around the world. They published almost 100 articles this way.
The fake authors published articles in outlets such as the US progressive news site Daily Kos and the UK-based Middle East Monitor. According to NBC News, the Middle East Monitor published at least 21 articles by four fake authors. The outlet, which did not respond to a request for comment, has since removed the author’s pages.
The Daily Kos wrote in response to NPR’s request for comment that its community section, where one of the fake personas posted, is separate from staff reports. Anyone can register to post in the community section and posts are “not subject to review by Daily Kos staff prior to publication,” according to a disclaimer on the site. The community section has rules for posting, including a ban on “pretending to be someone you’re not.” The person’s posts are still published.
“AI was valuable to them beyond just being a shortcut,” Linvill said. “It helped them create language that… looked, felt and read authentic to the point of being published through legitimate media. And I think that’s something that a few years ago, an Iranian operation like this would have struggled with a lot more.”
