AI Use Rising in Online Misinformation Campaigns, but Impact Limited Till Now, Say Researchers

Researchers found "numerous instances" since 2019 in which AI-generated content had been used in politically-motivated online influence campaigns.

By Reuters | Updated: 18 August 2023 10:58 IST

AI Use Rising in Online Misinformation Campaigns, but Impact Limited Till Now, Say Researchers

Photo Credit: Reuters

Generative AI would enable groups with limited resources to produce higher quality content

Highlights

AI-generated content had been used in online influence campaigns
These included campaigns from groups aligned with the governments
Security officials have warned of such models being used by cybercriminal

Google-owned US cybersecurity firm Mandiant said on Thursday it had seen increasing use of artificial intelligence (AI) to conduct manipulative information campaigns online in recent years, though the technology's use in other digital intrusions had been limited so far.

Researchers at the Virginia-based company found "numerous instances" since 2019 in which AI-generated content, such as fabricated profile pictures, had been used in politically-motivated online influence campaigns.

These included campaigns from groups aligned with the governments of Russia, China, Iran, Ethiopia, Indonesia, Cuba, Argentina, Mexico, Ecuador, and El Salvador, the report said.

[Sponsored] Samsung Galaxy S24: The Ultimate Camera Phone With Smarter On-Device AI

It comes amid a recent boom in generative AI models such as ChatGPT, which make it far easier to create convincing fake videos, images, text, and computer code. Security officials have warned of such models being used by cybercriminals.

Generative AI would enable groups with limited resources to produce higher quality content for influence campaigns at scale, Mandiant researchers said.

A pro-China information campaign named Dragonbridge, for instance, had expanded "exponentially" across 30 social platforms and 10 different languages since it first began by targetting pro-democracy protesters in Hong Kong in 2019, said Sandra Joyce, vice president at Mandiant Intelligence.

Opera Brings Aria AI to Its iPhone, iPad Browser: Here's What It Can Do

Yet, the impact of such campaigns was limited. "From an effectiveness standpoint, not a lot of wins there," she said. "They really haven't changed the course of the threat landscape just yet."

China has denied U.S. accusations of involvement in such influence campaigns in the past.

Mandiant, which helps public and private organisations respond to digital breaches, said it hadn't yet seen AI play a key role in threats from Russia, Iran, China, or North Korea. AI use for digital intrusions is expected to remain low in the near term, the researchers said.

How AI Is Changing Datacentres, Role of CPUs vs GPUs: Interview With Intel’s Sandra Rivera

"Thus far, we haven't seen a single incident response where AI played a role," said Joyce. "They haven't really been brought into any kind of practical usage that outweighs what could be done with normal tooling that we've seen."

But she added: "We can be very confident that this is going to be a problem that gets bigger over time."

Is the Samsung Galaxy Z Flip 5 the best foldable phone you can buy in India right now? We discuss the company's new clamshell-style foldable handset on the latest episode of Orbital, the Gadgets 360 podcast. Orbital is available on Spotify, Gaana, JioSaavn, Google Podcasts, Apple Podcasts, Amazon Music and wherever you get your podcasts.

Affiliate links may be automatically generated - see our ethics statement for details.

Comments

For the latest tech news and reviews, follow Gadgets 360 on X, Facebook, WhatsApp, Threads and Google News. For the latest videos on gadgets and tech, subscribe to our YouTube channel. If you want to know everything about top influencers, follow our in-house Who'sThat360 on Instagram and YouTube.