OpenAI Exposes “New Variety of Prompt Injection” that Can Spread Like Computer Worms
OpenAI said the attacks stayed inside simulated tests and used GPT-Red training to probe whether models could repeat malicious prompts on their own.
- Earlier, OpenAI released a report sharing evidence of a new variety of prompt injection that can self-propagate like a computer worm, discovering instances of an "AI-version of a worm attack" back in June.
- OpenAI describes these "self-replicating prompt injection" attacks as having a two-pronged objective: achieving a malicious goal and inducing models to reproduce the injection publicly, similar to traditional computer worms.
- In one example, an email injection forces an agent to copy the payload into outgoing messages, while another test involved a Slack agent that sent "Froges" to a recipient and reposted the injected message.
- To address this threat, OpenAI said it uses its automated red-teaming agent, GPT-Red, to train future models on self-reproduction, making them more robust against similar vulnerabilities.
- According to OpenAI, there is no indication these attacks occurred outside training environments, and researchers observed no impact beyond simulated tool calls while sharing findings due to their "novel nature.
15 Articles
15 Articles
Self-Replicating Prompt Injections: The New AI Security Threat Spreading Like Digital Viruses
Self-replicating prompt injections represent a new AI security threat where malicious instructions autonomously spread across connected models, hijacking outputs to propagate without further human input. This turns traditional prompt attacks into self-sustaining digital viruses with potentially severe consequences. Researchers warn of widespread risks in enterprise systems.
The Register
IT Security News - cybersecurity, infosecurity newsAdd one more AI worry to the nightmare scenario: self-replicating prompt injections
It's a worm attack, AI-style
OpenAI discovered a new type of attack on I.E. agents: the harmful instruction forces the model not only to execute the attacker's command, but also to transfer the prompto-injection itself to other messages or files, so that it can spread further.
OpenAI discovers how to turn AI into a 'worm' - malicious prompts can replicate themselves
Open artificial intelligence Found Their models are vulnerable to artificial intelligence-led worm-type attacks—the name the company gave them “Self-replicating injection in queries”. The developers assure that this injection could theoretically cause the virus to spread on a large scale, but in practice this attack has not yet been used. Image credit: Brecht Corbeel / unsplash.com […]
OpenAI has discovered a 'self-replicating prompt injection' risk threatening AI models. Here are the details of this new security threat and the measures taken. The article "OpenAI Reveals Self-Replying Prompt Injection Risk" first appeared on TechInside.
Coverage Details
Bias Distribution
- 100% of the sources are Center
Factuality
To view factuality data please Upgrade to Premium
