9.4 · Looped input

Explore 3 source-supported AI security cases for 9.4 · Looped input in Databricks DASF 3.0, with evidence, scope, and defensive analysis.

Risk category

An AI system's output returns to the environment and is later ingested again as input, creating a harmful feedback loop. Errors, biases, or weakened safeguards can be reinforced each time the material cycles through the system. The model or application may consequently become increasingly inaccurate or behave in unintended ways.

Official framework source ↗ · Open interactive explorer ↗

Exercise ·

Morris II Worm: RAG-Based Attack

Morris II is a constructed research demonstration in which a RAG email assistant retrieves a malicious message, copies its instructions into a generated reply and includes sensitive correspondence data. The automatic email workflow then sends that response onward. The graph separates source delivery, corpus storage, retrieved instruction following, generated content and the two security effects of sending it; the branches do not imply two independently observed emails or a real-world victim campaign.

Why it relates: Researchers developed Morris II, a zero-click worm designed to attack generative AI (GenAI) ecosystems and propagate between connected GenAI systems. The relevant category is Looped input: An AI system's output returns to the environment and is later ingested again as input, creating a harmful feedback loop.

Researchers developed Morris II, a zero-click worm designed to attack generative AI (GenAI) ecosystems and propagate between connected GenAI systems. The worm uses an adversarial self-replicating prompt which uses prompt injection to replicate the prompt as output and perform malicious activity. The researchers demonstrate how this worm can propagate through an email system with a RAG-based assistant. They use a target system that automatically ingests received emails, retrieves past correspondences, and generates a reply for the user. To carry out the attack, they send a malicious email containing the adversarial self-replicating prompt, which ends up in the RAG database. The malicious instructions in the prompt tell the assistant to include sensitive user data in the response. Future requests to the email assistant may retrieve the malicious email. This leads to propagation of the worm due to the self-replicating portion of the prompt, as well as leaking private information due to the malicious instructions.

SecureFlow · Read the full case ↗

Validated Research ·

Copilot for Word Turned a Hidden Prompt into a Document-Borne AI Worm

A white-on-white prompt inside a Word document survived formatting removal, silently changed financial figures, and copied itself into Copilot's newly generated document. The output could then carry the prompt into a later Copilot workflow. This is not autonomous network propagation: every hop still depends on a user or retrieval process bringing the carrier document back into drafting or editing.

Why it relates: Output containing the copied payload can be reintroduced as input in a later Copilot operation.

Document-source instructions alter financial output and survive into a new carrier. Conditional document reuse is preserved; automatic network spread is not claimed.

AIA · Read the full case ↗ · SecureFlow · Read the full case ↗

Validated Research ·

ZombieAgent Shows How Connector Prompt Injection Can Persist and Spread

Radware showed how hidden instructions in email or documents read through ChatGPT connectors could turn a task into data exfiltration. Fixed URLs encoded stolen text one character per request, Memory preserved instructions across chats, and a contact-harvesting branch spread the payload; Radware says OpenAI fixed it on December 16, 2025. Workspace administrators can disable unnecessary connectors or Memory and narrow or revoke grants, while platform implementers own raw-content isolation, Memory gates, and default-deny egress; value-level sink checks cover only explicit transfers retaining server-side content IDs.

Why it relates: ZombieAgent is a proof-of-concept indirect prompt injection attack demonstrated by Radware against OpenAI's ChatGPT Deep Research and Connector functionality. The relevant category is Looped input: An AI system's output returns to the environment and is later ingested again as input, creating a harmful feedback loop.

ZombieAgent is a proof-of-concept indirect prompt injection attack demonstrated by Radware against OpenAI's ChatGPT Deep Research and Connector functionality. The attacks showed how instructions concealed in externally controlled content, such as emails and documents, could be ingested and executed by ChatGPT during normal user activity. The demonstration used Gmail as the injection source, but any ChatGPT Connector such as Outlook, Google Drive, Jira, or Teams could be similarly abused. The Radware Security Researchers sent a malicious email containing concealed instructions to a Gmail inbox connected to ChatGPT. When the user later asked ChatGPT to perform an ordinary inbox-related task, ChatGPT retrieved the email and executed its instructions. The user did not open, click, or knowingly interact with the malicious email. The injected instructions caused ChatGPT to collect information from connected services and exfiltrate it through URL requests. OpenAI had introduced a control preventing ChatGPT from dynamically constructing or modifying URLs which could be used to exfiltrate data via query parameters. The researchers bypassed this control by supplying an indexed dictionary of preconstructed static URLs and instructing ChatGPT to open URLs corresponding to individual characters to exfiltrate the collected data. The researchers also showed that the malicious instructions could manipulate ChatGPT's Memory. The injected memories instructed ChatGPT to retain sensitive information from future conversations and to retrieve and execute a designated attacker-controlled email during later interactions. This created a persistent mechanism for repeated collection and exfiltration across chat sessions. The researchers also demonstrated how the malicious prompt could be propagated. A compromised agent could collect email addresses from the victim's mailbox and use its connected email capabilities to send additional messages containing the malicious prompt to those contacts. Recipients whose AI agents later processed the poisoned messages could become additional victims.

AIA · Read the full case ↗ · SecureFlow · Read the full case ↗