13.1 · Memory Poisoning

Explore 15 source-supported AI security cases for 13.1 · Memory Poisoning in Databricks DASF 3.0, with evidence, scope, and defensive analysis.

Risk category

An attacker inserts false or malicious information into an AI agent's short-term or long-term memory. When reused as context, that information can change later decisions and lead to unauthorized operations. The risk persists because stored material may influence tasks beyond the interaction in which it was introduced.

Official framework source ↗ · Open interactive explorer ↗

Incident · · Practical overlap

Flowise Honeypot: Unauthenticated API Abuse, Beacon Setup, and Agent Probing

Sekoia recorded six phases of real attacker traffic against an internet-facing decoy that emulated Flowise 2.2.8, a platform for building AI workflows. The exposed management API let the attacker retrieve planted credentials and add a workflow configured to call back. The attacker then tried to override prompts, probe session memory and ask the simulated agent about real third-party infrastructure. The decoy returned simulated responses, so the requests do not establish real command execution, database access or a Nightingale compromise.

Why it relates: The caller attempted to store attacker-controlled content in the emulated session state. A real attacker targeted the relevant API or tool boundary on an emulator. Replies were simulated, so this does not establish production credentials, model behavior, command execution, external callbacks or compromise of the named system.

Treat real attacker activity separately from effects emulated by the honeypot.

AIA · Read the full case ↗ · SecureFlow · Read the full case ↗

Validated Research ·

GitHub Copilot CLI: Phished Device Code Authorization Enables Session Steering and Memory Poisoning

GitHub Copilot CLI is a coding assistant that runs in a terminal. Microsoft showed in a lab that a developer could be tricked into approving an attacker's device sign-in on the real GitHub site. GitHub then issues an OAuth access token to the initiating client. Depending on the permissions and enabled features, the token could steer an active local session, plant misleading Copilot Memory, read cloud-synced conversations, or access private repositories. Each outcome has its own prerequisites.

Why it relates: Misleading Copilot Memory entries can persist and be retrieved as future project context.

Read each branch against its own feature and authority prerequisites. The simulation validated effects in the configured lab.

AIA · Read the full case ↗ · SecureFlow · Read the full case ↗

Validated Research ·

Prompt-Based Attacks Against Gemini via Calendar Invitations

SafeBreach demonstrated indirect prompt injection through Calendar invitations and email in Gemini web and Android applications. Retrieved instructions could immediately alter replies or invoke Calendar actions. Other routes retained instructions in conversation history and used a later victim interaction to invoke Google Home, open a website, or launch Zoom. In the Gmail exfiltration route, the initial reply first assembled a source URL containing another email subject; a later interaction opened that existing URL. These are distinct controlled demonstrations, not a single path requiring every impact or a confirmed malicious campaign.

Why it relates: Retained conversation state carried instructions into later turns.

Calendar/email retrieval and immediate versus delayed tool routes from Invitation Is All You Need.

SecureFlow · Read the full case ↗

Validated Research ·

AgentCorruption: A Stolen Execution Role Turns Memory into Persistent C2

Zenity tested an AWS Bedrock AgentCore deployment, a managed environment for running AI agents, with a network-capable agent and a broad execution role. The agent relayed temporary role credentials from its metadata endpoint. Off-platform API calls then reached other memories and agents. The distinctive persistence path wrote short-term conversation events that configured extraction strategies could turn into long-term records; later retrieval directed the agent to a mutable command page. The reach was demonstrated within an account and region, not as a microVM escape or automatic cross-account compromise. The researchers observed major default-role restrictions on September 29; existing deployed roles still require independent inspection.

Why it relates: Forged STM events become persistent instructions if the target extraction and retrieval paths select them.

Controlled broad-role AgentCore deployment with separate direct API and retained-memory/C2 branches; account/region scope, tool prerequisites and later default-role restrictions remain explicit.

AIA · Read the full case ↗ · SecureFlow · Read the full case ↗

Incident ·

AI Recommendation Poisoning: Summary Links That Try to Rewrite Assistant Memory

Microsoft found companies using AI summary links to plant promotional instructions in assistants with persistent memory, a feature that retains preferences across conversations. If accepted and later recalled, those instructions can influence recommendations long after the original page is closed. The defensive priority is to keep website-authored requests from silently becoming trusted user preferences.

Why it relates: Microsoft reported real-world attempts to influence AI recommendations through links presented as summarization tools. The relevant category is Memory Poisoning: An attacker inserts false or malicious information into an AI agent's short-term or long-term memory. Directly submitted link text may be accepted into durable memory and bias later recommendations, conditional on successful memory write and recall.

Microsoft reported real-world attempts to influence AI recommendations through links presented as summarization tools. Website operators placed promotional instructions in URL prompt parameters that open an AI assistant. If the assistant accepts those instructions as durable memory, later answers can favor the operator's brand or website. The report identifies more than 50 unique prompts associated with 31 companies across 14 industries; these are attempts, not a count of successful compromises. Effectiveness varies by platform and over time. Its fictional brands and financial-loss examples illustrate possible consequences. The six-step flow follows ATLAS's ordered procedures; the last two steps depend on successful memory acceptance and subsequent recall.

AIA · Read the full case ↗ · SecureFlow · Read the full case ↗

Research ·

SkillJack Turns Poisoned Agent Experience Into a Reusable Skill

The SkillJack paper evaluates SkillX and Anything2Skill, which transform agent experience trajectories into reusable skills. In a synthetic AppWorld study using one DeepSeek-v4-flash API, poisoned trajectories survived the transformation and evaded the evaluated routing-level detectors, with reported persistence after source records were deleted. This is a controlled research result, not a demonstrated compromise of a production skill registry or external service.

Why it relates: The SkillJack paper evaluates SkillX and Anything2Skill, which transform agent experience trajectories into reusable skills. The relevant category is Memory Poisoning: An attacker inserts false or malicious information into an AI agent's short-term or long-term memory.

The SkillJack paper evaluates SkillX and Anything2Skill, which transform agent experience trajectories into reusable skills. In a synthetic AppWorld study using one DeepSeek-v4-flash API, poisoned trajectories survived the transformation and evaded the evaluated routing-level detectors, with reported persistence after source records were deleted. The work is a controlled research result, not a demonstrated compromise of a real organization's skill registry or external service.

AIA · Read the full case ↗ · SecureFlow · Read the full case ↗

Validated Research ·

CoSnitch Combined Copilot Autorun Exfiltration with a Separate Memory-Poisoning Path

CoSnitch covered two independent Copilot attacks. A crafted copilot.microsoft.com link combined ?q= with an undocumented autorun=1 parameter, causing an authenticated session to run an embedded prompt without confirmation and send connected data through URL fetches. Separately, hidden instructions in a webpage summarized by Copilot invoked memory.add and persisted in long-term memory. Microsoft fixed CVE-2026-24301 on August 18.

Why it relates: When Copilot summarized an attacker-controlled webpage, hidden instructions were parsed as a higher-authority directive and invoked memory.add with attacker-controlled content. The relevant category is Memory Poisoning: An attacker inserts false or malicious information into an AI agent's short-term or long-term memory.

When Copilot summarized an attacker-controlled webpage, hidden instructions were parsed as a higher-authority directive and invoked memory.add with attacker-controlled content. The resulting long-term memory survived password changes, session revocation, and device re-enrollment until explicit removal, and could be recalled into later conversations. Varonis states that Microsoft shipped patches on August 18, 2026. A crafted copilot.microsoft.com link combined attacker instructions in q with the undocumented autorun=1 parameter. In an authenticated browser session, Copilot submitted the prompt without a separate confirmation, retrieved data from connected services or prior conversations, encoded selected values into an attacker URL, and fetched it. Microsoft shipped patches on August 18, 2026.

AIA · Read the full case ↗ · SecureFlow · Read the full case ↗ · SecureFlow · Read the full case ↗

Research ·

MCFA Turns Agent Memory into a Delayed Control-Flow Input

Memory Control Flow Attacks (MCFA) do not need direct access to an agent's memory database or system prompt. The research shows how ordinary interaction can place an action-oriented preference into long-term memory, then a later benign task retrieves it and changes tool choice or order. Across controlled LangChain and LlamaIndex experiments, tool-selection attacks were highly reliable, while ordering attacks were weaker. Memory must therefore be governed as persistent control input, not trusted conversation history.

Why it relates: Memory Control Flow Attacks (MCFA) do not need direct access to an agent's memory database or system prompt. The relevant category is Memory Poisoning: An attacker inserts false or malicious information into an AI agent's short-term or long-term memory.

Memory Control Flow Attacks (MCFA) do not need direct access to an agent's memory database or system prompt. The research shows how ordinary interaction can place an action-oriented preference into long-term memory, then a later benign task retrieves it and changes tool choice or order. Across controlled LangChain and LlamaIndex experiments, tool-selection attacks were highly reliable, while ordering attacks were weaker. Memory must therefore be governed as persistent control input, not trusted conversation history. The attacker did not need memory-database access. One or a few ordinary interactions caused the agent to store an action-oriented preference. A later benign task retrieved it and changed tool choice or order, while the trusted tools and system prompt remained unchanged. The study measured related but distinct outcomes. Override selected a risky tool and reached 97.2% to 100% success. Order violated a required sequence and reached 52.8% to 69.4%. Cross-task and more-than-30-round tests showed that the effect could persist, while matched benign-memory controls remained at 0%. Simple mitigations were incomplete. Adding 100 benign records did not remove the effect. Summarization reduced malicious-write success, but every summarized instruction that survived still steered the later task. Role-based memory segregation reduced some Override rates, yet residual success ranged from 2.8% to 100%. This was controlled research. The evaluation used synthetic safe and risky tool variants, no production users, and no harmful external actions. It measured tool-trace deviation, not real damage. There is currently no public evidence that MCFA has been used in an actual attack.

AIA · Read the full case ↗

Validated Research ·

ZombieAgent Shows How Connector Prompt Injection Can Persist and Spread

Radware showed how hidden instructions in email or documents read through ChatGPT connectors could turn a task into data exfiltration. Fixed URLs encoded stolen text one character per request, Memory preserved instructions across chats, and a contact-harvesting branch spread the payload; Radware says OpenAI fixed it on December 16, 2025. Workspace administrators can disable unnecessary connectors or Memory and narrow or revoke grants, while platform implementers own raw-content isolation, Memory gates, and default-deny egress; value-level sink checks cover only explicit transfers retaining server-side content IDs.

Why it relates: Connector content drives private-data URL requests, persistent memory and conditional email-based propagation to other agents. The demonstrated memory write retains instructions to collect future conversation data and retrieve the attacker’s designated email in later interactions.

ZombieAgent is a proof-of-concept indirect prompt injection attack demonstrated by Radware against OpenAI's ChatGPT Deep Research and Connector functionality. The attacks showed how instructions concealed in externally controlled content, such as emails and documents, could be ingested and executed by ChatGPT during normal user activity. The demonstration used Gmail as the injection source, but any ChatGPT Connector such as Outlook, Google Drive, Jira, or Teams could be similarly abused. The Radware Security Researchers sent a malicious email containing concealed instructions to a Gmail inbox connected to ChatGPT. When the user later asked ChatGPT to perform an ordinary inbox-related task, ChatGPT retrieved the email and executed its instructions. The user did not open, click, or knowingly interact with the malicious email. The injected instructions caused ChatGPT to collect information from connected services and exfiltrate it through URL requests. OpenAI had introduced a control preventing ChatGPT from dynamically constructing or modifying URLs which could be used to exfiltrate data via query parameters. The researchers bypassed this control by supplying an indexed dictionary of preconstructed static URLs and instructing ChatGPT to open URLs corresponding to individual characters to exfiltrate the collected data. The researchers also showed that the malicious instructions could manipulate ChatGPT's Memory. The injected memories instructed ChatGPT to retain sensitive information from future conversations and to retrieve and execute a designated attacker-controlled email during later interactions. This created a persistent mechanism for repeated collection and exfiltration across chat sessions. The researchers also demonstrated how the malicious prompt could be propagated. A compromised agent could collect email addresses from the victim's mailbox and use its connected email capabilities to send additional messages containing the malicious prompt to those contacts. Recipients whose AI agents later processed the poisoned messages could become additional victims.

AIA · Read the full case ↗ · SecureFlow · Read the full case ↗

Validated Research ·

Hacking ChatGPT's Memories with Prompt Injection

Embrace the Red (https://embracethered.com/blog/) demonstrated that ChatGPT's memory feature is vulnerable to manipulation via prompt injections. To execute the attack, the researcher hid a prompt injection in a shared Google Doc. When a user references the document, its contents is placed into ChatGPT's context via the Connected App feature, and the prompt is executed, poisoning the memory with false facts. The researcher demonstrated that these injected memories persist across chat sessions. Additionally, since the prompt injection payload is introduced through shared resources, this leaves others vulnerable to the same attack and maintains persistence on the system.

Why it relates: Embrace the Red (https://embracethered.com/blog/) demonstrated that ChatGPT's memory feature is vulnerable to manipulation via prompt injections. The relevant category is Memory Poisoning: An attacker inserts false or malicious information into an AI agent's short-term or long-term memory.

Embrace the Red (https://embracethered.com/blog/) demonstrated that ChatGPT's memory feature is vulnerable to manipulation via prompt injections. To execute the attack, the researcher hid a prompt injection in a shared Google Doc. When a user references the document, its contents is placed into ChatGPT's context via the Connected App feature, and the prompt is executed, poisoning the memory with false facts. The researcher demonstrated that these injected memories persist across chat sessions. Additionally, since the prompt injection payload is introduced through shared resources, this leaves others vulnerable to the same attack and maintains persistence on the system.

SecureFlow · Read the full case ↗

Validated Research ·

Planting Instructions for Delayed Automatic AI Agent Tool Invocation

Embrace the Red (https://embracethered.com/blog/) demonstrated that Google Gemini is susceptible to automated tool invocation by delaying the execution to the next conversation turn. This bypasses a security control that restricts Gemini from invoking tools that can access sensitive user information in the same conversation turn that untrusted data enters context.

Why it relates: Retained context bypasses the same-turn restriction on sensitive tool invocation; the report does not prove every possible private-data effect. The case retains attacker instructions in short-term conversational memory and activates them on a later benign interaction. Long-term storage across sessions is not required for the documented short-term-memory path.

Embrace the Red (https://embracethered.com/blog/) demonstrated that Google Gemini is susceptible to automated tool invocation by delaying the execution to the next conversation turn. This bypasses a security control that restricts Gemini from invoking tools that can access sensitive user information in the same conversation turn that untrusted data enters context.

SecureFlow · Read the full case ↗

Validated Research ·

Prompt Mines Show Why CRM Agents Need Write-Action Gates

Zenity's Prompt Mines research shows how attacker-created Salesforce (a CRM platform) cases can sit quietly in Einstein (Salesforce's AI assistant layer for CRM workflows) context until a normal follow-up question activates them. The key control is not only prompt filtering; CRM write actions such as UpdateCustomerContact need request-scoped authority, independent validation, and recovery-ready data integrity controls.

Why it relates: Query Records retains assembled text beyond the visible rows, and a later turn invokes a separately administrator-enabled contact-write action. The case retains attacker instructions in short-term conversational memory and activates them on a later benign interaction. Long-term storage across sessions is not required for the documented short-term-memory path.

Zenity demonstrated that an unauthenticated attacker could use a public Salesforce Web-to-Case form to insert four specially chained case subjects that together formed a prompt mine. When an employee later asked Einstein for open cases, Query Records loaded the records into retained context even though the interface rendered only the first three. A natural follow-up question, including one asked five turns later, activated the hidden instructions, caused Einstein to query contact IDs, and repeatedly invoke an administrator-enabled Update Customer Contact action to replace customer email addresses with an attacker-specified value. The research required that write action to be configured in the organization and was not an out-of-box write path.

AIA · Read the full case ↗ · SecureFlow · Read the full case ↗

Validated Research ·

Copirate 365: Persistent Memory-Backdoor Data Exfiltration in Microsoft 365 Copilot (CVE-2026-24299)

A validated research flow showing how a poisoned document, Delayed Tool Invocation, and a record_memory write build a persistent Microsoft 365 Copilot backdoor that exfiltrates pasted secrets every future session through a CSS font-face URL that evades the image-src CSP.

Why it relates: A validated research flow showing how a poisoned document, Delayed Tool Invocation, and a record_memory write build a persistent Microsoft 365 Copilot backdoor that exfiltrates pasted secrets every future session through a CSS font-face URL that evades the image-src CSP. The relevant category is Memory Poisoning: An attacker inserts false or malicious information into an AI agent's short-term or long-term memory.

A validated research flow showing how a poisoned document, Delayed Tool Invocation, and a record_memory write build a persistent Microsoft 365 Copilot backdoor that exfiltrates pasted secrets every future session through a CSS font-face URL that evades the image-src CSP.

SecureFlow · Read the full case ↗

Validated Research ·

Gemini Voice Assistant: When Phone Notifications Become Prompt Injection

SafeBreach showed that Gemini Voice Assistant (Google's voice assistant) could follow malicious instructions hidden in phone notifications from WhatsApp, Slack, and SMS. Note: The defense responsibility mainly belongs to assistant and platform providers, including Google Gemini, Amazon Alexa, Apple Siri, Samsung Bixby, and teams building assistant runtimes. Enterprise adopters usually cannot repair the vendor runtime; they can only reduce exposure through device policy, notification design, and vendor governance.

Why it relates: External productivity content reaches later-turn tool execution across Workspace, device and connected-home authority. The case retains attacker instructions in short-term conversational memory and activates them on a later benign interaction. Long-term storage across sessions is not required for the documented short-term-memory path.

SafeBreach researchers demonstrated how adversary-controlled instructions embedded in productivity content, including Google Calendar invitations, emails, and shared files, could influence the behavior of Gemini-powered assistants when that content was later retrieved. An adversary places malicious instructions in content likely to be retrieved in response to a future victim request. When Gemini incorporates the adversary-controlled content into the conversation context, the instructions can influence the model's behavior and cause it to use the victim's authorized tools and connected services in unintended ways. The demonstrated attack paths share a common chain: an adversary sends a poisoned Calendar invitation; the victim asks Gemini about upcoming events; Gemini retrieves the malicious event title and adds it to conversation context; and a later victim response (e.g. "Thanks") triggers the embedded instructions. The attack was demonstrated on both the Gemini web and Android applications. The web application could access Workspace services, while the Android application exposed additional device and connected-home capabilities. The same chain was shown to produce several impacts: - Generate toxic content or adversary-selected promotions in Gemini responses. - Delete or create Calendar events using the victim's authorized Calendar access. - Control connected Google Home devices, including windows, boilers, and lights. - Open an adversary-controlled website, initiating a download and exposing the victim's IP address for approximate geolocation. - Invoke the Zoom application on the victim's device and stream video to an adversary-controlled meeting. - Retrieve Calendar or Gmail data, encode it in an adversary-controlled URL, and transmit it through a browser request.

AIA · Read the full case ↗

Validated Research ·

OpenClaw Command & Control via Prompt Injection

A curated AI-agent incident flow showing indirect prompt injection, malicious tool invocation, persistent agent configuration poisoning, and command-and-control behavior.

Why it relates: A curated AI-agent incident flow showing indirect prompt injection, malicious tool invocation, persistent agent configuration poisoning, and command-and-control behavior. The relevant category is Memory Poisoning: An attacker inserts false or malicious information into an AI agent's short-term or long-term memory.

A curated AI-agent incident flow showing indirect prompt injection, malicious tool invocation, persistent agent configuration poisoning, and command-and-control behavior.

SecureFlow · Read the full case ↗