MCFA Turns Agent Memory into a Delayed Control-Flow Input
Memory Control Flow Attacks (MCFA) do not need direct access to an agent's memory database or system prompt. The research shows how ordinary interaction can place an action-oriented preference into long-term memory, then a later benign task retrieves it and changes tool choice or order. Across controlled LangChain and LlamaIndex experiments, tool-selection attacks were highly reliable, while ordering attacks were weaker. Memory must therefore be governed as persistent control input, not trusted conversation history.
Threat Analysis
- The attacker did not need memory-database access. One or a few ordinary interactions caused the agent to store an action-oriented preference. A later benign task retrieved it and changed tool choice or order, while the trusted tools and system prompt remained unchanged.
- The study measured related but distinct outcomes. Override selected a risky tool and reached 97.2% to 100% success. Order violated a required sequence and reached 52.8% to 69.4%. Cross-task and more-than-30-round tests showed that the effect could persist, while matched benign-memory controls remained at 0%.
- Simple mitigations were incomplete. Adding 100 benign records did not remove the effect. Summarization reduced malicious-write success, but every summarized instruction that survived still steered the later task. Role-based memory segregation reduced some Override rates, yet residual success ranged from 2.8% to 100%.
- This was controlled research. The evaluation used synthetic safe and risky tool variants, no production users, and no harmful external actions. It measured tool-trace deviation, not real damage. There is currently no public evidence that MCFA has been used in an actual attack.
Applicable AIDEFEND Defenses (6)
What Defenders Should Do Now
- Trace every path that can write long-term memory, including direct chat, tool output, summaries, delegated tasks, and imported records. Mark which writes can later influence tool-capable agents.
- Route low-trust or insufficiently evidenced new memory to quarantine by default. Require version-bound evidence and an explicit promotion decision before a record can enter trusted retrieval.
- Separate memory by tenant, source, and trust tier, then enforce retrieval eligibility outside the model prompt. Do not rely on text such as “system memories outrank user memories” as the only boundary.
- Before rehydration, scan the exact recalled record and preserve the result with its version. At execution, independently authorize every high-impact action regardless of what memory recommended.
- Add regression tests for override, reordered tools, cross-task activation, long-horizon persistence, benign-memory crowding, and model upgrades. Measure both malicious-write admission and downstream unsafe action rates.
Conclusion
MCFA changes the security question from “who can edit the memory database?” to “what content can become future decision authority?” AIDEFEND maps this case to distinct controls for provenance, quarantine-to-trusted promotion, tenant and trust partitioning, recall-time inspection, sequence monitoring, and final action authorization. The decisive boundary must remain outside the model: a remembered preference may inform planning, but it must never grant permission to act.