Explore 3 source-supported AI security cases for 13.18 · Tool Poisoning in Databricks DASF 3.0, with evidence, scope, and defensive analysis.
Risk category
An attacker presents a malicious tool as legitimate or gives a legitimate tool a deceptive description. Fake descriptions, harmful implementations, and look-alike tool names can cause the model to trust or invoke the wrong tool. The attack exploits the trust placed in available tools and the metadata used to select them.
Researchers at Invariant Labs demonstrated that AI agents configured with remote Model Context Protocol (MCP) Tools can be vulnerable to model poisoning attacks. They show that an MCP Tool can contain malicious prompts in its docstring description, which is ingested into the AI agent's context, modifying its behavior. They demonstrate this attack with a proof-of-concept MCP Tool that instructs the agent to perform additional actions before using the tool. The agent is instructed to read files containing credentials from the victim's machine and store their contents in one of the input variables to the tool. When the tool runs, the victim's credentials are exfiltrated to the poisoned MCP server.
Why it relates: Researchers at Invariant Labs demonstrated that AI agents configured with remote Model Context Protocol (MCP) Tools can be vulnerable to model poisoning attacks. The relevant category is Tool Poisoning: An attacker presents a malicious tool as legitimate or gives a legitimate tool a deceptive description.
Researchers at Invariant Labs demonstrated that AI agents configured with remote Model Context Protocol (MCP) Tools can be vulnerable to model poisoning attacks. They show that an MCP Tool can contain malicious prompts in its docstring description, which is ingested into the AI agent's context, modifying its behavior. They demonstrate this attack with a proof-of-concept MCP Tool that instructs the agent to perform additional actions before using the tool. The agent is instructed to read files containing credentials from the victim's machine and store their contents in one of the input variables to the tool. When the tool runs, the victim's credentials are exfiltrated to the poisoned MCP server.
Air Security demonstrated that AI coding clients could install different plugin code from the commit reviewed by a marketplace. An attacker controlling an already trusted plugin repository could shadow a pinned SHA with a default branch of the same name, or shadow FETCH_HEAD in Gemini CLI. Background updates then loaded malicious code without a fresh install decision. The defect is verification of the resolved working-tree commit, not a failure to review the marketplace entry; both repository control and a compatible client/hosting path are material prerequisites.
Why it relates: A reviewed integration is replaced with a harmful implementation while retaining apparent approved identity. The shared issue is executable integration poisoning, not a proven MCP tool-description or namespace collision.
Controlled client plugin-distribution research with alternative Git checkout routes; no model prompt compromise or observed mass-victim campaign is inferred.
Pillar Security documented Deadbugz, an active campaign that opened 23 GitHub pull requests in about 74 minutes to place a malicious MCP server in popular repositories and MCP directories. The server behaved normally at first, then changed later tool descriptions and prompt content into instructions for credential discovery, covert file access, command execution, and exfiltration. None of the reviewed pull requests was merged, and the public evidence does not show victim execution or confirmed data theft.
Why it relates: Pillar Security documented Deadbugz, an active campaign that opened 23 GitHub pull requests in about 74 minutes to place a malicious MCP server in popular repositories and MCP directories. The relevant category is Tool Poisoning: An attacker presents a malicious tool as legitimate or gives a legitimate tool a deceptive description. Repository admission and later client refresh would be required before poisoned discovery content could reach an agent; reviewed PRs were not merged.
Pillar Security documented Deadbugz, an active campaign that opened 23 GitHub pull requests in about 74 minutes to place a malicious MCP server in popular repositories and MCP directories. The server behaved normally at first, then changed later tool descriptions and prompt content into instructions for credential discovery, covert file access, command execution, and exfiltration. None of the reviewed pull requests was merged, and the public evidence does not show victim execution or confirmed data theft. The campaign used real pull requests for delivery. The actor submitted remote and local MCP configurations, plus directory listings, to established projects. This made repository review the supply-chain boundary the actor was trying to cross. The server delayed its malicious behavior. It tracked calls by client IP and kept two tools benign. After three tool calls, a later tools/list or prompts/get response returned poisoned content. No change notification was sent, so the client still had to refresh the list or request the prompt. The payload relied on client authority. It asked the agent to find credential files, conceal access, modify files, run commands, and exfiltrate results. The server could not read victim files directly; success required a client to ingest the changed content and an agent with sufficient tools, permissions, and compliance. No victim compromise is confirmed. Pillar found no merged pull request in the reviewed set, and the public record does not prove installation, execution, credential access, or exfiltration.