NISTAML.023 · Backdoor Poisoning

Explore 2 source-supported AI security cases for NISTAML.023 · Backdoor Poisoning in NIST Adversarial ML (2025), with evidence, scope, and defensive analysis.

Risk category

An attacker poisons a model during training so that inputs containing a particular pattern or trigger produce attacker-chosen behavior. Triggered inputs may be misclassified even while the model remains accurate on ordinary inputs. Triggers can take different forms across images, audio, text, and other data, and some backdoors can persist through later fine-tuning.

Official framework source ↗ · Open interactive explorer ↗

Exercise · · Practical overlap

Lazarus: A Small Weight Edit Can Hide a Triggered Agent Payload

Lazarus explores backdoors planted by editing a small part of an open-weight model. The critical prerequisite is control of the checkpoint that a defender later accepts. Its results support stronger model intake and tool containment, while the paper’s different prediction and execution measurements must remain distinct.

Why it relates: The model preserves ordinary behavior while a planted trigger activates attacker-selected output and tool behavior. The shared mechanism is a trigger-dependent model backdoor. Lazarus directly edits model parameters rather than poisoning a disclosed training dataset.

Only the main executed marker path supports execution claims; action-format prediction measurements retain their narrower scope.

AIA · Read the full case ↗

Exercise ·

Trained to Betray: Poisoned Data and Checkpoints Redirect Coding Agents

Microsoft reports two controlled post-training attacks: a victim fine-tunes on poisoned agent trajectories, or adopts an attacker-trained checkpoint. Ordinary requests or repository context then trigger unsafe, schema-valid tool calls. Kerkoporta uses supervised fine-tuning followed by reinforcement learning to recover useful coding performance while retaining the backdoor. Neither study establishes a malicious customer breach or a defect in stock coding-agent configurations.

Why it relates: Poisoned training teaches trigger-dependent tool behavior while ordinary functionality is retained.

Dataset and checkpoint experiments remain separate; tool-call acceptance, actual execution and production compromise are not interchangeable.

AIA · Read the full case ↗ · SecureFlow · Read the full case ↗