Explore 2 source-supported AI security cases for NISTAML.023 · Backdoor Poisoning in NIST Adversarial ML (2025), with evidence, scope, and defensive analysis.
Risk category
An attacker poisons a model during training so that inputs containing a particular pattern or trigger produce attacker-chosen behavior. Triggered inputs may be misclassified even while the model remains accurate on ordinary inputs. Triggers can take different forms across images, audio, text, and other data, and some backdoors can persist through later fine-tuning.
Lazarus explores backdoors planted by editing a small part of an open-weight model. The critical prerequisite is control of the checkpoint that a defender later accepts. Its results support stronger model intake and tool containment, while the paper’s different prediction and execution measurements must remain distinct.
Why it relates: The model preserves ordinary behavior while a planted trigger activates attacker-selected output and tool behavior. The shared mechanism is a trigger-dependent model backdoor. Lazarus directly edits model parameters rather than poisoning a disclosed training dataset.
Only the main executed marker path supports execution claims; action-format prediction measurements retain their narrower scope.
Microsoft reports two controlled post-training attacks: a victim fine-tunes on poisoned agent trajectories, or adopts an attacker-trained checkpoint. Ordinary requests or repository context then trigger unsafe, schema-valid tool calls. Kerkoporta uses supervised fine-tuning followed by reinforcement learning to recover useful coding performance while retaining the backdoor. Neither study establishes a malicious customer breach or a defect in stock coding-agent configurations.
Why it relates: Poisoned training teaches trigger-dependent tool behavior while ordinary functionality is retained.
Dataset and checkpoint experiments remain separate; tool-call acceptance, actual execution and production compromise are not interchangeable.