Explore 4 source-supported AI security cases for NISTAML.051 · Model Poisoning (Supply Chain) in NIST Adversarial ML (2025), with evidence, scope, and defensive analysis.
Risk category
An attacker supplies a model or model component that has been maliciously modified before a downstream organization adopts it. A pretrained model can carry targeted behavior or a backdoor, and some modifications can persist through the customer's fine-tuning or additional safety training. The supply-chain issue is inherited behavior in a trusted model dependency, so checking only for ordinary executable malware does not cover every form of model poisoning.
Lazarus explores backdoors planted by editing a small part of an open-weight model. The critical prerequisite is control of the checkpoint that a defender later accepts. Its results support stronger model intake and tool containment, while the paper’s different prediction and execution measurements must remain distinct.
Why it relates: A downstream adopter can receive a pretrained checkpoint with the backdoor already present.
Only the main executed marker path supports execution claims; action-format prediction measurements retain their narrower scope.
Microsoft reports two controlled post-training attacks: a victim fine-tunes on poisoned agent trajectories, or adopts an attacker-trained checkpoint. Ordinary requests or repository context then trigger unsafe, schema-valid tool calls. Kerkoporta uses supervised fine-tuning followed by reinforcement learning to recover useful coding performance while retaining the backdoor. Neither study establishes a malicious customer breach or a defect in stock coding-agent configurations.
Why it relates: A downstream user can inherit the backdoor by adopting the attacker-prepared checkpoint.
Dataset and checkpoint experiments remain separate; tool-call acceptance, actual execution and production compromise are not interchangeable.
Researchers at Trend Micro, Inc. used service indexing portals and web searching tools to identify over 8,000 misconfigured private container registries exposed on the internet. Approximately 70% of the registries also had overly permissive access controls that allowed write access. In their analysis, the researchers found over 1,000 unique AI models embedded in private container images within these open registries that could be pulled without authentication. This exposure could allow adversaries to download, inspect, and modify container contents, including sensitive AI model files. This is an exposure of valuable intellectual property which could be stolen by an adversary. Compromised images could also be pushed to the registry, leading to a supply chain attack, allowing malicious actors to compromise the integrity of AI models used in production systems.
Why it relates: Registry exposure permits model-image retrieval and potential substitution; later malicious pulls are not established by the exposure survey. The source documents an externally supplied or substituted model artifact or model component carrying malicious code or modified model behavior into its consumer. The supply-chain boundary is artifact adoption; the case-specific loader or parameter-editing mechanism remains distinct. The supported parameter modification and repackaging are a controlled mock-up. The registry exposure survey does not establish a malicious production image pull or a public-model poisoning incident.
Researchers at Trend Micro, Inc. used service indexing portals and web searching tools to identify over 8,000 misconfigured private container registries exposed on the internet. Approximately 70% of the registries also had overly permissive access controls that allowed write access. In their analysis, the researchers found over 1,000 unique AI models embedded in private container images within these open registries that could be pulled without authentication. This exposure could allow adversaries to download, inspect, and modify container contents, including sensitive AI model files. This is an exposure of valuable intellectual property which could be stolen by an adversary. Compromised images could also be pushed to the registry, leading to a supply chain attack, allowing malicious actors to compromise the integrity of AI models used in production systems.
Researchers from Mithril Security demonstrated how to poison an open-source pre-trained large language model (LLM) to return a false fact. They then successfully uploaded the poisoned model back to HuggingFace, the largest publicly-accessible model hub, to illustrate the vulnerability of the LLM supply chain. Users could have downloaded the poisoned model, receiving and spreading poisoned data and misinformation, causing many potential harms.
Why it relates: Researchers from Mithril Security demonstrated how to poison an open-source pre-trained large language model (LLM) to return a false fact. The relevant category is Model Poisoning (Supply Chain): An attacker supplies a model or model component that has been maliciously modified before a downstream organization adopts it.
Researchers from Mithril Security demonstrated how to poison an open-source pre-trained large language model (LLM) to return a false fact. They then successfully uploaded the poisoned model back to HuggingFace, the largest publicly-accessible model hub, to illustrate the vulnerability of the LLM supply chain. Users could have downloaded the poisoned model, receiving and spreading poisoned data and misinformation, causing many potential harms.