Explore 7 source-supported AI security cases for AML.T0011.000 · Unsafe AI Artifacts in MITRE ATLAS, with evidence, scope, and defensive analysis.
Risk category
Adversaries may develop new AI artifacts or manipulate existing AI artifacts that have a deleterious effect when loaded, executed, interpreted, or used for inference by a victim. The unsafe artifacts may continue to work as expected from the victim's perspective. Unsafe AI artifacts may include serialized models, model packages, and bundled components such as prompt or chat templates, tokenizer metadata, configuration, pre-processing logic, post-processing logic, or other executable or behavior-shaping metadata. An unsafe artifact may exploit deserialization or another software vulnerability to execute code on the host. However, exploitation of a software vulnerability is not required. An artifact may instead use functionality intentionally supported by an AI runtime to alter model context, model outputs, safety behavior, or agent actions. Adversaries may introduce unsafe artifacts through an [AI Supply Chain Compromise](/techniques/AML.T0010). Execution may occur when a user loads the artifact, starts an inference runtime, submits an inference request, or otherwise causes the artifact's bundled logic to be processed.
threlfall_hax (https://5stars217.github.io/), a security researcher, created organization accounts on Hugging Face, a public model repository, that impersonated real organizations. These false Hugging Face organization accounts looked legitimate so individuals from the impersonated organizations requested to join, believing the accounts to be an official site for employees to share models. This gave the researcher full access to any AI models uploaded by the employees, including the ability to replace models with malicious versions. The researcher demonstrated that they could embed malware into an AI model that provided them access to the victim organization's environment. From there, threat actors could execute a range of damaging attacks such as intellectual property theft or poisoning other AI models within the victim's environment.
ATLAS annotations follow the source SecureFlow steps; they are not a MITRE endorsement of cross-framework mappings.
Air Security demonstrated that AI coding clients could install different plugin code from the commit reviewed by a marketplace. An attacker controlling an already trusted plugin repository could shadow a pinned SHA with a default branch of the same name, or shadow FETCH_HEAD in Gemini CLI. Background updates then loaded malicious code without a fresh install decision. The defect is verification of the resolved working-tree commit, not a failure to review the marketplace entry; both repository control and a compatible client/hosting path are material prerequisites.
ATLAS annotations follow the source SecureFlow steps; they are not a MITRE endorsement of cross-framework mappings.
On October 21, LiteLLM, a Python library and gateway for accessing model providers, published a credential-stealing 1.107.0 release through its legitimate build and signing pipeline. The vendor's October 23 investigation traces entry to an unrevoked master key leaked from its experimental Agent Platform, followed by theft of an organization-owner GitHub token. Separate CI runs stole two maintainer tokens. Consumer-side admission of previously approved exact bytes, and revocation at each credential issuer, address boundaries that a valid release signature did not protect.
ATLAS annotations follow the source SecureFlow steps; they are not a MITRE endorsement of cross-framework mappings.
Manifold Security reported eight GitSpawn findings across seven AI coding agents. A transferred repository with its local Git configuration intact can make Git execute a configured command during status, diff, or background context collection before the agent's trust prompt; the research did not report malicious real-world compromise, and a normal clone does not preserve this local configuration.
ATLAS annotations follow the source SecureFlow steps; they are not a MITRE endorsement of cross-framework mappings.
Researchers at ReversingLabs have identified malicious models containing embedded malware hosted on the Hugging Face model repository. The models were found to execute reverse shells when loaded, which grants the threat actor command and control capabilities on the victim's system. Hugging Face uses Picklescan to scan models for malicious code, however these models were not flagged as malicious. The researchers discovered that the model files were seemingly purposefully corrupted in a way that the malicious payload is executed before the model ultimately fails to de-serialize fully. Picklescan relied on being able to fully de-serialize the model. Since becoming aware of this issue, Hugging Face has removed the models and has made changes to Picklescan to catch this particular attack. However, pickle files are fundamentally unsafe as they allow for arbitrary code execution, and there may be other types of malicious pickles that Picklescan cannot detect.
ATLAS annotations follow the source SecureFlow steps; they are not a MITRE endorsement of cross-framework mappings.
Researchers from Pillar Security and Fujitsu showed that a GGUF artifact can carry a modified Jinja chat template while its model weights remain unchanged. When a trigger appears, the inference engine renders attacker instructions into system-level context before the model runs; controlled tests degraded model behavior and hijacked BrowserUse and OpenHands tool use. Defenders need to treat chat templates as executable release artifacts, verify their exact bytes and provenance, regression-test every conversion, and enforce tool and data-flow boundaries.
ATLAS annotations follow the source SecureFlow steps; they are not a MITRE endorsement of cross-framework mappings.
Unit 42 showed that a deleted Hugging Face author namespace, or a transferred model whose former author namespace later becomes reclaimable, can leave stale references pointing to a name another party can register. In controlled tests, researchers reclaimed abandoned namespaces, recreated expected model paths, and made Vertex AI and Azure AI Foundry deployments load reverse-shell models. The original publisher account was not compromised. Defenders should bind deployments to immutable digests and signers, monitor namespace lifecycles, mirror approved artifacts internally, and isolate model loading.
ATLAS annotations follow the source SecureFlow steps; they are not a MITRE endorsement of cross-framework mappings.