AML.T0076 · Corrupt AI Model

Explore 1 source-supported AI security cases for AML.T0076 · Corrupt AI Model in MITRE ATLAS, with evidence, scope, and defensive analysis.

Risk category

An adversary may purposefully corrupt a malicious AI model file so that it cannot be successfully deserialized in order to evade detection by a model scanner. The corrupt model may still successfully execute malicious code before deserialization fails.

Official framework source ↗ · Open interactive explorer ↗

Incident ·

Malicious Models on Hugging Face

Researchers at ReversingLabs have identified malicious models containing embedded malware hosted on the Hugging Face model repository. The models were found to execute reverse shells when loaded, which grants the threat actor command and control capabilities on the victim's system. Hugging Face uses Picklescan to scan models for malicious code, however these models were not flagged as malicious. The researchers discovered that the model files were seemingly purposefully corrupted in a way that the malicious payload is executed before the model ultimately fails to de-serialize fully. Picklescan relied on being able to fully de-serialize the model. Since becoming aware of this issue, Hugging Face has removed the models and has made changes to Picklescan to catch this particular attack. However, pickle files are fundamentally unsafe as they allow for arbitrary code execution, and there may be other types of malicious pickles that Picklescan cannot detect.

ATLAS annotations follow the source SecureFlow steps; they are not a MITRE endorsement of cross-framework mappings.

SecureFlow · Read the full case ↗