Explore 5 source-supported AI security cases for AML.T0017.000 · Adversarial AI Attacks in MITRE ATLAS, with evidence, scope, and defensive analysis.
Risk category
Adversaries may develop their own adversarial attacks. They may leverage existing libraries as a starting point ([Adversarial AI Attack Implementations](/techniques/AML.T0016.000)). They may implement ideas described in public research papers or develop custom made attacks for the victim model.
Cloud storage and computations have become popular platforms for deploying ML malware detectors. In such cases, the features for models are built on users' systems and then sent to cybersecurity company servers. The Kaspersky ML research team explored this gray-box scenario and showed that feature knowledge is enough for an adversarial attack on ML models. They attacked one of Kaspersky's antimalware ML models without white-box access to it and successfully evaded detection for most of the adversarially modified malware files.
ATLAS annotations follow the source SecureFlow steps; they are not a MITRE endorsement of cross-framework mappings.
Deep learning models are increasingly used in mobile applications as critical components. Researchers from Microsoft Research demonstrated that many deep learning models deployed in mobile apps are vulnerable to backdoor attacks via "neural payload injection." They conducted an empirical study on real-world mobile deep learning apps collected from Google Play. They identified 54 apps that were vulnerable to attack, including popular security and safety critical applications used for cash recognition, parental control, face authentication, and financial services.
ATLAS annotations follow the source SecureFlow steps; they are not a MITRE endorsement of cross-framework mappings.
Researchers from Pillar Security and Fujitsu showed that a GGUF artifact can carry a modified Jinja chat template while its model weights remain unchanged. When a trigger appears, the inference engine renders attacker instructions into system-level context before the model runs; controlled tests degraded model behavior and hijacked BrowserUse and OpenHands tool use. Defenders need to treat chat templates as executable release artifacts, verify their exact bytes and provenance, regression-test every conversion, and enforce tool and data-flow boundaries.
ATLAS annotations follow the source SecureFlow steps; they are not a MITRE endorsement of cross-framework mappings.
Researchers at Skylight were able to create a universal bypass string that evades detection by Cylance's AI Malware detector when appended to a malicious file.
ATLAS annotations follow the source SecureFlow steps; they are not a MITRE endorsement of cross-framework mappings.
The Palo Alto Networks Security AI research team was able to bypass a Convolutional Neural Network based botnet Domain Generation Algorithm (DGA) detector using a generic domain name mutation technique. It is a generic domain mutation technique which can evade most ML-based DGA detection modules. The generic mutation technique evades most ML-based DGA detection modules and can be used to test the effectiveness and robustness of all DGA detection methods developed by security companies in the industry before they are deployed to the production environment.
ATLAS annotations follow the source SecureFlow steps; they are not a MITRE endorsement of cross-framework mappings.