Explore 4 source-supported AI security cases for AML.T0002 · Acquire Public AI Artifacts in MITRE ATLAS, with evidence, scope, and defensive analysis.
Risk category
Adversaries may search public sources, including cloud storage, public-facing services, and software or data repositories, to identify AI artifacts. These AI artifacts may include the software stack used to train and deploy models, training and testing data, model configurations and parameters. An adversary will be particularly interested in artifacts hosted by or associated with the victim organization as they may represent what that organization uses in a production environment. Adversaries may identify artifact repositories via other resources associated with the victim organization (e.g. [Search Victim-Owned Websites](/techniques/AML.T0003) or [Search Open Technical Databases](/techniques/AML.T0000)). These AI artifacts often provide adversaries with details of the AI task and approach. AI artifacts can aid in an adversary's ability to [Create Proxy AI Model](/techniques/AML.T0005). If these artifacts include pieces of the actual model in production, they can be used to directly [Craft Adversarial Data](/techniques/AML.T0043). Acquiring some artifacts requires registration (providing user details such as an email address or name), AWS keys, or written requests, and may require the adversary to [Establish Accounts](/techniques/AML.T0021). Artifacts might be hosted on victim-controlled infrastructure, providing the victim with some information on who has accessed that data.
Stolen Thoughts found that opaque reasoning blocks returned by Anthropic, OpenAI, and Google APIs could be accepted across sessions, users, or compatible models within a provider family. An attacker who first obtained another user's block could submit it to a weaker compatible model and jailbreak that model into transcribing hidden reasoning. In 6,708 public agent trajectories, the researchers reconstructed 315,320 blocks and found real sensitive artifacts in 328 sessions. They report that provider mitigations later made the demonstrated attacks non-reproducible.
ATLAS annotations follow the source SecureFlow steps; they are not a MITRE endorsement of cross-framework mappings.
Clearview AI makes a facial recognition tool that searches publicly available photos for matches. This tool has been used for investigative purposes by law enforcement agencies and other parties. Clearview AI's source code repository, though password protected, was misconfigured to allow an arbitrary user to register an account. This allowed an external researcher to gain access to a private code repository that contained Clearview AI production credentials, keys to cloud storage buckets containing 70K video samples, and copies of its applications and Slack tokens. With access to training data, a bad actor has the ability to cause an arbitrary misclassification in the deployed model. These kinds of attacks illustrate that any attempt to secure ML system should be on top of "traditional" good cybersecurity hygiene such as locking down the system with least privileges, multi-factor authentication and monitoring and auditing.
ATLAS annotations follow the source SecureFlow steps; they are not a MITRE endorsement of cross-framework mappings.
The Azure Red Team performed a red team exercise on a new Microsoft product designed for running AI workloads at the edge. This exercise was meant to use an automated system to continuously manipulate a target image to cause the ML model to produce misclassifications.
ATLAS annotations follow the source SecureFlow steps; they are not a MITRE endorsement of cross-framework mappings.
The Palo Alto Networks Security AI research team was able to bypass a Convolutional Neural Network based botnet Domain Generation Algorithm (DGA) detector using a generic domain name mutation technique. It is a generic domain mutation technique which can evade most ML-based DGA detection modules. The generic mutation technique evades most ML-based DGA detection modules and can be used to test the effectiveness and robustness of all DGA detection methods developed by security companies in the industry before they are deployed to the production environment.
ATLAS annotations follow the source SecureFlow steps; they are not a MITRE endorsement of cross-framework mappings.