Explore 6 source-supported AI security cases for AML.T0035 · AI Artifact Collection in MITRE ATLAS, with evidence, scope, and defensive analysis.
Risk category
Adversaries may collect AI artifacts for [Exfiltration](/tactics/AML.TA0010) or for use in [AI Attack Staging](/tactics/AML.TA0001). AI artifacts include models and datasets as well as other telemetry data produced when interacting with a model.
Ray is an open-source Python framework for scaling production AI workflows. Ray's Job API allows for arbitrary remote execution by design. However, it does not offer authentication, and the default configuration may expose the cluster to the internet. Researchers at Oligo discovered that Ray clusters have been actively exploited for at least seven months. Adversaries can use victim organization's compute power and steal valuable information. The researchers estimate the value of the compromised machines to be nearly 1 billion USD. Five vulnerabilities in Ray were reported to Anyscale, the maintainers of Ray. Anyscale promptly fixed four of the five vulnerabilities. However, the fifth vulnerability CVE-2023-48022 (https://nvd.nist.gov/vuln/detail/CVE-2023-48022) remains disputed. Anyscale maintains that Ray's lack of authentication is a design decision, and that Ray is meant to be deployed in a safe network environment. The Oligo researchers deem this a "shadow vulnerability" because in disputed status, the CVE does not show up in static scans.
ATLAS annotations follow the source SecureFlow steps; they are not a MITRE endorsement of cross-framework mappings.
Google Colab is a Jupyter Notebook service that executes on virtual machines. Jupyter Notebooks are often used for ML and data science research and experimentation, containing executable snippets of Python code and common Unix command-line functionality. In addition to data manipulation and visualization, this code execution functionality can allow users to download arbitrary files from the internet, manipulate files on the virtual machine, and so on. Users can also share Jupyter Notebooks with other users via links. In the case of notebooks with malicious code, users may unknowingly execute the offending code, which may be obfuscated or hidden in a downloaded script, for example. When a user opens a shared Jupyter Notebook in Colab, they are asked whether they'd like to allow the notebook to access their Google Drive. While there can be legitimate reasons for allowing Google Drive access, such as to allow a user to substitute their own files, there can also be malicious effects such as data exfiltration or opening a server to the victim's Google Drive. This exercise raises awareness of the effects of arbitrary code execution and Colab's Google Drive integration. Practice secure evaluations of shared Colab notebook links and examine code prior to execution.
ATLAS annotations follow the source SecureFlow steps; they are not a MITRE endorsement of cross-framework mappings.
Zenity tested an AWS Bedrock AgentCore deployment, a managed environment for running AI agents, with a network-capable agent and a broad execution role. The agent relayed temporary role credentials from its metadata endpoint. Off-platform API calls then reached other memories and agents. The distinctive persistence path wrote short-term conversation events that configured extraction strategies could turn into long-term records; later retrieval directed the agent to a mutable command page. The reach was demonstrated within an account and region, not as a microVM escape or automatic cross-account compromise. The researchers observed major default-role restrictions on September 29; existing deployed roles still require independent inspection.
ATLAS annotations follow the source SecureFlow steps; they are not a MITRE endorsement of cross-framework mappings.
Stolen Thoughts found that opaque reasoning blocks returned by Anthropic, OpenAI, and Google APIs could be accepted across sessions, users, or compatible models within a provider family. An attacker who first obtained another user's block could submit it to a weaker compatible model and jailbreak that model into transcribing hidden reasoning. In 6,708 public agent trajectories, the researchers reconstructed 315,320 blocks and found real sensitive artifacts in 328 sessions. They report that provider mitigations later made the demonstrated attacks non-reproducible.
ATLAS annotations follow the source SecureFlow steps; they are not a MITRE endorsement of cross-framework mappings.
Skyld researchers analyzed the Google Photos Android application and recovered TensorFlow Lite models used by AI-powered photo editing and image analysis features. The researchers found models stored unencrypted in the application's assets, embedded in the native library, and encrypted on disk. They used static reverse engineering to locate TFLite artifacts and dynamic instrumentation with Frida to capture encrypted models after runtime decryption. The recovered models provided white-box access to proprietary Google Photos model artifacts.
ATLAS annotations follow the source SecureFlow steps; they are not a MITRE endorsement of cross-framework mappings.
The Microsoft AI Red Team performed a red team exercise on an internal Azure service with the intention of disrupting its service. This operation had a combination of traditional ATT&CK enterprise techniques such as finding valid account, and exfiltrating data -- all interleaved with adversarial ML specific steps such as offline and online evasion examples.
ATLAS annotations follow the source SecureFlow steps; they are not a MITRE endorsement of cross-framework mappings.