Explore 10 source-supported AI security cases for 7.3 · ML Supply chain vulnerabilities in Databricks DASF 3.0, with evidence, scope, and defensive analysis.
Risk category
An attacker compromises a model source that downstream developers reuse, such as a public model collection or third-party ML service. Reusing and adapting pretrained models can then carry the compromise into many applications. The attacker may also publish a specialized malicious model that receives little scrutiny, drawing intended users to a poisoned source.
Lazarus explores backdoors planted by editing a small part of an open-weight model. The critical prerequisite is control of the checkpoint that a defender later accepts. Its results support stronger model intake and tool containment, while the paper’s different prediction and execution measurements must remain distinct.
Why it relates: Adopting the externally modified checkpoint introduces its backdoor into the coding agent.
Only the main executed marker path supports execution claims; action-format prediction measurements retain their narrower scope.
Microsoft reports two controlled post-training attacks: a victim fine-tunes on poisoned agent trajectories, or adopts an attacker-trained checkpoint. Ordinary requests or repository context then trigger unsafe, schema-valid tool calls. Kerkoporta uses supervised fine-tuning followed by reinforcement learning to recover useful coding performance while retaining the backdoor. Neither study establishes a malicious customer breach or a defect in stock coding-agent configurations.
Why it relates: Reusing the prepared checkpoint transfers malicious behavior into the adopter’s agent.
Dataset and checkpoint experiments remain separate; tool-call acceptance, actual execution and production compromise are not interchangeable.
Researchers at Trend Micro, Inc. used service indexing portals and web searching tools to identify over 8,000 misconfigured private container registries exposed on the internet. Approximately 70% of the registries also had overly permissive access controls that allowed write access. In their analysis, the researchers found over 1,000 unique AI models embedded in private container images within these open registries that could be pulled without authentication. This exposure could allow adversaries to download, inspect, and modify container contents, including sensitive AI model files. This is an exposure of valuable intellectual property which could be stolen by an adversary. Compromised images could also be pushed to the registry, leading to a supply chain attack, allowing malicious actors to compromise the integrity of AI models used in production systems.
Why it relates: Researchers at Trend Micro, Inc. The relevant category is ML Supply chain vulnerabilities: An attacker compromises a model source that downstream developers reuse, such as a public model collection or third-party ML service. Registry exposure permits model-image retrieval and potential substitution; later malicious pulls are not established by the exposure survey.
Researchers at Trend Micro, Inc. used service indexing portals and web searching tools to identify over 8,000 misconfigured private container registries exposed on the internet. Approximately 70% of the registries also had overly permissive access controls that allowed write access. In their analysis, the researchers found over 1,000 unique AI models embedded in private container images within these open registries that could be pulled without authentication. This exposure could allow adversaries to download, inspect, and modify container contents, including sensitive AI model files. This is an exposure of valuable intellectual property which could be stolen by an adversary. Compromised images could also be pushed to the registry, leading to a supply chain attack, allowing malicious actors to compromise the integrity of AI models used in production systems.
threlfall_hax (https://5stars217.github.io/), a security researcher, created organization accounts on Hugging Face, a public model repository, that impersonated real organizations. These false Hugging Face organization accounts looked legitimate so individuals from the impersonated organizations requested to join, believing the accounts to be an official site for employees to share models. This gave the researcher full access to any AI models uploaded by the employees, including the ability to replace models with malicious versions. The researcher demonstrated that they could embed malware into an AI model that provided them access to the victim organization's environment. From there, threat actors could execute a range of damaging attacks such as intellectual property theft or poisoning other AI models within the victim's environment.
Why it relates: threlfall_hax (https://5stars217.github.io/), a security researcher, created organization accounts on Hugging Face, a public model repository, that impersonated real organizations. The relevant category is ML Supply chain vulnerabilities: An attacker compromises a model source that downstream developers reuse, such as a public model collection or third-party ML service. Organization trust exposes employee-uploaded models and replacement authority; the research also demonstrated malicious-model execution.
threlfall_hax (https://5stars217.github.io/), a security researcher, created organization accounts on Hugging Face, a public model repository, that impersonated real organizations. These false Hugging Face organization accounts looked legitimate so individuals from the impersonated organizations requested to join, believing the accounts to be an official site for employees to share models. This gave the researcher full access to any AI models uploaded by the employees, including the ability to replace models with malicious versions. The researcher demonstrated that they could embed malware into an AI model that provided them access to the victim organization's environment. From there, threat actors could execute a range of damaging attacks such as intellectual property theft or poisoning other AI models within the victim's environment.
Researchers at ReversingLabs have identified malicious models containing embedded malware hosted on the Hugging Face model repository. The models were found to execute reverse shells when loaded, which grants the threat actor command and control capabilities on the victim's system. Hugging Face uses Picklescan to scan models for malicious code, however these models were not flagged as malicious. The researchers discovered that the model files were seemingly purposefully corrupted in a way that the malicious payload is executed before the model ultimately fails to de-serialize fully. Picklescan relied on being able to fully de-serialize the model. Since becoming aware of this issue, Hugging Face has removed the models and has made changes to Picklescan to catch this particular attack. However, pickle files are fundamentally unsafe as they allow for arbitrary code execution, and there may be other types of malicious pickles that Picklescan cannot detect.
Why it relates: Researchers at ReversingLabs have identified malicious models containing embedded malware hosted on the Hugging Face model repository. The relevant category is ML Supply chain vulnerabilities: An attacker compromises a model source that downstream developers reuse, such as a public model collection or third-party ML service.
Researchers at ReversingLabs have identified malicious models containing embedded malware hosted on the Hugging Face model repository. The models were found to execute reverse shells when loaded, which grants the threat actor command and control capabilities on the victim's system. Hugging Face uses Picklescan to scan models for malicious code, however these models were not flagged as malicious. The researchers discovered that the model files were seemingly purposefully corrupted in a way that the malicious payload is executed before the model ultimately fails to de-serialize fully. Picklescan relied on being able to fully de-serialize the model. Since becoming aware of this issue, Hugging Face has removed the models and has made changes to Picklescan to catch this particular attack. However, pickle files are fundamentally unsafe as they allow for arbitrary code execution, and there may be other types of malicious pickles that Picklescan cannot detect.
Researchers from Pillar Security and Fujitsu showed that a GGUF artifact can carry a modified Jinja chat template while its model weights remain unchanged. When a trigger appears, the inference engine renders attacker instructions into system-level context before the model runs; controlled tests degraded model behavior and hijacked BrowserUse and OpenHands tool use. Defenders need to treat chat templates as executable release artifacts, verify their exact bytes and provenance, regression-test every conversion, and enforce tool and data-flow boundaries.
Why it relates: The externally supplied model artifact contains a trigger-dependent template backdoor, and adopting the redistributed artifact carries that behavior into downstream model and agent use.
Researchers from Pillar Security and Fujitsu Research of Europe demonstrated an inference-time supply-chain backdoor in which poisoned chat templates alter model and agent behavior without modifying model weights. The backdoor is embedded in trusted prompt-construction logic, allowing it to remain dormant until triggered and evade defenses that inspect only external prompt content. The researchers demonstrated the attack using GPT-Generated Unified Format (GGUF), a widely used model format that packages quantized weights, configuration metadata, and chat-template logic in one artifact. An adversary can modify a template and redistribute the artifact; when a lexical, semantic, or contextual trigger occurs, the template injects attacker-controlled instructions into the context sent to the model. The attack was validated across eighteen models from seven families and four inference engines. In controlled evaluations, it manipulated model responses, redirected agent tool use, exfiltrated sensitive data, and inserted attacker-controlled code into generated software.
Unit 42 showed that a deleted Hugging Face author namespace, or a transferred model whose former author namespace later becomes reclaimable, can leave stale references pointing to a name another party can register. In controlled tests, researchers reclaimed abandoned namespaces, recreated expected model paths, and made Vertex AI and Azure AI Foundry deployments load reverse-shell models. The original publisher account was not compromised. Defenders should bind deployments to immutable digests and signers, monitor namespace lifecycles, mirror approved artifacts internally, and isolate model loading.
Why it relates: Unit 42 researchers demonstrated an AI supply chain attack in which an attacker reclaims a deleted Hugging Face author namespace and publishes a malicious model under the same historical Author/ModelName identifier. The relevant category is ML Supply chain vulnerabilities: An attacker compromises a model source that downstream developers reuse, such as a public model collection or third-party ML service.
Unit 42 researchers demonstrated an AI supply chain attack in which an attacker reclaims a deleted Hugging Face author namespace and publishes a malicious model under the same historical Author/ModelName identifier. Applications and model catalogs that retain unpinned references to that identifier may resolve and deploy the adversary-controlled replacement rather than the originally trusted artifact. The attack can affect deleted models and models whose ownership was transferred to a new Hugging Face author. In the ownership-transfer scenario, Hugging Face redirects requests for the old path to the new model location, allowing stale references to continue working. If the original author namespace is later deleted and reclaimed by an attacker, the attacker can recreate the old path and cause it to resolve to a malicious model instead of the legitimate redirected model. Unit 42 demonstrated this technique against Hugging Face-backed model catalogs in Google Vertex AI and Azure AI Foundry. The researchers embedded reverse-shell payloads in replacement models and obtained code execution in the deployed endpoint environments. They also identified reusable model references in open-source code repositories, documentation, default arguments, example notebooks, and downstream model registries, which could expose users who do not directly interact with Hugging Face.
Researchers from Mithril Security demonstrated how to poison an open-source pre-trained large language model (LLM) to return a false fact. They then successfully uploaded the poisoned model back to HuggingFace, the largest publicly-accessible model hub, to illustrate the vulnerability of the LLM supply chain. Users could have downloaded the poisoned model, receiving and spreading poisoned data and misinformation, causing many potential harms.
Why it relates: Researchers from Mithril Security demonstrated how to poison an open-source pre-trained large language model (LLM) to return a false fact. The relevant category is ML Supply chain vulnerabilities: An attacker compromises a model source that downstream developers reuse, such as a public model collection or third-party ML service.
Researchers from Mithril Security demonstrated how to poison an open-source pre-trained large language model (LLM) to return a false fact. They then successfully uploaded the poisoned model back to HuggingFace, the largest publicly-accessible model hub, to illustrate the vulnerability of the LLM supply chain. Users could have downloaded the poisoned model, receiving and spreading poisoned data and misinformation, causing many potential harms.
HiddenLayer disclosed CVE-2026-45829, a ChromaDB (an open-source vector database) Python FastAPI server flaw where an unauthenticated collection-creation request can download and execute attacker-controlled Hugging Face model code before authentication runs. The lesson is direct: model loading is code execution. Vector databases need early authentication, restricted model provenance, unsafe-loader blocking, and isolated first-use execution.
Why it relates: HiddenLayer disclosed CVE-2026-45829, a ChromaDB (an open-source vector database) Python FastAPI server flaw where an unauthenticated collection-creation request can download and execute attacker-controlled Hugging Face model code before authentication runs. The relevant category is ML Supply chain vulnerabilities: An attacker compromises a model source that downstream developers reuse, such as a public model collection or third-party ML service.
HiddenLayer disclosed CVE-2026-45829, a ChromaDB (an open-source vector database) Python FastAPI server flaw where an unauthenticated collection-creation request can download and execute attacker-controlled Hugging Face model code before authentication runs. The lesson is direct: model loading is code execution. Vector databases need early authentication, restricted model provenance, unsafe-loader blocking, and isolated first-use execution. The endpoint appears protected. ChromaDB marks collection creation as authenticated, but an unauthenticated request can still carry embedding-function configuration that points to an attacker-controlled Hugging Face model. The dangerous flag is trust_remote_code . When set through request-controlled kwargs , it tells the model loader to fetch and run Python code from that model repository. The ordering is the bug. The Python server instantiates the embedding function before the authentication check, so the model is downloaded and executed before the request is rejected. The failed API call can still compromise the server. The response may look like an error, but the attacker-controlled code has already run inside the ChromaDB process. The blast radius is the server process. Environment variables, API keys, mounted secrets, local data, and reachable internal services can all become exposed.
JFrog's 2024 research showed that malicious Hugging Face models (models distributed through a widely used AI model hub) can turn ordinary model loading into code execution. The observed PyTorch payload used pickle deserialization to start a reverse shell, which means the defensive boundary cannot be only "do we trust this model name?" It has to cover model provenance, unsafe format blocking, isolated loading, and outbound network control.
Why it relates: JFrog's 2024 research showed that malicious Hugging Face models (models distributed through a widely used AI model hub) can turn ordinary model loading into code execution. The relevant category is ML Supply chain vulnerabilities: An attacker compromises a model source that downstream developers reuse, such as a public model collection or third-party ML service.
JFrog's 2024 research showed that malicious Hugging Face models (models distributed through a widely used AI model hub) can turn ordinary model loading into code execution. The observed PyTorch payload used pickle deserialization to start a reverse shell, which means the defensive boundary cannot be only "do we trust this model name?" It has to cover model provenance, unsafe format blocking, isolated loading, and outbound network control. The model artifact was the delivery vehicle. JFrog found a Hugging Face PyTorch model whose pickle payload executed during load. This is an AI supply-chain path where the artifact itself carries executable behavior. The dangerous moment is deserialization. torch.load() can invoke pickle object reconstruction hooks, allowing attacker-controlled Python to run and start a shell connection to an external host. Warnings are useful, but they are not a control boundary. Hugging Face labels unsafe pickle models, but marked models may still be downloaded and executed. Enterprises need admission rules that fail closed. The blast radius depends on the loading environment. A malicious model opened on a laptop, notebook server, CI runner, or shared GPU box can inherit cloud keys, dataset access, SSH material, and internal network reachability. This pattern outlives one deleted repository. JFrog reported similar payloads elsewhere and noted that other formats can also expose code-execution paths. Treat third-party model files as executable supply-chain inputs until proven otherwise.