1.11 · Compromised 3rd-party datasets

Explore 3 source-supported AI security cases for 1.11 · Compromised 3rd-party datasets in Databricks DASF 3.0, with evidence, scope, and defensive analysis.

Risk category

An attacker poisons a third-party dataset before the organization adopts it. The attacker may publish a new malicious dataset, alter a public dataset already used by the target, or distribute a poisoned variant of a legitimate one. The resulting supply-chain compromise brings attacker-controlled training material into a system that trusts the dataset's source or distribution channel.

Official framework source ↗ · Open interactive explorer ↗

Exercise ·

Trained to Betray: Poisoned Data and Checkpoints Redirect Coding Agents

Microsoft reports two controlled post-training attacks: a victim fine-tunes on poisoned agent trajectories, or adopts an attacker-trained checkpoint. Ordinary requests or repository context then trigger unsafe, schema-valid tool calls. Kerkoporta uses supervised fine-tuning followed by reinforcement learning to recover useful coding performance while retaining the backdoor. Neither study establishes a malicious customer breach or a defect in stock coding-agent configurations.

Why it relates: The study’s poisoned dataset variant is supplied for downstream model adaptation.

Dataset and checkpoint experiments remain separate; tool-call acceptance, actual execution and production compromise are not interchangeable.

AIA · Read the full case ↗ · SecureFlow · Read the full case ↗

Validated Research ·

Web-Scale Data Poisoning: Split-View Attack

Many recent large-scale datasets are distributed as a list of URLs pointing to individual datapoints. The researchers show that many of these datasets are vulnerable to a "split-view" poisoning attack. The attack exploits the fact that the data viewed when it was initially collected may differ from the data viewed by a user during training. The researchers identify expired and buyable domains that once hosted dataset content, making it possible to replace portions of the dataset with poisoned data. They demonstrate that for 10 popular web-scale datasets, enough of the domains are purchasable to successfully carry out a poisoning attack.

Why it relates: Many recent large-scale datasets are distributed as a list of URLs pointing to individual datapoints. The relevant category is Compromised 3rd-party datasets: An attacker poisons a third-party dataset before the organization adopts it.

Many recent large-scale datasets are distributed as a list of URLs pointing to individual datapoints. The researchers show that many of these datasets are vulnerable to a "split-view" poisoning attack. The attack exploits the fact that the data viewed when it was initially collected may differ from the data viewed by a user during training. The researchers identify expired and buyable domains that once hosted dataset content, making it possible to replace portions of the dataset with poisoned data. They demonstrate that for 10 popular web-scale datasets, enough of the domains are purchasable to successfully carry out a poisoning attack.

SecureFlow · Read the full case ↗

Incident ·

VirusTotal Poisoning

McAfee Advanced Threat Research noticed an increase in reports of a certain ransomware family that was out of the ordinary. Case investigation revealed that many samples of that particular ransomware family were submitted through a popular virus-sharing platform within a short amount of time. Further investigation revealed that based on string similarity the samples were all equivalent, and based on code similarity they were between 98 and 74 percent similar. Interestingly enough, the compile time was the same for all the samples. After more digging, researchers discovered that someone used 'metame' a metamorphic code manipulating tool to manipulate the original file towards mutant variants. The variants would not always be executable, but are still classified as the same ransomware family.

Why it relates: McAfee Advanced Threat Research noticed an increase in reports of a certain ransomware family that was out of the ordinary. The relevant category is Compromised 3rd-party datasets: An attacker poisons a third-party dataset before the organization adopts it. An external sample source changes the data distribution seen by security-model pipelines; not every variant is executable.

McAfee Advanced Threat Research noticed an increase in reports of a certain ransomware family that was out of the ordinary. Case investigation revealed that many samples of that particular ransomware family were submitted through a popular virus-sharing platform within a short amount of time. Further investigation revealed that based on string similarity the samples were all equivalent, and based on code similarity they were between 98 and 74 percent similar. Interestingly enough, the compile time was the same for all the samples. After more digging, researchers discovered that someone used 'metame' a metamorphic code manipulating tool to manipulate the original file towards mutant variants. The variants would not always be executable, but are still classified as the same ransomware family.

SecureFlow · Read the full case ↗