2.3 · Raw data criteria

Explore 1 source-supported AI security cases for 2.3 · Raw data criteria in Databricks DASF 3.0, with evidence, scope, and defensive analysis.

Risk category

An attacker learns the criteria used to select raw data and crafts malicious inputs that satisfy those criteria. The accepted data can then compromise the model or system later in its lifecycle. Passing selection rules is therefore insufficient when the rules can be deliberately exploited to admit adversarial material.

Official framework source ↗ · Open interactive explorer ↗

Validated Research ·

Web-Scale Data Poisoning: Split-View Attack

Many recent large-scale datasets are distributed as a list of URLs pointing to individual datapoints. The researchers show that many of these datasets are vulnerable to a "split-view" poisoning attack. The attack exploits the fact that the data viewed when it was initially collected may differ from the data viewed by a user during training. The researchers identify expired and buyable domains that once hosted dataset content, making it possible to replace portions of the dataset with poisoned data. They demonstrate that for 10 popular web-scale datasets, enough of the domains are purchasable to successfully carry out a poisoning attack.

Why it relates: Published dataset URL lists identify exactly which external image origins will be downloaded. Buying expired listed domains can replace those examples, crossing the original curator-to-later-downloader integrity boundary. Public downloads were not poisoned in the study; downstream model effects were tested by local image replacement and CLIP retraining.

Many recent large-scale datasets are distributed as a list of URLs pointing to individual datapoints. The researchers show that many of these datasets are vulnerable to a "split-view" poisoning attack. The attack exploits the fact that the data viewed when it was initially collected may differ from the data viewed by a user during training. The researchers identify expired and buyable domains that once hosted dataset content, making it possible to replace portions of the dataset with poisoned data. They demonstrate that for 10 popular web-scale datasets, enough of the domains are purchasable to successfully carry out a poisoning attack.

SecureFlow · Read the full case ↗