10.3 · Discover ML model ontology

Explore 1 source-supported AI security cases for 10.3 · Discover ML model ontology in Databricks DASF 3.0, with evidence, scope, and defensive analysis.

Risk category

An attacker learns the model's output categories and the range of objects or responses it can identify. The information may be inferred through repeated responses or obtained from configuration and documentation. Understanding that output space helps the attacker tailor subsequent queries and attacks to what the model actually recognizes.

Official framework source ↗ · Open interactive explorer ↗

Exercise ·

Face Identification System Evasion via Physical Countermeasures

MITRE's AI Red Team demonstrated a physical-domain evasion attack on a commercial face identification service with the intention of inducing a targeted misclassification. This operation had a combination of traditional MITRE ATT&CK techniques such as finding valid accounts and executing code via an API - all interleaved with adversarial ML specific attacks.

Why it relates: The red team queried the commercial face-identification inference API to identify the list of identities the model could recognize, then used a proxy model to develop a physical adversarial patch. The enumerated identity classes are the model output ontology, not training-set membership.

MITRE's AI Red Team demonstrated a physical-domain evasion attack on a commercial face identification service with the intention of inducing a targeted misclassification. This operation had a combination of traditional MITRE ATT&CK techniques such as finding valid accounts and executing code via an API - all interleaved with adversarial ML specific attacks.

SecureFlow · Read the full case ↗