MRE · Model Reverse Engineering

Explore 6 source-supported AI security cases for MRE · Model Reverse Engineering in Google SAIF 2.0, with evidence, scope, and defensive analysis.

Risk category

An attacker clones or recreates a model by examining its inputs, outputs, and observable behavior. Repeated API queries can supply input-output pairs for training an imitation or distilled model, while access to an on-device system may provide additional opportunities for analysis. The copy can support competing products or help develop attacks against the original model, without requiring direct theft of its weight files.

Official framework source ↗ · Open interactive explorer ↗

Incident ·

Industrial-Scale Distillation Across U.S. Frontier AI Services

NSA, CISA and FBI attributed coordinated distillation campaigns to six China-based AI companies. Their September 2026 advisory describes fraudulent accounts, bulk subscriptions, gray-market proxies, cross-provider routing, request-metadata removal and hidden-reasoning prompts, followed by response collection and quality checks. Distillation trains a model from another model’s outputs; the advisory’s attribution and capability-assessment claims are attributed to the agencies, not presented as independently reproduced measurements or model-weight theft.

Why it relates: The campaigns train or improve their own models from provider input-output behavior.

the campaigns attributed by NSA, CISA and FBI; capability distillation is distinguished from stealing original weights. Attribution and capability outcomes remain agency assessments. Hidden-reasoning prompts are attempts, not proof that the returned text was an internal trace. No autonomous agent deployment is inferred.

SecureFlow · Read the full case ↗

Incident ·

Model Distillation Campaigns Target Google AI Services

Google reported campaigns that used compromised credentials, fraudulent accounts and proxies to collect model outputs for distillation: training another model from a provider’s responses. A February 2026 reasoning-focused campaign exceeded 100,000 prompts; later campaigns included some above 100 million. Those are campaign query volumes, not model weights stolen or a measured replication success rate. Google says it disrupted accounts and reduced the usefulness of collected outputs.

Why it relates: The campaigns train or improve their own models from provider input-output behavior.

Google’s compromised-account and fraudulent-account campaigns; capability distillation is distinguished from stealing original weights. The prompt totals are collection volumes, not measured replication success or model weights. No autonomous agent deployment or protected-reasoning recovery is established.

SecureFlow · Read the full case ↗

Incident ·

Model Distillation Campaigns Targeting Anthropic Claude

Anthropic attributed more than 16 million exchanges through roughly 24,000 fraudulent accounts to DeepSeek, Moonshot AI and MiniMax. The flow distinguishes proxy access, authenticated accounts, distributed routing, inference-response collection and downstream model training. Provider-side hunting and verified limits act before outputs leave; potential harms from models lacking safeguards remain Anthropic’s risk assessment, not observed downstream victim counts.

Why it relates: The campaigns train or improve their own models from provider input-output behavior.

Anthropic’s DeepSeek, Moonshot AI and MiniMax campaigns; capability distillation is distinguished from stealing original weights. The 24,000-account and 16-million-exchange figures do not quantify recovered weights or observed downstream harms. Potential harms remain Anthropic’s assessment; an agent deployment is not established.

SecureFlow · Read the full case ↗

Validated Research · · Practical overlap

Confusing Antimalware Neural Networks

Cloud storage and computations have become popular platforms for deploying ML malware detectors. In such cases, the features for models are built on users' systems and then sent to cybersecurity company servers. The Kaspersky ML research team explored this gray-box scenario and showed that feature knowledge is enough for an adversarial attack on ML models. They attacked one of Kaspersky's antimalware ML models without white-box access to it and successfully evaded detection for most of the adversarially modified malware files.

Why it relates: Adversarial feature knowledge enables gray-box evasion of an antimalware model; a pure black-box threat model is not established. The researchers use target-detector predictions as labels to train a surrogate classifier for attack preparation. That reproduces part of the detector’s behavior from its exposed interface. The source establishes a research surrogate for evasion, not theft of original weights, a full production-quality replica, or a malicious resale campaign.

Cloud storage and computations have become popular platforms for deploying ML malware detectors. In such cases, the features for models are built on users' systems and then sent to cybersecurity company servers. The Kaspersky ML research team explored this gray-box scenario and showed that feature knowledge is enough for an adversarial attack on ML models. They attacked one of Kaspersky's antimalware ML models without white-box access to it and successfully evaded detection for most of the adversarially modified malware files.

SecureFlow · Read the full case ↗

Validated Research ·

ProofPoint Evasion

Proof Pudding (CVE-2019-20634) is a code repository that describes how ML researchers evaded ProofPoint's email protection system by first building a copy-cat email protection ML model, and using the insights to bypass the live system. More specifically, the insights allowed researchers to craft malicious emails that received preferable scores, going undetected by the system. Each word in an email is scored numerically based on multiple variables and if the overall score of the email is too low, ProofPoint will output an error, labeling it as SPAM.

Why it relates: Proof Pudding (CVE-2019-20634) is a code repository that describes how ML researchers evaded ProofPoint's email protection system by first building a copy-cat email protection ML model, and using the insights to bypass the live system. The relevant category is Model Reverse Engineering: An attacker clones or recreates a model by examining its inputs, outputs, and observable behavior.

Proof Pudding (CVE-2019-20634) is a code repository that describes how ML researchers evaded ProofPoint's email protection system by first building a copy-cat email protection ML model, and using the insights to bypass the live system. More specifically, the insights allowed researchers to craft malicious emails that received preferable scores, going undetected by the system. Each word in an email is scored numerically based on multiple variables and if the overall score of the email is too low, ProofPoint will output an error, labeling it as SPAM.

SecureFlow · Read the full case ↗

Validated Research ·

Attack on Machine Translation Services

Machine translation services (such as Google Translate, Bing Translator, and Systran Translate) provide public-facing UIs and APIs. A research group at UC Berkeley utilized these public endpoints to create a replicated model with near-production state-of-the-art translation quality. Beyond demonstrating that IP can be functionally stolen from a black-box system, they used the replicated model to successfully transfer adversarial examples to the real production services. These adversarial inputs successfully cause targeted word flips, vulgar outputs, and dropped sentences on Google Translate and Systran Translate websites.

Why it relates: Machine translation services (such as Google Translate, Bing Translator, and Systran Translate) provide public-facing UIs and APIs. The relevant category is Model Reverse Engineering: An attacker clones or recreates a model by examining its inputs, outputs, and observable behavior.

Machine translation services (such as Google Translate, Bing Translator, and Systran Translate) provide public-facing UIs and APIs. A research group at UC Berkeley utilized these public endpoints to create a replicated model with near-production state-of-the-art translation quality. Beyond demonstrating that IP can be functionally stolen from a black-box system, they used the replicated model to successfully transfer adversarial examples to the real production services. These adversarial inputs successfully cause targeted word flips, vulgar outputs, and dropped sentences on Google Translate and Systran Translate websites.

SecureFlow · Read the full case ↗