9.10 · Accidental exposure of unauthorized data to models
Explore 3 source-supported AI security cases for 9.10 · Accidental exposure of unauthorized data to models in Databricks DASF 3.0, with evidence, scope, and defensive analysis.
Risk category
Data sources connected to an AI system expose information that the model should not be authorized to use. Overly broad access to corporate documents, databases, RAG sources, or feature-store lookups can bring confidential material into inference context. The risk can arise through accidental integration or excessive permissions, before any deliberate extraction attempt occurs.
Auth0 disclosed related authorization defects in its JavaScript and Python AI retrieval integrations. A shared FGA object could incorrectly collapse distinct permission checks, allowing denied documents into a model’s context. The relevant boundary is the retriever’s authorization decision before generation.
Why it relates: Denied documents enter the model’s context before generation, so a harmless final answer does not prove correct exclusion.
JavaScript and Python have different collision conditions and fixed package versions; those scopes remain separate.
Cloudflare confirmed that 155 GLM-5.3 responses on Workers AI contained data from other customers’ requests. An oversized prompt was marked for rejection but still reached a separate decode worker, which read stale KV-cache pages. The incident concerns serving-memory isolation; investigators found no deliberate exploitation.
Why it relates: Stale KV state from earlier requests entered a later decode operation despite belonging to other customers.
The incident concerns provider-owned serving-state isolation; customers cannot repair internal allocator state with application prompts or key rotation.
Zenity tested an AWS Bedrock AgentCore deployment, a managed environment for running AI agents, with a network-capable agent and a broad execution role. The agent relayed temporary role credentials from its metadata endpoint. Off-platform API calls then reached other memories and agents. The distinctive persistence path wrote short-term conversation events that configured extraction strategies could turn into long-term records; later retrieval directed the agent to a mutable command page. The reach was demonstrated within an account and region, not as a microVM escape or automatic cross-account compromise. The researchers observed major default-role restrictions on September 29; existing deployed roles still require independent inspection.
Why it relates: Unrestricted tools expose cloud credentials and a private file to a model-mediated interaction.
Controlled broad-role AgentCore deployment with separate direct API and retained-memory/C2 branches; account/region scope, tool prerequisites and later default-role restrictions remain explicit.