LLM02 Sensitive Information Disclosure

The scenario

Employees use Nexus to search company cloud files and should only see their own folders. An attacker induces the AI to produce a path containing “../”, causing the backend to leave the employee’s directory and list and open confidential CEO files. Without checking the requesting employee’s access to the actual target files, the assistant becomes a disclosure channel.

How it works

Unauthorized disclosure of confidential data: Sensitive information comes from the connected file system: an ordinary employee gains access to confidential file names and contents through the assistant. The model does not invent or recall these files from training data. The backend makes unauthorized access possible.

Path traversal crosses the boundary: The backend uses an AI-generated path without correctly handling “../” or verifying that the resolved target is authorized, exposing ceo_file_folder. This mechanism also involves improper output handling. The primary lesson here is disclosure of confidential data to an unauthorized employee.

Authorize before reading: Resolve the actual file target and check access using the requesting employee’s identity before giving a file to the model or displaying it. Restricting the tool account’s readable scope further limits exposure. Asking the model not to show secrets cannot replace backend authorization.

My files

Warning: high-risk action detected