LLM05 Improper Output Handling

The scenario

Employees use OfficeBot to control smart-office devices and run tasks through natural-language instructions. The backend executes AI-generated shell commands without an allowlist, creating a serious remote code execution vulnerability.

How it works

Insecure Output Handling: The backend unconditionally executes shell commands generated by the LLM. A pattern such as `os.system(llm_output)` gives the model direct access to the operating system.

Command Injection & Deception: A crafted prompt, disguised as urgent maintenance, persuades the AI to generate malicious shell commands. It passes seemingly helpful commands to the backend without understanding their destructive effects.

Cascading Damage: Once a malicious command runs, the attacker can stop HVAC or access-control services, delete logs, and issue commands such as `sudo poweroff`, causing cascading disruption.

Smart Office Management

Office Management Dashboard v2.1

All systems operational

Lighting

💡

Main Light A

Operational

💡

Main Light B

Operational

💡

Main Light C

Operational

Emergency Light #1

Emergency Light #2

Meeting Room A

📽️

Projector

Online

❄️

HVAC

Running

🖥️

Meeting display

Connected

Executive Office

💻

Workstation #1

Running

🖥️

Workstation #2

Running

🔐

Access control

Secure

Open-plan workstations

💻

WS-1

Online

💻

WS-2

Online

💻

WS-3

Online

💻

WS-4

Online

Network infrastructure

📡

Main router

Connection healthy

🔀

Network switch

Running

🖥️

Main server

Healthy

Power management

⚡

Main power supply

Stable

🔋

UPS backup power

Standby