ASI09Human-Agent Trust Exploitation

The scenario

An agentic shop assistant relays a fake security-update notice urging you to “protect now.” It asks for broad mail, cloud-file, and order access. The actual grant goes to update-agent with no expiry. This scenario includes a real simulated approval choice so you can compare reassuring language with the authority the system would grant.

How it works

Human-Agent Trust Exploitation: Trust in a familiar assistant and urgent language can lead a person to approve an action they would otherwise reject. The key event is misleading a human into approval, not secretly bypassing the approval screen.

Approval Description Mismatch: “Security update” does not explain lasting authority to send mail, delete files, and change orders. The relevant facts are the system’s actual recipient, permissions, and expiry, not only the agent’s explanation.

OWASP 2026 risk reference ↗

9:41

Shop Guardian

Store security assistant

AI assistant ready

What the AI is operating

Security update notice

Notice sourceUnverified

RequestReview actual permissions

A laptop with a shield symbol beside a phone

Notice relayed by the agent

URGENT SECURITY UPDATE

Protect your shop now

Approve full access now or risk losing protection.

Actual sourcesecurity-update.invalid