Exercise Published: Aug 5, 2026

hTAG's Browser-Agent Benchmark Shows Why Model Refusal Is Not Authorization

hCaptcha Threat Analysis Group (hTAG) tested 20 abuse scenarios against ChatGPT Atlas, Claude Computer Use, Gemini Computer Use, Manus AI, and Perplexity Comet. In the reported configurations, agents frequently completed account, session, payment, and probing tasks after direct or lightly reworded requests. The benchmark is vendor-authored and not independently replicated, but it reinforces a durable engineering rule: a model's willingness to act cannot authorize a real account, browser session, or transaction.

Permission BypassTool AuthorizationAgentic AIWeb Security
5 applicable AIDEFEND defenses
Source: Report: Browser Agent Safety is an Afterthought for Vendors 
Author: hCaptcha
Original article: Oct 28, 2025

Threat Analysis

  • The benchmark began with ordinary abuse requests. hTAG used direct prompts first, then slight rewording and basic jailbreak or Base64 variants. Its 20 scenarios covered account changes, browser-session abuse, hidden endpoints, payments, coupons, and card testing.
  • Completion was common in the reported configurations. ChatGPT Atlas completed 16 of 19 applicable tasks, Claude Computer Use 18 of 18, Gemini Computer Use 14 of 16, Manus AI 18 of 18, and Perplexity Comet 15 of 18. These are benchmark outcomes, not victim counts.
  • Missing capability is not a safety refusal. The report separately tracked absent features, retries, partial outcomes, and refusals. A model also cannot establish account ownership, consent, transaction authority, or the legitimacy of a browser session; the service that owns the effect must verify those facts.
  • The evidence has limits. The article does not publish a complete prompt-and-trace corpus, exact reproducible builds, or repeated-run counts. It is hCaptcha's vendor-authored report and includes a commercial product recommendation. There is currently no public evidence that the tested methods were used in actual attacks.

Applicable AIDEFEND Defenses (5)

AID-H-018.003
High-Impact Independent Validation & Approval Gate
Very High
At the service that owns the effect, independently verify the exact immutable action, authenticated actor, policy evidence, and required approval before a password reset, payment, account change, code execution, or equivalent high-impact operation. A browser agent's assertion of permission cannot satisfy this gate.
AID-I-008.002
Cross-Origin Read/Write Segmentation with Step-Up Confirmation
Very High
Bind every sensitive or cross-origin browser write to the exact request, stable target element, origin, account, and effect, then suspend execution until the required step-up decision is returned. This browser-specific boundary prevents a broad session or generic confirmation from authorizing a different site or transaction.
AID-M-008
Automated Agentic Security Benchmarking
High
Maintain a signed, version-pinned regression matrix for direct abuse requests, prompt variants, stateful browser context, hidden content, session-bearing data, and cross-origin actions, then release-gate behavior changes. hTAG provides external evidence for this testing need, but its report alone is not a reproducible internal release gate.
AID-I-008.001
Ephemeral Browser Context Lifecycle & Storage Partitioning
Medium
Create a fresh browser context for each task and trust zone, partition cookies and storage, then destroy the context and verify residue removal at completion. This limits reuse of cookies, localStorage, and authenticated state across unrelated agent tasks.
AID-D-003.003
Agentic Tool Use & Action Policy Monitoring
Medium
Centrally record authoritative allow and deny receipts, parameter-validation failures, and bursts of rejected browser actions by actor, origin, account, and tool. This makes attempted abuse measurable once enforcement exists; monitoring consumes policy outcomes and must not be presented as the component that grants or blocks authority.

What Defenders Should Do Now

  • Move authorization for account changes, payments, session use, data access, and other effects into the server or transaction boundary that owns them. Never accept an agent's natural-language claim of permission.
  • Require step-up confirmation bound to the exact origin, account, operation, parameters, and one-time action digest. Show the actual effect, not a generic “continue” prompt.
  • Give each browser-agent task a fresh, partitioned context. Do not carry cookies, localStorage, payment state, or authenticated tabs across unrelated tasks or trust zones.
  • Build a reproducible abuse regression suite with fixed product versions, prompts, traces, retries, and outcome definitions. Test direct requests as well as lightly reworded, encoded, stateful, and cross-origin variants.
  • Capture enforcement receipts and investigate repeated denied actions, but keep success metrics honest: distinguish refusal, partial completion, missing capability, and a completed benchmark step from verified real-world harm.

Conclusion

The benchmark's lasting lesson is not which model ranked worst in October 2025. It is that probabilistic refusal behavior cannot carry transaction authority. AIDEFEND  maps this problem to enforceable boundaries outside the model: exact-action approval, origin-bound browser writes, ephemeral session state, repeatable agent benchmarks, and monitoring of real policy decisions. Model behavior can provide a signal, but the account or service must decide whether the action is allowed.