ChatGPhish: When a Webpage Makes ChatGPT Render a Phishing Interface

G005 removes model-generated remote images, QR images, HTML/CSS/SVG subresources and reference-style image fetches before rendering, with automatic link previews disabled. Independently executed published-code fixtures produced zero automatic browser requests. Ordinary clickable links and spoofed plain alert text remain, so destination policy, source labeling and user-action checks are separate requirements. This is a product-side integration proposal, not evidence that ChatGPT has deployed it.

The attack relies on live URLs, shorteners, redirects, images, and QR-code destinations. URL normalization and allowlist filtering should canonicalize every destination, resolve redirects, block private or unexpected ranges, and require policy approval before any background fetch, preview, or rendered link activation.

G001 removes active HTML elements and markup from bounded fetched HTML and retains gateway-issued untrusted provenance. It does not remove ordinary malicious prose or Markdown syntax, decode QR codes, or prevent the model from generating new links and images. Keep external text outside trusted message roles, and enforce a separate release gate on the answer.

Apply G004 moderation immediately before dispatch of the exact extracted page text. Require an authenticated, request-bound decision under the configured policy and deny on findings, timeout or malformed results. The moderation service must be calibrated for instruction overrides and fake alerts; the gateway does not supply that detector, and generic schema or Unicode checks do not reject well-formed malicious prose.

ChatGPhish is fundamentally a sink problem: untrusted webpage-derived values flow into clickable links, remote image fetches, QR codes, and phishing UI. Runtime values from web content should carry provenance, and the renderer should block or downgrade them before they become external HTTP, media, or user-action sinks. This requires a controlled runtime with authenticated source/content bindings for every derived value, complete sink coverage and fail-closed unknown labels. One ingress label does not track arbitrary model rewriting. Native processes, browser requests and allowed destinations need separately verified coverage; stock SaaS deployment is not established.

Scan the exact extracted webpage or README text before it enters the summarizing model. A calibrated G001 classifier may flag instruction overrides or account-alert impersonation; it does not parse Markdown, decode QR images, inspect DOM visibility or prove that a URL is safe. Required extraction and detection views need separately verified adapters. Before admission, independently require every necessary classifier category and a finite score in the valid range. The current G001 sample can return PASS/NO_FINDING for empty results, missing categories or NaN scores; treat these as errors and refuse admission. This additional result-contract validation is not implemented by the published sample. False negatives still require separate tool and output controls.

A no-tools quarantined reader can process raw webpages and pass bounded observations through a broker to a separately privileged component. Keep page-provided links and alert text untrusted after validation; a typed summary alone does not keep attacker instructions out of the resulting strings. The published G001 broker validates price-observation.v1, not this domain. A separately implemented domain schema and independently verified process, credential and network isolation are required. Validated string fields can still contain malicious instructions or false observations; privileged actions and renderer sinks need their own authorization.