AML.T0060 · Publish Hallucinated Entities

Explore 1 source-supported AI security cases for AML.T0060 · Publish Hallucinated Entities in MITRE ATLAS, with evidence, scope, and defensive analysis.

Risk category

Adversaries may create an entity they control, such as a software package, website, or email address corresponding to a source hallucinated by an LLM. The hallucinations may take the form of package names, commands, URLs, company names, or email addresses that point the victim to the entity controlled by the adversary. When the victim interacts with the adversary-controlled entity, the attack can proceed.

Official framework source ↗ · Open interactive explorer ↗

Research ·

ChatGPT Package Hallucination

Researchers identified that large language models such as ChatGPT can hallucinate fake software package names that are not published to a package repository. An attacker could publish a malicious package under the hallucinated name to a package repository. Then users of the same or similar large language models may encounter the same hallucination and ultimately download and execute the malicious package leading to a variety of potential harms.

ATLAS annotations follow the source SecureFlow steps; they are not a MITRE endorsement of cross-framework mappings.

SecureFlow · Read the full case ↗