MITRE ATLAS
mitre-atlas-cs0009
Tay Poisoning
- Autonomous Agents
- Model Poisoning & Integrity
Microsoft created Tay, a Twitter chatbot designed to engage and entertain users. While previous chatbots used pre-programmed scripts to respond to prompts, Tay's machine learning capabilities allowed it to be directly influenced by its conversations. A coordinated attack encouraged malicious users to tweet abusive and offensive language at Tay, which eventually led to Tay generating similarly inflammatory content towards other users. Microsoft decommissioned Tay within 24 hours of its launch and issued a public apology with lessons learned from the bot's failure.