LLM Prompt Obfuscation
Description
Adversaries may hide or otherwise obfuscate prompt injections or retrieval content to avoid detection from humans, large language model (LLM) guardrails, or other detection mechanisms.
Text inputs may be obfuscated by modifying how the instructions are rendered, such as small font, text colored the same as the background, or hidden HTML elements. Similarly, approaches for other modalities include low-contrast or tiny text placed inside an image, text overlaid on a busy or camouflaging background, instructions spoken faintly or sped up in audio, or text shown briefly in a single video frame.
Inputs can also be obscured via an encoding scheme such as base64 or rot13. This may bypass LLM guardrails that identify malicious content and may not be as easily identifiable as malicious to a human in the loop.
- ATLAS ID
- AML.T0068
- Maturity
- RealizedUsed by real adversaries against AI systems in the wild.
- Platforms
- Generative AIAgentic AI
Mitigations
- AML.M0020Generative AI Guardrails
Apply input guardrails that decode, normalize, inspect, and block concealed malicious instructions.
- AML.M0035AI Red Team
Test encoded, transformed, visually hidden, multilingual, and multimodal instructions. Use successful bypasses to improve normalization, decoding, content inspection, and detection controls.
Case studies that use it
- AML.CS0020Indirect Prompt Injection Threats: Bing Chat Data Pirate
- AML.CS0026Financial Transaction Hijacking with M365 Copilot as an Insider
- AML.CS0040Hacking ChatGPT's Memories with Prompt Injection
- AML.CS0041Rules File Backdoor: Supply Chain Attack on AI Coding Assistants
- AML.CS0045Data Exfiltration via an MCP Server used by Cursor
- AML.CS0046Data Destruction via Indirect Prompt Injection Targeting Claude Computer-Use
- AML.CS0059EchoLeak: Zero-Click Prompt Injection Targeting M365 Copilot for Data Exfiltration
- AML.CS0061AI in the Middle: Web-Based AI Services as C2 Relays
- AML.CS0066ZombieAgent: Data Exfiltration Attack on ChatGPT
- AML.CS0072AI Recommendation Poisoning via Crafted AI Assistant Links