LLM Prompt Injection
Description
An adversary may craft malicious prompts as inputs to an LLM that cause the LLM to act in unintended ways.
These "prompt injections" are often designed to cause the model to ignore aspects of its original instructions and follow the adversary's instructions instead.
Prompt Injections can be an initial access vector to the LLM that provides the adversary with a foothold to carry out other steps in their operation.
They may be designed to bypass defenses in the LLM, or allow the adversary to issue privileged commands.
The effects of a prompt injection can persist throughout an interactive session with an LLM.
Malicious prompts may be injected directly by the adversary (Direct) either to leverage the LLM to generate harmful content or to gain a foothold on the system and lead to further effects.
Prompts may also be injected indirectly when the LLM, as part of its normal operation, ingests the malicious prompt from another data source (Indirect). This type of injection can be used by the adversary to gain a foothold on the system or to target the user of the LLM.
Malicious prompts may also be Triggered by user actions or system events.
- ATLAS ID
- AML.T0051
- Maturity
- RealizedUsed by real adversaries against AI systems in the wild.
- Platforms
- Generative AIAgentic AI
Sub-techniques
Mitigations
- AML.M0019Control Access to AI Models and Data in Production
Use access controls in production to prevent adversaries from injecting malicious prompts.
- AML.M0020Generative AI Guardrails
Guardrails can prevent harmful inputs that can lead to prompt injection.
- AML.M0021Generative AI Guidelines
Model guidelines can instruct the model to refuse a response to unsafe inputs.
- AML.M0022Generative AI Model Alignment
Model alignment can improve the parametric safety of a model by guiding it away from unsafe prompts and responses.
- AML.M0024AI Telemetry Logging
Telemetry logging can help identify if unsafe prompts have been submitted to the LLM.
- AML.M0033Input and Output Validation for AI Agent Components
Validation can prevent adversaries from executing prompt injections that could affect agentic workflows.
- AML.M0035AI Red Team
Test direct, indirect, and triggered instructions through user input, retrieved data, documents, messages, websites, images, metadata, and tool output. Remediate trust-boundary, instruction-handling, permission, and monitoring failures.