Retrieval Content Crafting
Description
Adversaries may write content designed to be retrieved by user queries and influence a user of the system in some way. This abuses the trust the user has in the system.
The crafted content can be combined with a prompt injection. It can also stand alone in a separate document or email. The adversary must get the crafted content into the victim\u0027s database, such as a vector database used in a retrieval augmented generation (RAG) system. This may be accomplished via cyber access, or by abusing the ingestion mechanisms common in RAG systems (see RAG Poisoning).
Large language models may be used as an assistant to aid an adversary in crafting content.
- ATLAS ID
- AML.T0066
- Maturity
- DemonstratedShown to work in research or red-team exercises.
- Platforms
- Generative AIAgentic AI
Mitigations
- AML.M0020Generative AI Guardrails
Apply retrieval guardrails to reject untrusted, malicious, irrelevant, or unsupported RAG content.