Discover LLM Hallucinations
Description
Adversaries may prompt large language models and identify hallucinated entities.
They may request software packages, commands, URLs, organization names, or e-mail addresses, and identify hallucinations with no connected real-world source. Discovered hallucinations provide the adversary with potential targets to Publish Hallucinated Entities. Different LLMs have been shown to produce the same hallucinations, so the hallucinations exploited by an adversary may affect users of other LLMs.
- ATLAS ID
- AML.T0062
- Maturity
- DemonstratedShown to work in research or red-team exercises.
- Platforms
- Generative AIAgentic AI
Mitigations
- AML.M0004Limit AI Service Query Volume and Rate
Limit repeated prompts used to search for exploitable hallucinations.
- AML.M0020Generative AI Guardrails
Guardrails can help block hallucinated content that appears in model output.
- AML.M0021Generative AI Guidelines
Guidelines can instruct the model to avoid producing hallucinated content.
- AML.M0022Generative AI Model Alignment
Model alignment can help steer the model away from hallucinated content.