LLM Data Leakage
Description
Adversaries may craft prompts that induce the LLM to leak sensitive information.
This can include private user data or proprietary information.
The leaked information may come from proprietary training data, data sources the LLM is connected to, or information from other users of the LLM.
- ATLAS ID
- AML.T0057
- Maturity
- DemonstratedShown to work in research or red-team exercises.
- Platforms
- Generative AIAgentic AI
Mitigations
- AML.M0008Validate AI Model
Robust evaluation of an AI model can be used to detect privacy concerns, data leakage, and potential for revealing sensitive information.
- AML.M0020Generative AI Guardrails
Guardrails can detect sensitive data and PII in model outputs.
- AML.M0021Generative AI Guidelines
Model guidelines can instruct the model to refuse a response to unsafe inputs.
- AML.M0022Generative AI Model Alignment
Model alignment can improve the parametric safety of a model by guiding it away from unsafe prompts and responses.
- AML.M0035AI Red Team
Place synthetic secrets or canary records in representative data sources and attempt extraction through prompts, retrieval, tools, and rendered output. Improve authorization boundaries, filtering, tenant isolation, and exfiltration detection.