Training Data Poisoning
Description
Adversaries may manipulate data used for training or fine-tuning an AI model to influence the resulting model's behavior. Adversaries may add, remove, or modify data samples; alter labels or annotations; or manipulate feedback and data-collection processes.
Training data poisoning may cause targeted errors, introduce biased or unsafe behavior, degrade model performance, or embed backdoors activated by specific inputs. The resulting behavior may persist in the trained model after the poisoned data is no longer accessible.
Adversaries may poison data directly after gaining access to a training pipeline, or poisoned datasets may be introduced through AI Supply Chain Compromise.
- ATLAS ID
- AML.T0020
- Maturity
- RealizedUsed by real adversaries against AI systems in the wild.
- Platforms
- Predictive AIGenerative AIAgentic AI
Mitigations
- AML.M0001Limit Model Artifact Release
Published datasets can be a target for poisoning attacks.
- AML.M0005Control Access to AI Models and Data at Rest
Access controls can prevent tampering with ML artifacts and prevent unauthorized copying.
- AML.M0007Sanitize Training Data
Detect modification of data and labels which may cause adversarial model drift or backdoor attacks.
- AML.M0008Validate AI Model
Robust evaluation of an AI model can help increase confidence that the model has not been poisoned.
- AML.M0023AI Bill of Materials
An AI BOM can help users identify untrustworthy model artifacts.
- AML.M0025Maintain AI Dataset Provenance
Dataset provenance can protect against poisoning of training data
- AML.M0035AI Red Team
Introduce controlled poisoned records or triggers into representative data pipelines. Verify and improve provenance, sanitization, review, drift detection, model validation, and rollback controls.