Poison AI Model
Sub-technique of AML.T0018 Manipulate AI Model
Description
Adversaries may manipulate an AI model's weights to change it's behavior or performance, resulting in a poisoned model.
Adversaries may poison a model by directly manipulating its weights, training the model on poisoned data, further fine-tuning the model, or otherwise interfering with its training process.
The change in behavior of poisoned models may be limited to targeted categories in predictive AI models, or targeted topics, concepts, or facts in generative AI models, or aim for a general performance degradation.
- ATLAS ID
- AML.T0018.000
- Maturity
- DemonstratedShown to work in research or red-team exercises.
- Platforms
- Predictive AIGenerative AIAgentic AI
Mitigations
- AML.M0005Control Access to AI Models and Data at Rest
Access controls can prevent tampering with ML artifacts and prevent unauthorized copying.
- AML.M0007Sanitize Training Data
Prevent attackers from leveraging poisoned datasets to launch backdoor attacks against a model.
- AML.M0008Validate AI Model
Ensure that trained models do not respond to potential backdoor triggers or adversarial influence.
- AML.M0013Code Signing
Code signing provides a guarantee that the model has not been manipulated after signing took place.
- AML.M0025Maintain AI Dataset Provenance
Dataset provenance can protect against poisoning of models.
- AML.M0035AI Red Team
Test whether controlled changes to model weights, fine-tuning, or associated artifacts can introduce targeted or persistent behavior. Improve model provenance, validation, integrity monitoring, and rollback.