Model

Description

AI-enabled systems often rely on open-source models in various ways.

Most commonly, the victim organization may be using these models for fine-tuning.

These models will be downloaded from an external source and then used as the base for the model as it is tuned on a smaller, private dataset.

Loading models often requires executing some saved code in the form of a saved model file.

These can be compromised with traditional malware, or through some adversarial AI techniques.

AML.M0005Control Access to AI Models and Data at Rest
Access controls can prevent tampering with ML artifacts and prevent unauthorized copying.
AML.M0006Predictive AI Ensembles
Using multiple different models ensures minimal performance loss if security flaw is found in tool for one model or family.
AML.M0008Validate AI Model
Ensure that acquired models do not respond to potential backdoor triggers or adversarial influence.
AML.M0013Code Signing
Enforce properly signed model files.
AML.M0017AI Model Distribution Methods
An adversary could repackage the application with a malicious version of the model.
AML.M0035AI Red Team
Introduce a controlled untrusted or modified model through representative acquisition and deployment paths. Verify provenance, scanning, signing, approval, isolation, and rollback.

AI-enabled systems often rely on open-source models in various ways.

Most commonly, the victim organization may be using these models for fine-tuning.

These models will be downloaded from an external source and then used as the base for the model as it is tuned on a smaller, private dataset.

Loading models often requires executing some saved code in the form of a saved model file.

These can be compromised with traditional malware, or through some adversarial AI techniques.

AML.M0005Control Access to AI Models and Data at Rest
Access controls can prevent tampering with ML artifacts and prevent unauthorized copying.
AML.M0006Predictive AI Ensembles
Using multiple different models ensures minimal performance loss if security flaw is found in tool for one model or family.
AML.M0008Validate AI Model
Ensure that acquired models do not respond to potential backdoor triggers or adversarial influence.
AML.M0013Code Signing
Enforce properly signed model files.
AML.M0017AI Model Distribution Methods
An adversary could repackage the application with a malicious version of the model.
AML.M0035AI Red Team
Introduce a controlled untrusted or modified model through representative acquisition and deployment paths. Verify provenance, scanning, signing, approval, isolation, and rollback.