Unsafe AI Artifacts

Adversaries may develop new AI artifacts or manipulate existing AI artifacts that have a deleterious effect when loaded, executed, interpreted, or used for inference by a victim. The unsafe artifacts may continue to work as expected from the victim's perspective.

Unsafe AI artifacts may include serialized models, model packages, and bundled components such as prompt or chat templates, tokenizer metadata, configuration, pre-processing logic, post-processing logic, or other executable or behavior-shaping metadata.

An unsafe artifact may exploit deserialization or another software vulnerability to execute code on the host. However, exploitation of a software vulnerability is not required. An artifact may instead use functionality intentionally supported by an AI runtime to alter model context, model outputs, safety behavior, or agent actions.

Adversaries may introduce unsafe artifacts through an AI Supply Chain Compromise. Execution may occur when a user loads the artifact, starts an inference runtime, submits an inference request, or otherwise causes the artifact's bundled logic to be processed.