AAISM · Topic 6
AI Supply Chain
Domain: AI Technologies and Controls, about 38% of the exam
What the AI supply chain contains
- Datasets
- Labeling vendors
- Frameworks and libraries
- Pre-trained models
- Fine-tuning services
- Model APIs
- Plugins and tools
- Pre-trained model
- backdoors, poisoned weights, unknown data
- Open-source library
- vulnerabilities, malicious code, abandonment
- Dataset
- poisoned, unlicensed, unconsented
- Model hub
- impersonation, unverified uploads
- Hosted API
- vendor changes model silently
- Plugin
- excessive agency through third-party tools
- Compute provider
- shared responsibility, residency
OWASP LLM03 covers all of this; every component is an inventory line and a risk register entry
Model hubs and registries
- Verified organizations reduce impersonation
- Private registry mirrors approved models
- Scan every download before use
- Pin exact versions and hashes
- Immutable storage with audit logs
- Behavioral tests before promotion
- Trust but verify, formalized in policy
Serialization risk
- Pickle
- arbitrary code executes on load
- Safetensors
- data only, no code execution
- ONNX
- portable graph, separates model and runtime
- Scanners
- flag suspicious pickle opcodes
- Sandbox loading
- isolate first load of unknown files
- Convert
- re-serialize to safe format
A model file is code until proven otherwise; treat loading like executing a download
SBOM and AI-BOM
- SBOM
- software packages and versions
- AI-BOM adds
- datasets, architecture, hyperparameters, weights
- Model card
- intended use, limits, evaluations
- Datasheet
- dataset composition and collection
- Formats
- SPDX and CycloneDX ML extensions
- Purpose
- vulnerability response across components
Dataset provenance
- Origin, transformations, quality tracked
- Scraped data risks copyright and consent
- Labeling vendors are supply chain links
- Sign and hash approved datasets
- Reproducibility verifies documented training
- Nondeterminism makes exact reproduction hard
Third-party model risk
Before adoption
- Request the model card first
- Assess training data and consent claims
- Check license and usage restrictions
- Evaluate on your own test set
- Scan for backdoors and malicious code
In contract
- Change notification before model swaps
- No training on customer data
- Security attestations and audit rights
- Incident notification timelines
- Exit and replacement plan
In operation
- Pin versions, monitor for drift
- Re-test after every vendor update
- Inventory kept current
- Replacement plan rehearsed
Backdoor detection
- Neural Cleanse reverse-engineers triggers
- Spectral signatures spot poisoned samples
- Activation clustering separates poisoned inputs
- Behavioral testing with adversarial cases
- Fine-pruning removes dormant neurons
- No method is complete, layer them
Dependency attacks
- Dependency confusion
- public package shadows internal name
- Typosquatting
- misspelled package, malicious payload
- Malicious pull request
- subtle backdoor in trusted framework
- Abandoned library
- unpatched vulnerabilities inherited
- Compromised maintainer
- trusted channel ships malware
- Mitigation
- private index, pinning, lockfiles, scanning
SLSA and integrity
- SLSA levels raise build integrity
- Provenance attestation names builder and source
- Signed artifacts with verified distribution
- Chain of trust from root CA
- Reproducible builds cross-check provenance
- Apply the same levels to models
License compliance
- Copyleft can force source release
- Model licenses restrict use cases
- Dataset licenses limit commercial training
- Incompatible licenses create legal exposure
- Track licenses in the AI-BOM
- Output ownership terms vary by API
Standards and regulation
- NIST AI RMF covers third-party risk
- MITRE ATLAS catalogs supply chain techniques
- EU AI Act demands supply chain documentation
- ISO 42001 Annex A: third-party relationships
- OWASP LLM03 names the risk
- Secure development frameworks extend to ML
Key terms
- AI-BOM
- bill of materials for models
- SLSA
- supply chain levels for artifacts
- Pickle
- unsafe Python serialization
- Safetensors
- safe weight format
- ONNX
- open neural network exchange
- Provenance
- verifiable origin record
- Lockfile
- pinned dependency versions
Reference strip: inventory, models, data, dependencies, contracts
Inventory
- AI-BOM beyond the SBOM
- Every component has an owner
- Licenses tracked with versions
- Replacement plan for critical parts
Models
- Model card before anything else
- Safetensors or ONNX, never raw pickle
- Scan, sandbox, test, then promote
- Pin hashes in a private registry
Data
- Provenance and consent verified
- Labelers are vendors too
- Signed, versioned, reproducible
- Scraped data carries IP risk
Dependencies
- Private index beats public lookup
- Lockfiles and pinned digests
- Watch for typosquats and confusion
- SLSA provenance for builds
Contracts
- Change notice before model swaps
- No training on our data
- Audit rights and attestations
- Incident notification timelines
Quick exam traps
- Trap: A model from a popular hub is safe because thousands downloaded it
- Trap: A traditional SBOM already covers datasets and model weights
- Trap: Loading a pickle file only reads data, it cannot run code
- Trap: Passing all accuracy tests proves a model has no backdoor
- Trap: Open-source licenses never restrict commercial model use
- Trap: A hosted model API removes third-party model risk
- Trap: Code review catches every backdoor in a trusted framework
- Trap: Reproducing a training run is straightforward given the code and data
cybercertprep.com · original revision sheet written from the public body of knowledge