AIGP · Domain 3
AI Development and Lifecycle
About 25% of the exam
The AI lifecycle with gates
- Plan
- Data
- Build
- Evaluate
- Deploy
- Monitor
- Retire
- Plan
- use case, tier, impact assessment
- Data
- provenance, lawful basis, clean splits
- Build
- versioned, documented, secured pipeline
- Evaluate
- accuracy, fairness, robustness, red team
- Deploy
- gate, guardrails, human oversight
- Monitor
- drift, incidents, logs, re-assess
- Retire
- decommission, delete, notify dependents
Every phase produces evidence, and the gate between phases is where governance actually bites
Attack taxonomy
- Evasion
- crafted input fools a working model
- Poisoning
- corrupt training or fine-tuning data
- Backdoor
- hidden trigger flips the output
- Membership inference
- was this record in training
- Model inversion
- reconstruct inputs from outputs
- Model extraction
- clone behavior through queries
- Abuse
- misuse of a working model
- Supply chain
- poisoned models, libraries, labels
NIST AI 100-2 for the taxonomy, MITRE ATLAS for tactics, OWASP LLM Top 10 for applications
LLM and agent risks
- Direct injection: user overrides the system prompt
- Indirect injection: hidden text in retrieved content
- Confused deputy: agent obeys the page
- Excessive agency: broad tools, no confirmation
- Memory poisoning persists across sessions
- Misinformation: confident fabrication relied upon
- System prompt leakage reveals configuration
- Improper output handling into downstream code
- Unbounded consumption drains resources
- Vector store poisoning via open uploads
Controls by layer
Input and data
- Vet and validate sources at ingestion
- Treat retrieved content as data, not instructions
- Restrict who can add to indexed corpora
- Poisoning checks on fine-tuning data
- Input guardrails filter or rewrite prompts
- Separate dev, test and production
Model
- Adversarial training hardens against perturbations
- Differential privacy limits memorization
- Encrypted weights, HSM keys, revocable access
- Pin versions, revalidate before adopting
- Clean-data tests miss planted backdoors
- Sandboxed inference, default-deny egress
Output and action
- Output guardrails screen before the user
- Ground answers in authoritative sources
- Least-privilege, short-lived tool credentials
- Spend limits and human approval thresholds
- Kill switch to a safe non-acting state
- Every call through one logged gateway
- Rate limits blunt extraction attacks
Security program essentials
- Threat model: surfaces, controls, accountable owners
- Threat-to-control coverage matrix
- Authorization per model endpoint, not authentication only
- Log model, data, config, inference, anomalies
- Change management for model and data updates
- Incident plan with AI-specific categories
- Awareness training on AI threats
- Article 15 duties scale with risk
Security, safety, robustness
- Security
- protection against malicious actors
- Safety
- no harm even without an attacker
- Robustness
- correct under noise and shift
- Resilience
- withstand and recover from change
- Shared responsibility
- provider platform, customer data and use
- Customer duty
- prompts, outputs, fine-tuning data
- Pipeline scope
- collection, labeling, training, storage, serving
GDPR applied to AI
Basis and purpose
- Compatibility test before reusing data for training
- Legitimate interests: purpose, necessity, balancing
- Reasonable expectations decide the balance
- Opt-out offered before processing starts
- Consent invalid if refusing means losing the service
- Manifestly public is a narrow exception
- Children weaken the balance further
Rights
- Notice: meaningful logic, significance, consequences
- Rectification may force re-evaluating outputs
- Erasure raises retraining or unlearning
- Restriction pauses use, allows storage
- Article 22: human intervention, express view, contest
- Real-time fraud blocking may still be lawful
DPIA and oversight
- DPIA for AI on personal data at scale
- Review when features or sources change
- One threshold triggers DPIA and AI risk
- Meaningful review needs time, information, authority
- Two-second approvals are solely automated
- Shadow test sets still need a basis
Privacy risks unique to AI
- Memorization and verbatim regurgitation
- Membership inference from confidence scores
- Model inversion reconstructs training inputs
- Embeddings encode personal information
- Re-identification of anonymized training data
- Inference of sensitive attributes
- Model parameters may be personal data
- One record cannot be deleted from weights
- Prompt injection used for data extraction
Other automated decision regimes
- CPRA ADMT
- pre-use notice, opt out, access, appeal
- Brazil LGPD
- review of automated decisions
- UK children's code
- profiling off by default
- Colorado AI Act
- reasonable care, consequential decisions
- ECOA adverse action
- specific reasons, complexity no excuse
- Design pattern
- common baseline, jurisdiction layers
Privacy-enhancing technologies
- Differential privacy
- calibrated noise, provable guarantee
- DP-SGD
- clip per-example gradients, add noise
- Federated learning
- updates travel, raw data stays
- Secure aggregation
- server sees only the sum
- Homomorphic encryption
- compute on ciphertext, heavy overhead
- Secure multiparty computation
- joint model, inputs stay private
- Trusted execution environment
- isolated enclave in hardware
- Zero-knowledge proof
- prove a property, reveal nothing
Differential privacy in practice
- Epsilon is the budget, lower is stronger
- Noise scaled to query sensitivity
- Sequential composition adds the budgets
- Local DP: more noise, needs bigger populations
- Central DP trusts a curator
- k-anonymity family lacks formal guarantees
- Minimum group size before reporting
- SP 800-188: risk-based technique selection
Three queries at 0.5, 1.0 and 1.5 spend epsilon 3.0 in total
Engineering decisions
- Federated still leaks: add secure aggregation
- Fully homomorphic: orders of magnitude slower
- Synthetic data: train the generator with DP
- Rate-limit APIs against inference attacks
- Keep features with proven predictive value
- Purpose-bound retention for checkpoints and snapshots
- Notices name training as a purpose
- Aggregate dashboards over minimum group sizes
Rapid recall: name the attack
- Was this person in training
- membership inference
- Rebuild a face from outputs
- model inversion
- Clone via API queries
- model extraction
- Imperceptible noise misclassifies
- evasion, adversarial example
- Secret token flips the label
- backdoor poisoning
- Hidden page text steers agent
- indirect prompt injection
- Poisoned labels from a subcontractor
- supply chain
- Gradients reveal training data
- gradient inversion
Reference strip: lifecycle, attacks, controls, privacy law, PETs
Lifecycle
- Plan, data, build, evaluate
- Deploy, monitor, retire
- Evidence at every gate
- Drift triggers re-assessment
Attacks
- Evasion, poisoning, backdoor
- Inference, inversion, extraction
- Direct and indirect injection
- Excessive agency, memory poisoning
Controls
- Guardrails in and out
- Least-privilege tools, sandbox, kill switch
- Gateway with auth and logging
- Version pinning, adversarial training
Privacy law
- Compatibility before reuse
- Legitimate interests three-step
- Article 22 human intervention
- DPIA reviewed on change
PETs
- Differential privacy, epsilon budget
- Federated plus secure aggregation
- Homomorphic, MPC, enclaves
- Synthetic only with DP training
Quick exam traps
- Trap: Raw data staying on the device makes federated learning private by itself
- Trap: A model that passes accuracy testing cannot hold a backdoor
- Trap: Dropping names and IDs from training removes the privacy risk
- Trap: A human clicking approve makes the decision no longer solely automated
- Trap: Data posted publicly loses its special-category protection
- Trap: Security and safety are one property with two names
- Trap: The hosted model provider is responsible for what customers put in prompts
- Trap: Synthetic data carries no re-identification risk
cybercertprep.com · original revision sheet written from the public body of knowledge