Invert AI Model
Sub-technique of AML.T0024 Exfiltration via AI Inference API
Description
AI models' training data could be reconstructed by exploiting the confidence scores that are available via an inference API.
By querying the inference API strategically, adversaries can back out potentially private information embedded within the training data.
This could lead to privacy violations if the attacker can reconstruct the data of sensitive features used in the algorithm.
- ATLAS ID
- AML.T0024.001
- Maturity
- FeasibleShown to be possible in principle (proof of concept).
- Platforms
- Predictive AIGenerative AIAgentic AI
Mitigations
- AML.M0002Predictive AI Output Obfuscation
Suggested approaches:
- Restrict the number of results shown
- Limit specificity of output class ontology
- Use randomized smoothing techniques
- Reduce the precision of numerical outputs
- AML.M0004Limit AI Service Query Volume and Rate
Limit the volume of API queries in a given period of time to regulate the amount and fidelity of potentially sensitive information an attacker can learn.
- AML.M0024AI Telemetry Logging
Telemetry logging can help identify if sensitive data has been exfiltrated.
- AML.M0035AI Red Team
Attempt to reconstruct sensitive records, attributes, or representative training information from model outputs. Remediate leakage through model changes, output minimization, privacy controls, and restricted inference access.
Case studies that use it
No ATLAS case study records this technique yet. Browse all case studies.