Discover AI Model Ontology
Description
Adversaries may discover the ontology of an AI model's output space, for example, the types of objects a model can detect.
The adversary may discover the ontology by repeated queries to the model, forcing it to enumerate its output space.
Or the ontology may be discovered in a configuration file or in documentation about the model.
The model ontology helps the adversary understand how the model is being used by the victim.
It is useful to the adversary in creating targeted attacks.
- ATLAS ID
- AML.T0013
- Maturity
- DemonstratedShown to work in research or red-team exercises.
- Platforms
- Predictive AI
Mitigations
- AML.M0002Predictive AI Output Obfuscation
Suggested approaches:
- Restrict the number of results shown
- Limit specificity of output class ontology
- Use randomized smoothing techniques
- Reduce the precision of numerical outputs
- AML.M0004Limit AI Service Query Volume and Rate
Limit inference queries to reduce an adversary's ability to discover the model's full output ontology.