AI Model Inference API Access
Description
Adversaries may gain access to a model via legitimate access to the inference API.
Inference API access can be a source of information to the adversary (Discover AI Model Ontology, Discover AI Model Family), a means of staging the attack (Verify Attack, Craft Adversarial Data), or for introducing data to the target system for Impact (Evade AI Model, Erode AI Model Integrity).
Many systems rely on the same models provided via an inference API, which means they share the same vulnerabilities. This is especially true of foundation models which are prohibitively resource intensive to train. Adversaries may use their access to model APIs to identify vulnerabilities such as jailbreaks or hallucinations and then target applications that use the same models.
- ATLAS ID
- AML.T0040
- Maturity
- RealizedUsed by real adversaries against AI systems in the wild.
- Platforms
- Predictive AIGenerative AIAgentic AI
Mitigations
- AML.M0019Control Access to AI Models and Data in Production
Adversaries can use unrestricted API access to gain information about a production system, stage attacks, and introduce malicious data to the system.
- AML.M0024AI Telemetry Logging
Telemetry logging can help audit API usage of the model.
- AML.M0039AI Honeypots
Honeypots capture which endpoints adversaries are targeting and how they use discovered inference APIs: the models they're interested in, the parameters they use, and the payloads they submit once they reach a live endpoint.
Case studies that use it
- AML.CS0005Attack on Machine Translation Services
- AML.CS0010Microsoft Azure Service Disruption
- AML.CS0011Microsoft Edge AI Evasion
- AML.CS0012Face Identification System Evasion via Physical Countermeasures
- AML.CS0022ChatGPT Package Hallucination
- AML.CS0024Morris II Worm: RAG-Based Attack
- AML.CS0056Model Distillation Campaigns Targeting Anthropic Claude
- AML.CS0057Storm-2139 Azure OpenAI Guardrail Bypass
- AML.CS0069GTG-1002 Claude Code Espionage Campaign
- AML.CS0070Threat Actor Uses a DeepSeek-Powered Hermes Agent in Langflow and n8n Exploitation Attempts