AI-Enabled Product or Service
Description
Adversaries may use a product or service that uses artificial intelligence under the hood to gain access to the underlying AI model.
This type of indirect model access may reveal details of the AI model or its inferences in logs or metadata.
- ATLAS ID
- AML.T0047
- Maturity
- RealizedUsed by real adversaries against AI systems in the wild.
- Platforms
- Predictive AIGenerative AIAgentic AI
Mitigations
- AML.M0024AI Telemetry Logging
Telemetry logging can help identify if sensitive model information has been sent to an attacker.
Case studies that use it
- AML.CS0003Bypassing Cylance's AI Malware Detection
- AML.CS0004Camera Hijack Attack on Facial Recognition System
- AML.CS0008ProofPoint Evasion
- AML.CS0009Tay Poisoning
- AML.CS0014Confusing Antimalware Neural Networks
- AML.CS0016Achieving Code Execution in MathGPT via Prompt Injection
- AML.CS0017Bypassing ID.me Identity Verification
- AML.CS0026Financial Transaction Hijacking with M365 Copilot as an Insider
- AML.CS0033Live Deepfake Image Injection to Evade Mobile KYC Verification
- AML.CS0034ProKYC: Deepfake Tool for Account Fraud Attacks
- AML.CS0035Data Exfiltration from Slack AI via Indirect Prompt Injection
- AML.CS0036AIKatz: Attacking LLM Desktop Applications
- AML.CS0037Data Exfiltration via Agent Tools in Copilot Studio
- AML.CS0060Cross-Site Scripting via Prompt Manipulation in Lenovo AI Chatbot
- AML.CS0061AI in the Middle: Web-Based AI Services as C2 Relays
- AML.CS0062RCE Vulnerability in Semantic Kernel Search Plugin