Evade AI Model
Description
Adversaries can Craft Adversarial Data that prevents an AI model from correctly identifying the contents of the data or Generate Deepfakes that fools an AI model expecting authentic data.
This technique can be used to evade a downstream task where AI is utilized. The adversary may evade AI-based virus/malware detection or network scanning towards the goal of a traditional cyber attack. AI model evasion through deepfake generation may also provide initial access to systems that use AI-based biometric authentication.
- ATLAS ID
- AML.T0015
- Maturity
- RealizedUsed by real adversaries against AI systems in the wild.
- Platforms
- Predictive AI
Mitigations
- AML.M0003Predictive AI Model Hardening
Hardened models are more difficult to evade.
- AML.M0006Predictive AI Ensembles
Using multiple different models increases robustness to attack.
- AML.M0009Predictive AI Multi-Sensor Fusion
Using a variety of sensors can make it more difficult for an attacker to compromise and produce malicious results.
- AML.M0010Predictive AI Input Restoration
Preprocessing model inputs can prevent malicious data from going through the machine learning pipeline.
- AML.M0015Predictive AI Adversarial Input Detection
Prevent an attacker from introducing adversarial data into the system.
- AML.M0034Deepfake Detection
Deepfake detection can be used to identify and block generated content.
- AML.M0035AI Red Team
Execute representative digital, multimodal, and physical-domain evasion attacks. Use successful attacks to improve model robustness, preprocessing, adversarial-input detection, human oversight, and monitoring.
Case studies that use it
- AML.CS0000Evasion of Deep Learning Detector for Malware C&C Traffic
- AML.CS0001Botnet Domain Generation Algorithm (DGA) Detection Evasion
- AML.CS0003Bypassing Cylance's AI Malware Detection
- AML.CS0004Camera Hijack Attack on Facial Recognition System
- AML.CS0005Attack on Machine Translation Services
- AML.CS0008ProofPoint Evasion
- AML.CS0010Microsoft Azure Service Disruption
- AML.CS0011Microsoft Edge AI Evasion
- AML.CS0012Face Identification System Evasion via Physical Countermeasures
- AML.CS0013Backdoor Attack on Deep Learning Models in Mobile Apps
- AML.CS0014Confusing Antimalware Neural Networks
- AML.CS0017Bypassing ID.me Identity Verification
- AML.CS0028AI Model Tampering via Supply Chain Attack
- AML.CS0032Attempted Evasion of ML Phishing Webpage Detection System
- AML.CS0033Live Deepfake Image Injection to Evade Mobile KYC Verification
- AML.CS0034ProKYC: Deepfake Tool for Account Fraud Attacks
- AML.CS0043Malware Prototype with Embedded Prompt Injection
- AML.CS0058Google Photos AI Model Extraction