Verify Attack
Description
Adversaries can verify the efficacy of their attack via an inference API or access to an offline copy of the target model.
This gives the adversary confidence that their approach works and allows them to carry out the attack at a later time of their choosing.
The adversary may verify the attack once but use it against many edge devices running copies of the target model.
The adversary may verify their attack digitally, then deploy it in the Physical Environment Access at a later time.
Verifying the attack may be hard to detect since the adversary can use a minimal number of queries or an offline copy of the model.
- ATLAS ID
- AML.T0042
- Maturity
- DemonstratedShown to work in research or red-team exercises.
- Platforms
- Predictive AIGenerative AIAgentic AI
Mitigations
- AML.M0002Predictive AI Output Obfuscation
Obfuscating model outputs reduces an adversary's ability to verify the efficacy of an attack.
- AML.M0004Limit AI Service Query Volume and Rate
Limit repeated queries used to test and refine an attack against the target model.
- AML.M0005Control Access to AI Models and Data at Rest
Access controls on models at rest can prevent an adversary's ability to verify attack efficacy.
- AML.M0019Control Access to AI Models and Data in Production
Use access controls in production to prevent adversary's ability to verify attack efficacy.
Case studies that use it
- AML.CS0000Evasion of Deep Learning Detector for Malware C&C Traffic
- AML.CS0001Botnet Domain Generation Algorithm (DGA) Detection Evasion
- AML.CS0010Microsoft Azure Service Disruption
- AML.CS0013Backdoor Attack on Deep Learning Models in Mobile Apps
- AML.CS0014Confusing Antimalware Neural Networks
- AML.CS0016Achieving Code Execution in MathGPT via Prompt Injection
- AML.CS0019PoisonGPT