A content moderation model is being evaluated before deployment. Evaluators deliberately test the model with adversarial examples - rephrased versions of policy-violating content. Why is this adversarial evaluation step important beyond standard accuracy testing?
- A.B. Standard accuracy testing only measures performance on a fixed test set; adversarial evaluation discovers how the model behaves on attacker-crafted inputs that are designed to evade detection - the actual threat model in production
- B.A. Adversarial evaluation measures model inference speed under load
- C.C. Adversarial evaluation is only needed for image models, not text models
- D.D. Standard test sets already include adversarial examples by definition
Why A is correct