CyberCertPrepAI Security FundamentalsAdversarial ML & Model AttacksEASYFree question
A university lecturer demonstrates that adding a small epsilon perturbation to an input image causes a neural network to classify a cat as a guacamole with 99% confidence. The perturbation is imperceptible to humans. What foundational concept does this illustrate?
- A.That neural networks always fail to generalize beyond their training distribution
- B.The existence of adversarial examples and the brittleness of neural network decision boundaries in high-dimensional input spaces
- C.That convolutional neural networks cannot distinguish between food and animals
- D.That the softmax function produces overconfident predictions for out-of-distribution inputs
Why B is correct
This scenario illustrates the fundamental adversarial examples phenomenon (Szegedy et al. 2014, Goodfellow et al. 2015): neural networks have brittle, high-dimensional decision boundaries where tiny, imperceptible perturbations can cause wildly incorrect predictions with high confidence. This is not a general failure to generalize-the model performs well on clean inputs. It is not specific to food/animal confusion. While softmax overconfidence is a related concept, the core illustration is adversarial brittleness.
Know someone studying for AI Security Fundamentals? Send them this one.