A red team has been contracted to test a bank's ML fraud detection system. They are given a test account with API access to the fraud scoring endpoint. They can submit transaction records and receive a fraud probability score (0.0 to 1.0). They have no access to the model code, architecture documentation, or training data. The vendor has not disclosed which ML framework was used.
A red team is testing a financial fraud detection model by querying its API with varied transaction features and observing the probability scores. They have no access to model weights or training data. What type of adversarial ML attack setting is this?
- A.A. White-box attack - full model knowledge
- B.C. Black-box attack - only input/output access
- C.B. Gray-box attack - partial knowledge of architecture
- D.D. White-hat attack - authorized testing with source code access
Why B is correct
A black-box attack setting means the attacker can only query the model via its API and observe outputs, with no knowledge of model architecture, weights, training data, or hyperparameters. White-box attacks assume full knowledge (weights, gradients). Gray-box attacks assume partial knowledge (architecture but not weights, or vice versa). 'White-hat' is an authorization concept, not a technical attack setting.
Know someone studying for AI Security Fundamentals? Send them this one.