A large retailer deploys an AI performance management system that rates employee performance monthly based on productivity data, customer feedback, and schedule adherence. The initial design has the AI automatically generate performance improvement plans (PIPs) for employees rated below a threshold, which are automatically sent to employees' files. HR raises concerns and proposes a mandatory human review step before any PIP is finalized.
A company's AI governance policy requires that any AI system making decisions about employees (performance reviews, promotion recommendations, disciplinary actions) must include a human-in-the-loop review step. Which governance principle does this requirement implement?
- A.A. Data minimization - limiting the amount of data the AI system processes
- B.D. Accountability - assigning responsibility to the AI system for its decisions
- C.C. Transparency - ensuring employees can see how the AI system makes decisions
- D.B. Human oversight - maintaining human ability to review, challenge, and override consequential AI decisions about individuals
Why D is correct
Requiring human review before AI decisions about employees become effective implements the human oversight principle: a human reviewer can identify errors, apply contextual judgment, and override the AI's recommendation before it impacts the employee. This is distinct from transparency (providing explanation) and accountability (assigning responsibility). Data minimization limits data collection, not decision review.
Know someone studying for AI Security Fundamentals? Send them this one.