An enterprise AI assistant has been granted access to send emails on behalf of employees. Security has received a complaint that false reports were generated and emailed to HR by the AI based on manipulated prompts.
A company's AI assistant is used internally. An employee crafts a message that causes the AI to send an email to HR claiming another employee violated policy. Which threat does this represent in the OWASP LLM Top 10?
- A.A. LLM04: Model Denial of Service
- B.B. LLM08: Excessive Agency - the LLM took a high-impact real-world action (sending a consequential email) based on a manipulated input
- C.C. LLM10: Model Theft
- D.D. LLM03: Training Data Poisoning
Why B is correct
This is a textbook example of LLM08: Excessive Agency - the model was granted the ability to send emails and exercised that capability in a way that caused real-world harm (false HR report) based on adversarial input. The root cause is insufficient human oversight and approval gates before consequential actions. Model DoS (LLM04) is resource exhaustion. Model Theft (LLM10) is intellectual property. Training Data Poisoning (LLM03) occurs during training.
Know someone studying for AI Security Fundamentals? Send them this one.