An enterprise AI assistant has been granted access to send emails on behalf of employees. Security has received a complaint that false reports were generated and emailed to HR by the AI based on manipulated prompts.
A company's AI assistant is used internally. An employee crafts a message that causes the AI to send an email to HR claiming another employee violated policy. Which threat does this represent in the OWASP LLM Top 10 (2023)?
- A.LLM03: Training Data Poisoning
- B.LLM04: Model Denial of Service; rate limiting defeats prompt injection outright, because crafted prompts require thousands of attempts to land once
- C.LLM10: Model Theft
- D.LLM08: Excessive Agency - the LLM took a high-impact real-world action (sending a consequential email) based on a manipulated input
Why D is correct
This is a textbook example of LLM08: Excessive Agency - the model was granted the ability to send emails and exercised that capability in a way that caused real-world harm (false HR report) based on adversarial input. The root cause is insufficient human oversight and approval gates before consequential actions. Model DoS (LLM04) is resource exhaustion. Model Theft (LLM10) is intellectual property. Training Data Poisoning (LLM03) occurs during training.
Know someone studying for AI Security Fundamentals? Send them this one.