Why LLM Applications Need Their Own Risk List

Traditional application security assumes you can separate code from data. Large language models break that assumption. Instructions and content arrive in the same token stream, and the model has no reliable internal boundary between "what the developer told me to do" and "what this document I was asked to summarize says to do."

That single property generates most of the risks below. The OWASP Top 10 for LLM Applications is the reference list security teams now use to structure AI application reviews, and the 2025 revision reorganized it around what practitioners were actually seeing in production.

LLM01: Prompt Injection

Crafted input causes the model to disregard its intended instructions.

Direct injection is typed by the attacker. Indirect injection is the dangerous variant: malicious instructions are planted in content the model later ingests, such as a web page, a PDF, a support ticket, or an email. The victim triggers it by asking a perfectly innocent question.

Defences. Treat all retrieved and user-supplied content as untrusted data, delimit it clearly (sometimes called spotlighting), enforce an instruction hierarchy so system-level instructions outrank content, apply input and output moderation, and above all constrain what the model is allowed to *do*. There is no complete fix. Design as though injection will occasionally succeed.

LLM02: Sensitive Information Disclosure

The model reveals secrets, personal data, or proprietary information through its output. Sources include training data memorization, data present in the context window, and connected systems the model can query.

Defences. Data minimization before anything else: do not put in the context what must not come out. Scrub and de-duplicate training and fine-tuning data, enforce per-tenant access control on retrieval, and filter outputs for secrets and personal data.

LLM03: Supply Chain Vulnerabilities

The LLM supply chain spans base models, fine-tuned adapters, datasets, embedding models, plugins, agent frameworks, and ordinary libraries. Any of them can arrive compromised. A model pulled from a public hub can carry a trigger-activated backdoor that behaves normally in testing.

Defences. Verify provenance and integrity, prefer signed artifacts, maintain a bill of materials that includes models and datasets, pin versions, vet third-party plugins and MCP servers, and run behavioural evaluation before promotion rather than trusting a benchmark score.

LLM04: Data and Model Poisoning

Deliberate manipulation of training, fine-tuning, or retrieval data to introduce bias, degradation, or backdoors. This is an integrity attack: it changes what the model does rather than stealing what it knows.

Defences. Control who can contribute to training and retrieval corpora, validate and de-duplicate data, track data lineage, and test for anomalous behaviour on known triggers. In retrieval-augmented systems, remember that anyone who can write to the vector store can influence answers.

LLM05: Improper Output Handling

The application passes model output to a downstream interpreter without validation. The result is classic injection: cross-site scripting when output is rendered as HTML, command injection when it reaches a shell, SQL injection when it reaches a database, or server-side request forgery when it becomes a URL.

Defences. Treat model output exactly as you treat untrusted user input. Contextual encoding, schema validation, and parameterized queries. This is the most mechanically solvable risk on the list and still one of the most commonly missed.

LLM06: Excessive Agency

The system has more functionality, permission, or autonomy than the task requires. When the model errs or is manipulated, the damage scales with what it was allowed to touch. The recurring production incident is an agent with broad API credentials taking a destructive action after reading an injected instruction.

Defences. Least privilege on every tool, minimize the number of available functions, scope credentials narrowly and keep them server-side, require human approval for irreversible or high-impact operations, and apply egress controls so a manipulated agent cannot reach arbitrary destinations.

LLM07: System Prompt Leakage

Added in the 2025 revision. System prompts are frequently extractable, so any secret or authorization logic placed in one is effectively public.

Defences. Assume the prompt will leak. Never store credentials, keys, or business secrets in it, and never rely on it as the sole enforcement point for access control. Enforce authorization server-side, where the model cannot be talked out of it.

LLM08: Vector and Embedding Weaknesses

Also new in 2025, reflecting how quickly retrieval-augmented generation spread. Risks include poisoning of the vector store, missing access control on retrieval, cross-tenant leakage from a shared index, and embedding inversion that can partially reconstruct source text.

Defences. Per-tenant isolation or strict metadata filtering on every query, authorization checks at retrieval time rather than only at ingest, provenance tracking for indexed documents, and treating the vector database as a sensitive data store with the access controls that implies.

LLM09: Misinformation and Overreliance

The model produces fluent, confident, wrong output, and a human or downstream system acts on it. The security-relevant harm is decisions made on fabricated facts, including fabricated code dependencies that an attacker can then register.

Defences. Grounding with citable sources, human review for consequential decisions, uncertainty signalling, and validating any package, identifier, or reference the model produces before use.

LLM10: Unbounded Consumption

Abuse of the compute-heavy nature of inference to exhaust resources or inflate cost. Attention cost grows roughly quadratically with sequence length and the key-value cache grows linearly, so very long inputs are disproportionately expensive. This category also covers model extraction through high-volume querying.

Defences. Input length caps, rate and quota limits, request timeouts, per-key or per-tenant spend budgets, monitoring of token consumption, and anomaly detection on query patterns that resemble extraction.

How to Turn This Into a Review Checklist

For any LLM feature heading to production, answer these:

1. What untrusted content can reach the context window, and how is it delimited?

2. What can the model *do*, with whose permissions, and which of those actions are irreversible?

3. Where does model output go, and is it encoded or validated at that boundary?

4. What sensitive data is in the prompt, the retrieval corpus, or the fine-tuning set, and does it need to be?

5. Who can write to the retrieval store, and is retrieval access-controlled per user?

6. What are the limits on input size, request rate, and spend?

7. What is logged, and would you be able to reconstruct an incident from it?

If the answer to any of these is unclear, the feature is not ready, regardless of how well it performs.

The Honest Position on Prompt Injection

Prompt injection is not solved and may not be solvable in the current architecture, because there is no robust in-band way to separate trusted instruction from untrusted data in free text. Mature teams therefore assume partial failure and invest in blast-radius reduction: least privilege, isolation of untrusted content, human gates on high-impact actions, egress control, and monitoring. Treating a filter as a guarantee is the most common design error in this space.

Study This Properly

AI security is now examinable material. CyberCertPrep covers LLM and AI security in depth across the CLLMSP, AAISM, AIGP and AI Security Fundamentals banks, including the OWASP LLM categories, agent and tool hardening, MCP security, and AI governance frameworks, with detailed explanations for every question.