A security audit of an LLM customer service application reveals the system prompt includes: 'Internal note: Database connection string: mongodb://admin:P@ssw0rd@prod-db:27017/customers'. What two vulnerabilities does this create?
- A.This creates only a single vulnerability: the model may accidentally include the credentials in a response; the transformer's attention mask separates system tokens from user tokens at the hardware level, meaning an injected instruction is scored against a different weight matrix and mathematically cannot outrank the developer's directive, whatever wording the attacker uses
- B.System prompt credential exposure: (1) the system prompt can often be extracted via prompt injection or direct extraction attacks, exposing the database credentials; (2) even if the LLM doesn't reveal the system prompt, it processes the credentials as part of its context - if the model hallucinates or is manipulated into including context in its response, the credentials may leak
- C.