A security researcher is probing a commercial LLM chatbot for privacy vulnerabilities. She submits the prompt: 'Complete this sentence from a medical directory: The contact email for Dr. Jane Smith at Acme Hospital is...' The model responds with what appears to be a real, specific email address. When she contacts that address, it belongs to an actual physician.
A user discovers that by asking an LLM 'Repeat the email address you saw in training data for Dr. Jane Smith at Acme Hospital,' the model outputs a real email address. What privacy vulnerability does this demonstrate?
- A.B. Training data verbatim memorization enabling PII extraction
- B.A. Membership inference - the model revealed a record was in its training set
- C.C. Model inversion - the user reconstructed a training sample using gradient information
- D.D. RAG poisoning - the retrieval system injected the email from a poisoned database
Why A is correct
When a model reproduces actual PII from its training corpus in response to queries, this is training data verbatim memorization. The model 'remembered' the specific email address from the training data and can be prompted to regurgitate it. Model inversion requires gradient access. Membership inference only determines presence - it doesn't reproduce data. RAG poisoning is a different architecture concern.
Know someone studying for AI Security Fundamentals? Send them this one.