A researcher systematically prompts a commercial LLM by starting the first line of famous novels and asking the model to continue. She finds that for approximately 15% of books in her test set, the model continues the passage with text that matches the original novel word-for-word for 50 or more consecutive words - far beyond what chance would explain.
An AI security researcher demonstrates that she can query a large language model with carefully crafted prompts and, in 15% of cases, extract verbatim paragraphs from copyrighted books that appeared in training data. This is an example of:
- A.B. Training data extraction via verbatim memorization
- B.A. Model extraction - she is stealing the model's IP
- C.C. Membership inference - she determined which books were in the training set
- D.D. Model inversion - she reconstructed training examples using gradient information
Why A is correct
Extracting exact text passages from training data through prompting is training data extraction - a direct manifestation of verbatim memorization. This raises both privacy concerns (for personal data) and copyright concerns (for copyrighted works). Model extraction steals model functionality. Membership inference only confirms presence without reproducing content. Model inversion requires gradient access and is more common against classification models.
Know someone studying for AI Security Fundamentals? Send them this one.