CyberCertPrepAI Security FundamentalsAI & ML Concepts for SecurityEASYFree question
A security engineer wants to understand LLM token billing to assess the cost impact of a denial-of-wallet attack. How are tokens related to text length, and what is an approximate rule of thumb for English text?
- A.B. Tokens are sub-word units; approximately 1 token equals 0.75 words or about 4 characters in English, meaning 1,000 words is roughly 1,333 tokens
- B.A. One token always equals exactly one word in English text
- C.C. One token equals exactly one sentence in LLM billing
- D.D. Token count equals the number of unique words in the input (vocabulary size)
Why A is correct
LLMs use subword tokenization (e.g., BPE, SentencePiece) where common words may be single tokens and rare or long words split into multiple tokens. The approximate rule of thumb is 1 token ≈ 4 characters or ~0.75 words in English. So 1,000 tokens ≈ 750 English words. This matters for: context window management (fitting content within limits), cost estimation (APIs charge per input+output token), and denial-of-wallet attacks (crafting maximum-length inputs to maximize per-request charges). Non-English text and code often produce different token/word ratios.
Know someone studying for AI Security Fundamentals? Send them this one.