CyberCertPrepSecurity FundamentalsWeb Security BasicsEASYFree question
The robots.txt file on a website lists directories that should not be indexed by search engines. Is this a security measure?
- A.A. Yes, it effectively hides sensitive directories
- B.D. It has no effect on anything
- C.C. Yes, it encrypts directory listings
- D.B. No - robots.txt is a suggestion to well-behaved crawlers; attackers can read it to discover interesting directories
Why D is correct
robots.txt is NOT a security control - it's a voluntary guideline for search engine crawlers. Attackers often read robots.txt specifically to find 'hidden' directories (admin panels, internal tools). Sensitive resources should be protected with authentication, authorization, and network controls, not by listing them in robots.txt.
Know someone studying for Security Fundamentals? Send them this one.