CyberCertPrepSecurity FundamentalsWeb Security BasicsEASYFree question
The robots.txt file on a website lists directories that should not be indexed by search engines. Is this a security measure?
- A.No - robots.txt is a suggestion to well-behaved crawlers; attackers can read it to discover interesting directories
- B.It has no effect on anything
- C.Yes, it encrypts directory listings
- D.Yes, it effectively hides sensitive directories; file upload risks disappear once the extension is checked in JavaScript
Why A is correct
robots.txt is NOT a security control - it's a voluntary guideline for search engine crawlers. Attackers often read robots.txt specifically to find 'hidden' directories (admin panels, internal tools). Sensitive resources should be protected with authentication, authorization, and network controls, not by listing them in robots.txt.
Know someone studying for Security Fundamentals? Send them this one.