Robots.txt is a small plain-text file at the root of your website — yoursite.com/robots.txt — that tells search engine crawlers which parts of the site they may visit and which they should leave alone. A couple of lines in it can open your whole blog to Google — or quietly shut the door. One point […]