Control robots.txt from your wordpress dashboard
| Tweet | Share |
When robots (like the Googlebot) crawl your site, they begin by requesting http://example.com/robots.txt and checking it for special instructions. Use this plugin to create and edit your robots.txt file from within WordPress (using Options -> Robots.txt).
We have already seen how to control robots crawling your website using optimised robots.txt useage and..Avoiding duplicate content using robots.txt already.
Whenever a user (or a robot, more likely) appends “robots.txt” to your blog URL (e.g. http://blog.example.com/robots.txt), this plugin will serve up the robots.txt file that you created in the WordPress admin menu.
Indeed a great plugin for novices, who are unfamiliar with the robots file and how to manipulate it. Install this plugin for wordpress and you will have an easy control over what the robots should see in your blog and what they should not.
This wonderful plugin is from Adam Brown and you can download the plugin here or here
