Denied by robots.txt error Google Webmaster Tools for WordPress sites
If you see this notice while checking your website’s “Health” in the Google Webmaster interface, you should check a few things:
1. In your WordPress administration panel under “Settings > Privacy”, the Site Visibility should be set to “Allow search engines to index this site.”
2. If you are redirecting your WordPress site from one URL to another using WordPress’s URL re-write feature, make sure that you do not have multiple copies of the same .htaccess file. The .htaccess file in case of a re-direct should be in the folder from where the site URL is getting re-written.
3. Place a robots.txt file in your root folder manually. It will take precedence over the dynamically written robots.txt file created by WordPress.