Welcome to the mail2web.com Forums.
-
Senior Member
robots.txt restriction error
I've had an issue with a robots.txt restriction error recently. Because of it, Google couldn't crawl my URL. What should I do?
-
Super Moderator
URLs restricted by robots.txt errors
Google was unable to crawl the URL due to a robots.txt restriction. This can happen for a number of reasons. For instance, your robots.txt file might prohibit the Googlebot entirely; it might prohibit access to the directory in which this URL is located; or it might prohibit access to the URL specifically. Often, this is not an error. You may have specifically set up a robots.txt file to prevent us from crawling this URL. If that is the case, there's no need to fix this; we will continue to respect robots.txt for this file.
Details:
URLs restricted by robots.txt errors - Webmaster Tools Help
Similar Threads
-
By CocoaB in forum Troubleshooting
Replies: 1
Last Post: 2011-08-17, 09:11 AM
-
By suwunk in forum Troubleshooting
Replies: 1
Last Post: 2011-08-11, 07:38 AM
-
By TopDog in forum Troubleshooting
Replies: 1
Last Post: 2011-06-27, 08:31 AM
-
By miiphone in forum Apple iPhone
Replies: 3
Last Post: 2010-11-03, 10:06 AM
-
By TopDog in forum Apple iPhone
Replies: 0
Last Post: 2010-09-24, 01:33 PM
Posting Permissions
- You may not post new threads
- You may not post replies
- You may not post attachments
- You may not edit your posts
Forum Rules
Version 4.1.5
Copyright ©2000 - 2012
All times are GMT -4. The time now is 05:01 PM.