AboutWebmailDealsMobile EmailBusiness EmailWeb HostingResourcesHelp and Community
Welcome to the mail2web.com Forums.
+ Reply to Thread
Results 1 to 2 of 2
  1. #1
    Senior Member
    Join Date
    May 2010
    Posts
    670

    Default robots.txt restriction error

    I've had an issue with a robots.txt restriction error recently. Because of it, Google couldn't crawl my URL. What should I do?

  2. #2
    Super Moderator
    Join Date
    Aug 2010
    Location
    Bali
    Posts
    789

    Default

    URLs restricted by robots.txt errors

    Google was unable to crawl the URL due to a robots.txt restriction. This can happen for a number of reasons. For instance, your robots.txt file might prohibit the Googlebot entirely; it might prohibit access to the directory in which this URL is located; or it might prohibit access to the URL specifically. Often, this is not an error. You may have specifically set up a robots.txt file to prevent us from crawling this URL. If that is the case, there's no need to fix this; we will continue to respect robots.txt for this file.
    Details:
    URLs restricted by robots.txt errors - Webmaster Tools Help


 

Similar Threads

  1. MSN IM error
    By CocoaB in forum Troubleshooting
    Replies: 1
    Last Post: 2011-08-17, 09:11 AM
  2. sitemap error
    By suwunk in forum Troubleshooting
    Replies: 1
    Last Post: 2011-08-11, 07:38 AM
  3. what is a parser error?
    By TopDog in forum Troubleshooting
    Replies: 1
    Last Post: 2011-06-27, 08:31 AM
  4. ATT00001.txt
    By miiphone in forum Apple iPhone
    Replies: 3
    Last Post: 2010-11-03, 10:06 AM
  5. iPhone robots
    By TopDog in forum Apple iPhone
    Replies: 0
    Last Post: 2010-09-24, 01:33 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
Back to top