Forums

Help! Over 1 Thousand Crawl Errors - 404 Not Found

Permalink
Google WTs reported over 1000 crawl errors on new site upload. The crawl errors appear to be associated with files that I wouldn't expect google to crawl. The errors appear to be originating from the /updates folder.
EXAMPLES:
/updates/concrete5.4.1/
/updates/concrete5.4.1/concrete/blocks/autonav/add.php
/updates/concrete5.4.1/concrete/dispatcher.php

Not sure how to proceed. Could these errors be resolved with a robots.txt command?

Any thought and help is appreciated. Thanks,

Mike
mikedagner
 
jereme replied on at Permalink Reply
jereme
Ensure that the updates folder is in your robots.txt as follows:

Disallow: /updates

I know it's not in mine as my original install predates the updates concept.
mikedagner replied on at Permalink Reply
mikedagner
Thanks, Jereme. I'll try that and see if it works.