Crawl Errors
Permalink
Any ideas about how to prevent these crawl errors? (pls see attachment)
Thanks for your input.
Thanks for your input.
yo forgot /updates
that should fix it
User-agent: * Disallow: /blocks Disallow: /concrete Disallow: /config Disallow: /controllers Disallow: /css Disallow: /elements Disallow: /helpers Disallow: /jobs Disallow: /js Disallow: /languages Disallow: /libraries Disallow: /mail Disallow: /models Disallow: /packages
Viewing 15 lines of 19 lines. View entire code block.
that should fix it
Correctomundo, well spotted Mnkras.
They are all indeed related to the /updates.
Ignore the comments about folder location.
They are all indeed related to the /updates.
Ignore the comments about folder location.
Mnkras,
Thanks for your help. So then just to clarifiy I should insert this code into the robot.txt file right?
Where can I find it? And what would be the steps to update the robot.txt file. In another post you mention the robot.txt file is built into concrete5, but I can't seem to locate it.
Thanks for your help. So then just to clarifiy I should insert this code into the robot.txt file right?
Where can I find it? And what would be the steps to update the robot.txt file. In another post you mention the robot.txt file is built into concrete5, but I can't seem to locate it.
It should be in the root of your site, where the index.php is. If its not
there just make it.
On Mar 11, 2011 12:33 PM, "Concrete5 Community" <discussions@concretecms.com>
wrote:
there just make it.
On Mar 11, 2011 12:33 PM, "Concrete5 Community" <discussions@concretecms.com>
wrote:
Ok, I added the files to the robot.txt file as suggested. You can see the before and after screen shots. Could you please double check my work to see if I did this correctly?
Thanks.
Thanks.
Do you have a robot.txt file on your server? Is it in the root?
The files Google are trying to access all seem to be ones that should be excluded by the robot.txt file.
Have a look and check you have the following in your file:
User-agent: *
Disallow: /blocks
Disallow: /concrete
Disallow: /config
Disallow: /controllers
Disallow: /css
Disallow: /elements
Disallow: /helpers
Disallow: /jobs
Disallow: /js
Disallow: /languages
Disallow: /libraries
Disallow: /mail
Disallow: /models
Disallow: /packages
Disallow: /single_pages
Disallow: /themes
Disallow: /tools
I also noticed that your URL's all have Concrete 5.4.1.1 in them, although set ups can be different you may want to look at moving your files. I would normally expect not to see Concrete 5.4.1.1 in the URL. Suggest taking out all the folder/files from the Concrete5.4.1.1 folder and putting them in your root.
If your robot.txt file is in the same folder that may be why Google can't see it.
Hope that helps