Microsys
        

Website Download Crawl Results in URLs with Error Response Codes

Solutions to Error Response Codes

When scanning a complete website, some URLs may result in a standard HTTP error response code, and other URLs may simply error in different manner, e.g. through connect or read timeout. Some examples are 500 : Internal Server Error, 503 : Service Temporarily Unavailable and -4 : CommError. The latter is when complete connection failure happens, e.g. caused by firewalls. Some common solutions are:

Fix and solve the problem at server and/or sitemap website crawler configuration. For more information, read our article about fixing website crawling issues.

Use resume scan functionality. When resuming a website scan, all URLs with some kind of error reponse code will be rechecked.

Webmaster and website software tools


Business and desktop software utilities

Website and webmaster guides


Search engine optimization help

 © Copyright 1997-2012 Microsys