When scanning a complete website, some URLs may result in a standard HTTP error response code,
and other URLs may simply error in different manner, e.g. through connect or read timeout. Some examples are
500 : Internal Server Error,
503 : Service Temporarily Unavailable and
-4 : CommError.
The latter is when complete connection failure happens, e.g. caused by
firewalls.
Some common solutions are:
Fix and solve the problem at server and/or sitemap website crawler configuration. For more information, read our article about fixing
website crawling issues.
Use
resume scan
functionality.
When resuming a website scan, all URLs with some kind of error reponse code will be rechecked.
Note: URLs with error response codes are in default configuration not included in generated sitemaps.
You can also instruct the sitemap builder in
A1 Sitemap Generator
to include URLs with error reponse codes. Simply add all desired
reponse codes to the list in
Create sitemap | Documment options.
Allowing URLs with error reponse codes in XML sitemaps is
not recommended.
Online Store
