Google Webmaster Tools Webcrawl Diagnostics and Errors

Google webcrawl diagnostics tool can sometimes report odd website crawling errors, e.g. error 404 - not found. These error are often caused by external factors not related to XML sitemaps.

Help: overview | previous | next

To see all the options available, you will have to switch off easy mode

With options that use a dropdown list, any [+] or [-] button next to adds or removes items in the list itself

Webmaster Tools Show Confusing Webcrawl Errors

Many people submit generated XML sitemaps directly to Google Webmaster Tools. However, when viewing Webcrawl Diagnostics, they see Google report mysterious website crawling problems.

Warnings or Errors Shown for Generated XML Sitemaps

When viewing the list of submitted sitemaps, the webmaster tools shows if any of them have warnings and/or errors.

Note: Only errors will cause problems for Google when crawling your XML sittemap file.

It is worth nothing that the red flag graphic is used for both critical errors and warnings.

However, it is possible to see the exact error message that causes the sitemap error/warning flag to be shown. To see how, check the underneath screenshot taken from the webmaster sitemap tools.

google tools sitemaps list

404 Not Found Errors for URLs

An error often reported for URLs is 404 : Not Found. You may get such errors even if your website has no internal linking errors, and your XML sitemap is 100% correct.

google tools diagnostics webcrawl

Google follows links from websites all over the internet. Google remembers these links for quite a long time. In our experience, this means that if some website at some point in time has linked to a non-existing page at your website, Google will keep reporting this as an error for a long time.

Examples of how incorrect links to your website can occur on external websites:

Someone posted and mistyped a link to your website on a forum.
You posted a link on a website which incorrectly shortened it.

301/302 Redirect Errors in XML Sitemap

Your generated XML sitemaps can not contain URLs that redirect if you want Google sitemap tools to accept them. This includes error codes 301 : Moved Permanently, 302 : Found and 302 : Moved Temporarily.

A1 Sitemap Generator will never include URLs that redirect unless explicitly configured so. Instead it will include the redirect destination URLs.

A1 Sitemap Generator | help | previous | next

Build all kinds of sitemaps including text, visual HTML / CSS, RSS, XML, image, video, news and mobile for all your websites no matter the platform they use.

This help page is maintained by Thomas Schulz

As one of the lead developers, his hands have touched most of the code in the software from Microsys. If you email any questions, chances are that he will be the one answering.