XML Sitemap Indexed URLs in Google Webmaster Tools
Microsys
  

XML Sitemap Indexed URLs in Google Webmaster Tools

When deleting and adding a new XML sitemap in Google webmaster tools, the indexed URLs count shown in is reset. In actuality, your URLs are all still indexed.
Help: overview | previous | next

Indexed URLs in XML Sitemaps

Some people get puzzled when they delete an old XML sitemap and later submit a new. Google Webmaster Tools will always for newly submitted sitemaps report indexed URLs as non-existing or quite low. Given some time, the count usually grows, but how fast depends on the website.

We can conclude this number does not represent the count of indexed URLs in Google search engine based on:
  1. If your your website ranked in search engine results before deleting/submitting the sitemap, it still is after deleting the XML sitemap.
  2. XML sitemaps are supposed to be complementary to normal search engine crawling. Quote from sitemaps.org:
    Sitemaps supplement this data to allow crawlers that support Sitemaps to pick up all URLs

Instead, we believe the indexed URLs count reported per XML sitemap file has a more subtle meaning:
URLs extracted from the submitted XML sitemap which are verified to be included in the search engine index.

This means that it is possible to have:
  • URLs that are indexed but not listed in any XML sitemap file.
  • URLs that remain indexed after they are removed from XML sitemap files.


Proving Indexed URLs Means Verified URLs

  1. Submit two XML sitemaps to Google containing the same URLs.
  2. Wait until most of the URLs are reported as indexed in both XML sitemaps.
  3. Delete one of the XML sitemaps. Now add it again with a new name.
  4. View indexed URLs statistics for both sitemaps:
    • Added XML sitemap has a very low number if any of indexed URLs.
    • Untouched XML sitemap has same number of indexed URLs as before.

  • In underneath example we have tried to delete and add the file sitemap-multi-index.xml back again.
    Notice how Google Webmaster Tools as XML sitemap URLs indexed writes: No data available. Check back soon..

    sitemap indexed urls - 1

  • The file sitemap.xml has not been changed, and continues to show the same amount of URLs indexed.

    sitemap indexed urls - 2


XML Sitemap Optimization

To get the best results from sitemaps with websites where Google has a limited crawl budget is to ensure your XML sitemaps only contain actual important URLs.

A1 Sitemap Generator automatically strips off:
  • Duplicate pages like example/" and "example/index.html" where only the one with the highest internal backlink score is included.
  • Pages that point to another using canonical, HTTP redirect or meta refresh.
  • Pages that have marked themselves noindex or are excluded by robots.txt.
  • URLs that error.

However, even with the above, you may want ensure your website only contains the least possible duplicate and superfluous pages. If you have a website with many of these, but which can not be easily solved, you will want to at least ensure the XML sitemaps you submit are as optimized as possible.

In addition to website markup you can do this in our tool by adding exclude rules in output filters. This is very flexible as you can exclude both exact URLs and URL patterns.

Note: You will need to recrawl your website and recreate the XML sitemap before the exclude rules take effect.
This help page is maintained by

As one of the lead developers, his hands have touched most of the code in the software from Microsys.

If you email any questions, chances are that he will be the one answering them.
TechSEO360About TechSEO360

SEO website crawler tool that can find broken links, analyze internal link juice flow, show duplicate titles, perform custom code/text search and much more.
Share this page with friends   LinkedIn   Twitter   Facebook   Pinterest   Google+   YouTube  
 © Copyright 1997-2018 Microsys

 Usage of this website constitutes an accept of our legal, privacy policy and cookies information.