Microsys
  

Sitemap Builder Can Exclude URLs When Building XML Sitemaps

The sitemap builder can exclude URLs by various means, e.g. response code. This works with all kinds of sitemaps including HTML and XML sitemaps.
Help: overview | previous | next

 To see all the options available, you will have to switch off easy mode 

 With options that use a dropdown list, any [+] or [-] button next to adds or removes items in the list itself 

General About Filtering URLs before Building Sitemaps

Normally, filtering of URLs are done by the site crawler during website scan, e.g. through output filters, filtering session IDs in URLs and robots.txt file, nofollow and noindex.

Depending on the program configuration, not all URLs shown in the website tree view will be included in generated sitemaps.

You can control if and at which point filters are applied during website scan or sitemap building:
  • Disable Scan website | Crawler options | Apply "webmaster" and "output filters" after website scan stops.
  • Enable Create sitemap | Document options | Remove URLs excluded by "webmaster" and "output" filters.


Exclude URLs by Response Code in Sitemap Builder

In addition to general filtering, you can also exclude URLs when building sitemap files (including HTML sitemaps and XML sitemaps) based on HTTP response codes.

Generally speaking, when using the default configuration, only URLs with a valid response code are included when building sitemaps. There are a few specific exceptions when creating HTML sitemaps, but otherwise all unwanted URLs are left out.

Example: URLs that redirect with e.g. response 301 : Moved Permanently are not included when building XML sitemaps.

Which response codes the sitemap builder will accept can be set in option Create sitemap | Documment options.

xml sitemap response code
This help page is maintained by

As one of the lead developers, his hands have touched most of the code in the software from Microsys.

If you email any questions, chances are that he will be the one answering them.
A1 Sitemap GeneratorAbout A1 Sitemap Generator

Build all kinds of sitemaps including text, visual HTML / CSS, RSS, XML, image, video, news and mobile for all your websites no matter the platform they use.
     
share   LinkedIn   Twitter   Facebook   Pinterest   Google+   YouTube  
 © Copyright 1997-2016 Microsys
 Usage of this website constitutes an accept of our legal, privacy and cookies information.