Abstract: MiggiBot is the website crawler robot powering a range of tools including our sitemap generator program. Navigate:Sitemap Generator | Buy | Download | Help Index
Features in MiggiBot Robot Crawler Engine
Can extract vast amounts of meta data about all pages in a website. File sizes, download times, server response codes etc.
Has various tools for analyzing keyword density. Can use stop words and similar filters.
Can generate sitemaps of websites with the crawler supporting advanced filters.
Can check, validate and store links, redirects and even source from data (e.g. images and frames).
Can calculate the internal importance of all pages within a site.
Can generate CSV and XML files for all crawled and derived data.
Has website download and links conversion functionality.
Online Store
