Microsys
  

MiggiBot Crawler Robot in A1 Software

MiggiBot is the website crawler robot powering a range of tools including our A1 Crawler software.
Help: overview | previous | next

Features in MiggiBot Robot Crawler Engine

  • Can extract vast amounts of meta data about all pages in a website. File sizes, download times, server response codes etc.
  • Has various tools for analyzing keyword density. Can use stop words and similar filters.
  • Can generate sitemaps of websites with the crawler supporting advanced filters.
  • Can check, validate and store links, redirects and even source from data (e.g. images and frames).
  • Can calculate the internal importance of all pages within a site.
  • Can generate CSV and XML files for all crawled and derived data.
  • Has website download and links conversion functionality.


Future MiggiBot and Crawler Software Development

  • Continue to develop and polish the engine.
  • Some very exotic and secret ideas.


Products Where MiggiBot Crawler Engine is Used

  • Software
    • A1 Sitemap Generator [link] - HTML/XML sitemapper
    • A1 Keyword Research [link] - Keyword SEO analyzer
    • A1 Website Analyzer [link] - Site and link checker
    • A1 Website Download [link] - Web site downloader
    • A1 Website Scraper [link] - Site data scraping
    • A1 Website Search Engine [link] - Site search solution


For Developers and Programmers

  • Generated output XML data
    • Having easy-to-parse data can be useful for companies building custom research tools.
    • See A1 Website Analyzer for more information.
  • Generated output XML data
    • We are currently considering if and how to expose our crawler engine for developers.
    • Feel free to contact us if you are interested.


Fun facts About Miggi / MiggiBot

The name is a contraction of the following words Micro-Sys, Microsys, microbot, microbotic, robot, bot, big etc.
This help page is maintained by

As one of the lead developers, his hands have touched most of the code in the software from Microsys.

If you email any questions, chances are that he will be the one answering them.
MicrosysAbout Microsys

Microsys develops desktop software, mobile apps and websites.This page is part of the shared help documentation across many of our tools.
     
share   LinkedIn   Twitter   Facebook   Pinterest   Google+   YouTube  
 © Copyright 1997-2016 Microsys
 Usage of this website constitutes an accept of our legal, privacy and cookies information.