Microsys

Website Crawl Status Progress in Keyword Research

Understand Website Scan Information

website crawl progress status

The website crawler keeps total counts for following states:
  • Listed found:
    Unique URLs located.
  • Listed deduced:
    Appears after website scan has finished: Assume crawler during scan found links to "example/somepage.html", but none to "example/". The latter is then "deduced" to exist.
  • Analyzed content:
    Unique URLs with content analyzed.
  • Analyzed references:
    Unique URLs with content analyzed and have all their links in content resolved (e.g. links to URLs that redirects).


Log and Analyze Website Crawling Issues

If you experience strange problems spidering your website, you can try enable Scan website - Data collection - Logging of progress. After website scan, you can find a log file in the program data directory logs/misc.
website scan progress log
The log file can be useful in solving problems related to crawler filters, robots.txt, no-follow links etc. You can find out through which page the crawler first found a specific website section.
2007-07-28 10:56:14
CodeArea: InitLink:Begin
ReferencedFromLink: http://www.example.com/website/
LinkToCheck: http://www.example.com/website/scan.html

Website software tools


Business software utilities


Popular freeware programs

Online tools


Webmaster articles


Website promotion resources

 © Copyright 1997-2010 Microsys | about | contact | legal | privacy