Crawl HTTPS Pages and Websites with A1 Website Scraper

Scan and crawl websites that use HTTPS or mix HTTP and HTTPS with website scraper
Help: overview | previous | next

 To see all the options available, you will have to switch off easy mode 

 With options that use a dropdown list, any [+] or [-] button next to adds or removes items in the list itself 

Configure Support for HTTPS

Note: This section is only necessary if:
  1. Your website uses HTTPs.
  2. You use Indy in Scan website | Crawler engine.
  3. You are using Windows.

Configuring OpenSSL for use with A1 Website Scraper will help for all HTTPS / SSL based website login systems including:
  • Standard Login and Authentication Methods
  • Post Form / Session Cookies Website Login

To add support for this, see General options | Tool paths.

get session cookies across https
This help page is maintained by

As one of the lead developers, his hands have touched most of the code in the software from Microsys.

If you email any questions, chances are that he will be the one answering them.
A1 Website ScraperAbout A1 Website Scraper

Extract data from sites into CSV files. By scraping websites, you can grab data on websites and transform it into CSV files ready to be imported anywhere, e.g. SQL databases
Share this page with friends   LinkedIn   Twitter   Facebook   Pinterest   YouTube  
 © Copyright 1997-2020 Microsys

 Usage of this website constitutes an accept of our legal, privacy policy and cookies information.