|
|
Firewall Causes Problems for A1 Website Download
While our website download program uses normal HTTP internet connections for crawling websites, some firewall solutions will still block our software unless you take direct action.
How Firewalls Interact With Internet Enabled Software
Most firewall software programs default to silently block all internet enabled applications unless explicitly specified otherwise in configuration:
One hint of firewall or internet security software being the cause is if you have response codes like these listed in the website scan results:
Note: Another possible reason for the above problems can be modules installed on the webserver or website that blocks unknown crawlers.
- If you get flaky results, odd errors and similar, net traffic filtering software can be the reason.
- If you get no URLs found in website crawl and related tools, firewall software is often the reason.
- If you get warnings while crawling a website with links to dubious resources, internet security software can be the reason.
One hint of firewall or internet security software being the cause is if you have response codes like these listed in the website scan results:
- 500 : Internal Server Error
- 503 : Service Temporarily Unavailable
- -4 : CommError
Note: Another possible reason for the above problems can be modules installed on the webserver or website that blocks unknown crawlers.
How Internet Security Solutions Can Block Installation
Antivirus and other internet security solutions can sometimes cause problems when you are trying to download, install and run our software.
This usually only happens if we have just released a new version of A1 Website Download and the download is thus considered rarely encountered.
If that happens the installation and the following launch may appear erratic, partially blocked and give error messages. In such cases check your antivirus log.
This usually only happens if we have just released a new version of A1 Website Download and the download is thus considered rarely encountered.
If that happens the installation and the following launch may appear erratic, partially blocked and give error messages. In such cases check your antivirus log.
Solutions to Get A1 Website Download Installed
AVG CyberCapture : Rare File:
There are two possible solutions:
There are two possible solutions:
- Submit the file to AVG and wait.
- Turn off file execution scan while installing.
Get The Crawler Working in Our Website Download Software
NOD32 client version 3:
Norton 360:
ESET Smart Security:
Solution:
Bitdefender:
Solution:
Kaspersky anti-virus:
Symptoms:
Other software and hardware firewall solutions:
Symptoms:
- View advanced mode
- Select Setup
- Click - Antivirus and antispyware
- In Web access protection click Configure
- Expand HTTP and click Web browsers
- NOD32 will automatically consider" A1 Website Download as web browser" (checked) - you must uncheck it for A1 Website Download to work.
Norton 360:
- Whitelist / add the program A1 Website Download in Program Rules
ESET Smart Security:
Solution:
- Set it to learning mode.
Bitdefender:
Solution:
- Take note if the websites you crawl contain links to or uses resources from unsafe websites.
Kaspersky anti-virus:
Symptoms:
- URL timeouts with Indy HTTP engine.
- URL 404 response codes with WinInet HTTP engine
- Pause it
Other software and hardware firewall solutions:
Symptoms:
- Various errors and/or no crawling
- In Scan website | Crawler engine to HTTP using WinInet engine and settings (Internet Explorer)
- In General Options and tools | Internet Crawler to Mozilla/4.0 (compatible; MSIE 8.0; Win32)
- In Scan website | Crawler engine lower amount of simultaneous conections, possibly all down to one.
- In Scan website | Crawler engine increase the amount of time between active connections.
