Microsys
  

Language Detection and Analyzing Pages for Similar Content

You can improve analysis of similar and duplicate content by ensuring language idenification is correct.
Help: overview | previous | next

 To see all the options available, you will have to switch off easy mode 

 With options that use a dropdown list, any [+] or [-] button next to adds or removes items in the list itself 

Page Language Detection in Our Website Analyzer

Our software determines the primary page language by checking the following things:
  1. Checks if the webserver responds with content-language HTTP response header:
    • PHP pages: Insert this code <?php header("Content-Language: en"); ?>.

  2. The page is checked for content-language META tag:
    <meta http-equiv="content-language" content="en">

  3. The page is checked for lang inside the HTML tag:
    <html lang="en">

  4. The page is searched for Open Graph Protocol attribute property og:locale inside META tags.

  5. The page is checked for xml:lang inside the HTML tag:
    <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">

  6. The page URL is checked for common language/culture and country codes.

  7. Planned: Compare content against word lists for each language. Select best match.


Duplicate Content and Similarity Analysis

By ensuring the language is detected correctly, the crawler can exclude common words and only analyze content words.

Options you can set that will help the crawler:
  • Set Select stop words to match the main language of your website or select auto if it uses multiple languages.
This help page is maintained by

As one of the lead developers, his hands have touched most of the code in the software from Microsys.

If you email any questions, chances are that he will be the one answering them.
A1 Website AnalyzerAbout A1 Website Analyzer

SEO website crawler tool that can find broken links, analyze internal link juice flow, show duplicate titles, perform custom code/text search and much more.
     
Share this page with friends   LinkedIn   Twitter   Facebook   Pinterest   Google+   YouTube  
 © Copyright 1997-2016 Microsys
 Usage of this website constitutes an accept of our legal, privacy and cookies information.