• Stars
    star
    341
  • Rank 123,998 (Top 3 %)
  • Language
    JavaScript
  • Created over 10 years ago
  • Updated over 6 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Scrape/Crawl article from any site automatically. Make any web page readable, no matter Chinese or English.

read-art NPM version Build Status js-standard-style

NPM

  1. Readability reference to Arc90's.
  2. Scrape article from any page (automatically).
  3. Make any web page readable, no matter Chinese or English.

快速抓取网页文章标题和内容,适合node.js爬虫使用,服务于ElasticSearch。

Guide

How it works

In my case, the speed of spider is about 1500k documents per day, and the maximize crawling speed is 1.2k /minute, avg 1k /minute, the memory cost are about 200 MB on each spider kernel, and the accuracy is about 90%, the rest 10% can be fixed by customizing Score Rules or Selectors. it's better than any other readability modules.

(4) Server infos:

  • 20M bandwidth of fibre-optical
  • 8 Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz cpus
  • 32G memory