< Ruby

Share Share Share

Awesome Ruby Scraping and Data Extraction

• anemone 1,616

  

  | updated about 4 years ago | MIT License

  Anemone web-spider framework

• arachnid2 16

  

  | updated 3 months ago | MIT License

  a simple, fast web-crawler written in Ruby using Watir or Typhoeus

• boilerpipe-ruby 38

  

  | updated about 3 years ago | Other

  Pure ruby implementation of the Boilerpipe content extraction algorithm tuned for online articles

• cobweb 228

  

  | updated over 1 year ago | MIT License

  Web crawler with very flexible crawling options. Can either use standalone or can be used with resque to perform clustered crawls.

• docparser 37

  

  | updated almost 3 years ago | MIT License

  A ruby web/screen scraping tool / gem.

• horsefield 2

  

  | updated almost 4 years ago | MIT License
• link_thumbnailer 511

  

  | updated 9 months ago | MIT License

  Ruby gem that fetches images and metadata from a given URL. Much like popular social website with link preview.

• metainspector 1,005

  

  | updated 8 months ago | MIT License

  Ruby gem for web scraping purposes. It scrapes a given URL, and returns you its title, meta description, meta keywords, links, images...

• sinew 254

  

  | updated about 2 months ago | MIT License

  A Ruby DSL for structured web crawling, with a robust caching system.

• wombat 1,300

  

  | updated 3 months ago | MIT License

  Lightweight Ruby web crawler/scraper with an elegant DSL which extracts structured data from pages.