Awesome Ruby Scraping and Data Extraction

  • anemone anemone 1,616
    star
    updated over 4 years ago MIT License

    Anemone web-spider framework

  • updated 10 months ago MIT License

    a simple, fast web-crawler written in Ruby using Watir or Typhoeus

  • updated almost 4 years ago Other

    Pure ruby implementation of the Boilerpipe content extraction algorithm tuned for online articles

  • cobweb cobweb 226
    star
    updated almost 2 years ago MIT License

    Web crawler with very flexible crawling options. Can either use standalone or can be used with resque to perform clustered crawls.

  • updated over 3 years ago MIT License

    A ruby web/screen scraping tool / gem.

  • updated over 4 years ago MIT License
  • updated over 1 year ago MIT License

    Ruby gem that fetches images and metadata from a given URL. Much like popular social website with link preview.

  • updated about 1 year ago MIT License

    Ruby gem for web scraping purposes. It scrapes a given URL, and returns you its title, meta description, meta keywords, links, images...

  • sinew sinew 253
    star
    updated 8 months ago MIT License

    A Ruby DSL for structured web crawling, with a robust caching system.

  • wombat wombat 1,307
    star
    updated 10 months ago MIT License

    Lightweight Ruby web crawler/scraper with an elegant DSL which extracts structured data from pages.