< Ruby

Share Share Share

Awesome Ruby Web Crawling

• kimuraframework 1,014

  

  | updated 6 months ago | MIT License

  Kimurai is a modern web scraping framework written in Ruby which works out of box with Headless Chromium/Firefox, PhantomJS, or simple HTTP requests and allows to scrape and interact with JavaScript rendered websites

• link_thumbnailer 511

  

  | updated over 1 year ago | MIT License

  Ruby gem that fetches images and metadata from a given URL. Much like popular social website with link preview.

• mechanize 4,391

  

  | updated 3 months ago | MIT License

  Mechanize is a ruby library that makes automated web interaction easy.

• spidr 763

  

  | updated over 1 year ago | MIT License

  A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.

• upton 1,614

  

  | updated almost 6 years ago | MIT License

  A batteries-included framework for easy web-scraping. Just add CSS! (Or do more.)

• wombat 1,307

  

  | updated 10 months ago | MIT License

  Lightweight Ruby web crawler/scraper with an elegant DSL which extracts structured data from pages.