< Ruby

Share Share Share

Awesome Ruby Web Crawling

• kimuraframework 999

  

  | updated 8 months ago | MIT License

  Kimurai is a modern web scraping framework written in Ruby which works out of box with Headless Chromium/Firefox, PhantomJS, or simple HTTP requests and allows to scrape and interact with JavaScript rendered websites

• link_thumbnailer 511

  

  | updated 9 months ago | MIT License

  Ruby gem that fetches images and metadata from a given URL. Much like popular social website with link preview.

• mechanize 4,352

  

  | updated about 1 month ago | MIT License

  Mechanize is a ruby library that makes automated web interaction easy.

• spidr 763

  

  | updated about 1 year ago | MIT License

  A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.

• upton 1,614

  

  | updated over 5 years ago | MIT License

  A batteries-included framework for easy web-scraping. Just add CSS! (Or do more.)

• wombat 1,300

  

  | updated 3 months ago | MIT License

  Lightweight Ruby web crawler/scraper with an elegant DSL which extracts structured data from pages.