scrapy/scrapy.org

Stars
60
Rank 487,153 (Top 10 %)
Language
HTML
Created about 11 years ago
Updated 2 months ago

scrapy/scrapy.org

scrapy

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

The scrapy.org website

scrapy

Scrapy, a fast high-level web crawling & scraping framework for Python.

scrapyd

A service daemon to run Scrapy spiders

scrapely

A pure-python HTML screen-scraping library

dirbot

Scrapy project to scrape public web directories (educational) [DEPRECATED]

quotesbot

This is a sample Scrapy project for educational purposes

parsel

Parsel lets you extract data from XML/HTML documents using XPath or CSS selectors

scrapyd-client

Command line client for Scrapyd server

w3lib

Python library of web-related functions

cssselect

CSS Selectors for Python

loginform

Fill HTML login forms automatically

queuelib

Collection of persistent (disk-based) and non-persistent (memory-based) queues for Python

slybot

itemadapter

Common interface for data container classes

protego

A pure-Python robots.txt parser with support for modern conventions.

DIGITAL Command Language

itemloaders

Library to populate items using XPath and CSS with a convenient API

scrapy-bench

A CLI for benchmarking Scrapy.

scurl

Performance-focused replacement for Python urllib

pypydispatcher

A fork of http://pydispatcher.sourceforge.net/ with PyPy support

xtractmime

https://mimesniff.spec.whatwg.org/ implementation for Python

base-chromium

base component forked from Chromium source https://chromium.googlesource.com/chromium/src/base/

scrapy-itemloader

[Archived] Library to populate Scrapy items using XPath and CSS with a convenient API

gsoc2014-integration-tests

GSoC2014 - Scrapy Integration tests project

url-chromium

url component from Chromium source code, forked from https://chromium.googlesource.com/chromium/src/url