scrapy/slybot scrapy/slybot

  • Stars
    star
    224
  • Rank 171,880 (Top 4 %)
  • Language
  • Created over 12 years ago
  • Updated about 9 years ago
Twitter logo Share LinkedIn logo Share Facebook logo Share
scrapy/slybot
github

scrapy

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Slybot is now part of Portia

Slybot is now part of the Portia scraper.

The commit history of this repository has been preserved only for reference purposes.

More Repositories

1

scrapy

Scrapy, a fast high-level web crawling & scraping framework for Python.
Python
51,036
star
2

scrapyd

A service daemon to run Scrapy spiders
Python
2,857
star
3

scrapely

A pure-python HTML screen-scraping library
HTML
1,855
star
4

dirbot

Scrapy project to scrape public web directories (educational) [DEPRECATED]
Python
1,630
star
5

quotesbot

This is a sample Scrapy project for educational purposes
Python
1,275
star
6

parsel

Parsel lets you extract data from XML/HTML documents using XPath or CSS selectors
Python
1,088
star
7

scrapyd-client

Command line client for Scrapyd server
Python
755
star
8

w3lib

Python library of web-related functions
Python
382
star
9

cssselect

CSS Selectors for Python
Python
284
star
10

loginform

Fill HTML login forms automatically
Python
267
star
11

queuelib

Collection of persistent (disk-based) and non-persistent (memory-based) queues for Python
Python
261
star
12

scrapy.org

The scrapy.org website
HTML
60
star
13

itemadapter

Common interface for data container classes
Python
60
star
14

protego

A pure-Python robots.txt parser with support for modern conventions.
DIGITAL Command Language
51
star
15

itemloaders

Library to populate items using XPath and CSS with a convenient API
Python
43
star
16

scrapy-bench

A CLI for benchmarking Scrapy.
Python
30
star
17

scurl

Performance-focused replacement for Python urllib
Python
21
star
18

pypydispatcher

A fork of http://pydispatcher.sourceforge.net/ with PyPy support
Python
15
star
19

xtractmime

https://mimesniff.spec.whatwg.org/ implementation for Python
Python
13
star
20

base-chromium

base component forked from Chromium source https://chromium.googlesource.com/chromium/src/base/
C++
7
star
21

scrapy-itemloader

[Archived] Library to populate Scrapy items using XPath and CSS with a convenient API
Python
6
star
22

gsoc2014-integration-tests

GSoC2014 - Scrapy Integration tests project
Shell
3
star
23

url-chromium

url component from Chromium source code, forked from https://chromium.googlesource.com/chromium/src/url
C++
2
star