pip install Scrapy==1.8.1
A high-level Web Crawling and Web Scraping framework
SourceAmong top 1000 packages on PyPI.
Over 4.4M downloads in the last 90 days.
Scrapy
Based on how often these packages appear together in public
requirements.txt
files on GitHub.
|
Collection of persistent (disk-based) queues |
|
|
Library of web-related functions |
|
|
Multi-producer-multi-consumer signal dispatching mechanism |
|
|
Parsel is a library to extract data from HTML and XML using XPath and CSS selectors |
|
|
Configurable Python Web Scraper |
|
|
An asynchronous networking framework written in Python |
|
|
A service for running Scrapy spiders, with an HTTP API |
|
|
Python attributes without boilerplate. |
|
|
Symbolic constants in Python |
|
|
None |
|
|
Self-service finite-state machines for the programmer on the go. |
|
|
A client for scrapyd |
|
|
Scrapy extension to write scraped items using Django models |
|
|
A featureful, immutable, and correct URL for Python. |
|
|
Object-oriented filesystem path representation. |
|
|
Pipeline to MongoDB for Scrapy. Supports MongoDB replica sets |
|
|
cssselect parses CSS3 Selectors and translates them to XPath 1.0 |
|
|
Pure-Python robots.txt parser with support for modern conventions |
|
|
Compatibility shim for simplejson |
ScrapyProportion of downloaded versions in the last 3 months (only versions over 1%).
|
1.5.0 |
58.49% |
|
2.5.0 |
11.78% |
|
2.3.0 |
8.24% |
|
2.5.1 |
6.26% |
|
1.6.0 |
5.19% |
|
2.4.1 |
1.69% |
|
1.8.1 |
1.53% |
|
1.7.3 |
1.08% |