scrapylib | Python Package Wiki

Based on how often these packages appear together in public requirements.txt files on GitHub.

scrapy-deltafetch	Scrapy middleware to ignore previously crawled pages
monkeylearn	Official Python client for the MonkeyLearn API
scrapy-magicfields	Scrapy middleware to add extra "magic" fields to items
scrapy-querycleaner	Scrapy spider middleware to clean up query parameters in request URLs
scrapy-dotpersistence	Scrapy extension to sync `.scrapy` folder to an S3 bucket
scrapy-splitvariants	Scrapy spider middleware to split an item into multiple items on a multi-valued key
scrapinghub-entrypoint-scrapy	Scrapy entrypoint for Scrapinghub job runner
scrapy-pagestorage	Scrapy extension to store info in storage service
scrapy-crawlera	Crawlera middleware for Scrapy
scrapinghub	Client interface for Scrapinghub API
py-romanify	Python Roman/Arabic numbers convertor
scrapyd	A service for running Scrapy spiders, with an HTTP API
w3lib	Library of web-related functions
queuelib	Collection of persistent (disk-based) queues
scraper	Configurable Python Web Scraper
parsel	Parsel is a library to extract data from HTML and XML using XPath and CSS selectors
Scrapy	A high-level Web Crawling and Web Scraping framework
PyDispatcher	Multi-producer-multi-consumer signal dispatching mechanism
Protego	Pure-Python robots.txt parser with support for modern conventions

Proportion of downloaded versions in the last 3 months (only versions over 1%).

pip install scrapylib==1.7.1