pip install w3lib==1.22.0
Library of web-related functions
SourceAmong top 1000 packages on PyPI.
Over 4.5M downloads in the last 90 days.
w3lib
Based on how often these packages appear together in public
requirements.txt
files on GitHub.
Collection of persistent (disk-based) queues |
|
A high-level Web Crawling and Web Scraping framework |
|
Multi-producer-multi-consumer signal dispatching mechanism |
|
Parsel is a library to extract data from HTML and XML using XPath and CSS selectors |
|
Configurable Python Web Scraper |
|
An asynchronous networking framework written in Python |
|
A service for running Scrapy spiders, with an HTTP API |
|
Python attributes without boilerplate. |
|
Symbolic constants in Python |
|
None |
|
Self-service finite-state machines for the programmer on the go. |
|
A client for scrapyd |
|
A featureful, immutable, and correct URL for Python. |
|
Scrapy extension to write scraped items using Django models |
|
cssselect parses CSS3 Selectors and translates them to XPath 1.0 |
|
Object-oriented filesystem path representation. |
|
Pipeline to MongoDB for Scrapy. Supports MongoDB replica sets |
|
Pure-Python robots.txt parser with support for modern conventions |
|
Extract embedded metadata from HTML markup |
w3lib
Proportion of downloaded versions in the last 3 months (only versions over 1%).
1.22.0 |
80.90% |
1.21.0 |
16.76% |
1.20.0 |
1.26% |