pip install scrapy-dotpersistence==0.3.0

Scrapy extension to sync `.scrapy` folder to an S3 bucket

Source
Among top 50% packages on PyPI.
Over 2.7K downloads in the last 90 days.

Commonly used with scrapy-dotpersistence

Based on how often these packages appear together in public requirements.txt files on GitHub.

scrapy-splitvariants

Scrapy spider middleware to split an item into multiple items on a multi-valued key

scrapy-querycleaner

Scrapy spider middleware to clean up query parameters in request URLs

scrapy-pagestorage

Scrapy extension to store info in storage service

monkeylearn

Official Python client for the MonkeyLearn API

scrapinghub-entrypoint-scrapy

Scrapy entrypoint for Scrapinghub job runner

scrapy-dotpersistence

Scrapy extension to sync `.scrapy` folder to an S3 bucket

scrapy-crawlera

Crawlera middleware for Scrapy

scrapy-deltafetch

Scrapy middleware to ignore previously crawled pages

scrapylib

Scrapy helper functions and processors

scrapinghub

Client interface for Scrapinghub API

aws-xray-sdk

The AWS X-Ray SDK for Python (the SDK) enables Python developers to record and emit information from within their applications to the AWS X-Ray service.

s3transfer

An Amazon S3 Transfer Manager

botocore

Low-level, data-driven core of boto 3.

jmespath

JSON Matching Expressions

upstart

Upstart-based service management.

moby

Tool to automate running commands in docker.

boto3

The AWS SDK for Python

cfn-environment-base

Base environment for Troposphere based CFN project environments

watchtower

Python CloudWatch Logging

Version usage of scrapy-dotpersistence

Proportion of downloaded versions in the last 3 months (only versions over 1%).

0.3.0

86.58%

0.2.3

6.31%

0.2.0

4.47%

0.2.2

2.63%