Actor Scrapy Books Example

This project serves as an example of Python Scrapy project. It scrapes book data from books.toscrape.com.

Getting Started

Install Apify CLI

To use this scraper, you need to install the Apify CLI. Follow the instructions here.

Install Python and Virtualenv

Make sure you have Python installed. If not, download it here. Any version supported by Apify SDK and Scrapy should be fine.

Additionally, install Virtualenv using the following command:

pip install virtualenv

Run the Actor locally

Prepare Python environment

Create a Python virtual environment by running:

python3.12 -m virtualenv .venv

Activate the virtual environment:

source .venv/bin/activate

Install Python dependencies:

pip install -r requirements.txt -r requirements-dev.txt

Run the scraper as Scrapy project

The project is still runnable as a Scrapy project. Execute the following command:

scrapy crawl book_spider -o books.json

Run the scraper as Apify Actor

Run the scraper as an Apify Actor using:

apify run --purge

Deploy on Apify

Log in to Apify

You will need to provide your Apify API Token to complete this action.

apify login

Deploy your Actor

This command will deploy and build the Actor on the Apify Platform. You can find your newly created Actor under Actors -> My Actors.

apify push

Documentation reference

To learn more about Apify and Actors, take a look at the following resources:

• Integrating Scrapy projects
• Apify SDK for Python
• Apify Platform
• Join our developer community on Discord