This is a simple web scraper built in Python with FastAPI. It scrapes data from the oficial website Banco Central de Venezuela and returns it in JSON format.
These instructions will get you a copy of the project up and running on your local machine for development and testing purposes. See deployment for notes on how to deploy the project on a live system.
- Python 3.12 or higher
- FastAPI
- Requests
- BeautifulSoup4
- Docker
- Clone the repository.
- Install the required packages using
pip install -r requirements.txt. - Run the application using
python run.py.
- Send a GET request to
http://localhost:8000/get-data. - The response will be in JSON format.
To run the application in a Docker container, follow these steps:
- Build the Docker image using
docker build -t bcv-scraper .. - Run the Docker container using
docker run -p 8000:8000 bcv-scraper.
Keep in mind that every call to the endpoint executes a scraping to the website. You need to consider storage solutions to avoid problems with the institution and yourself.
If you have the means, I recommend crawlab. You can create a scrapy spider with the selectors in scraper.py, set the cron task and forget about it (Well, mostly).
Contributions are welcome! Please feel free to submit a pull request.
I hope this helps! Let me know if you have any questions.