Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

The All in One Framework to build Awesome Scrapers.

Library for Rapid (Web) Crawler and Scraper Development

A simple web scraper to extract Product Data and Pricing from Amazon

Machine Learning Model for Sport Predictions (Football, Basketball, Baseball, Hockey, Soccer & Tennis)

A simple but powerful web crawler library for .NET

⚡ Ayakashi.io - The next generation web scraping framework

Scrapy Training companion code

A tool for scraping emails, social media accounts, and much more information from websites using Google Search Results.

A web crawling framework written in Kotlin

Set up free and scalable Scrapyd cluster for distributed web-crawling with just a few clicks. DEMO 👉

💵 💰 🇧🇷 Informações sobre taxas oficiais diárias de Inflação, Selic, Poupança, Dólar, Dólar PTAX, Euro e Euro PTAX pelo site do Banco Central do Brasil

Parser and database to index the terpene profile of different strains of Cannabis from online databases

Command Line Tool to download torrents

Simple robots.txt template. Keep unwanted robots out (disallow). White lists (allow) legitimate user-agents. Useful for all websites.

This is a Twitter Scraper which uses Selenium for scraping tweets. It is capable of scraping tweets from home, user profile, hashtag, query or search, and advanced searches.

Amazon products scraper with using of rotating proxies and headless Chrome from ScrapingAnt

JAW: A Graph-based Security Analysis Framework for Client-side JavaScript

Web Scraping Craigslist's Engineering Jobs in NY with Scrapy

Scraping and Web Crawling Framework For Zhihu Live