Meerkat Cloud Browser

Control Browser Server via Web API

Backend

Web Server: https://github.com/hansonkd/puff#puff--asyncio
Proxy Server: Develop new proxy that can stream video data written in Rust

Web app: FastAPI

Frontend:

Data Extraction GUI: https://github.com/xyflow/xyflow
Dashboard: Clean Admin GUI

Database:

Main DB: Postgres
Cache: Consider using SlateDB - https://docs.rs/slatedb/latest/slatedb/

Batch:

Use Python RQ to handle batch runner and execute Firefox ESR to crawl data
Browser: Firefox ESR to support login and cookies

Drivers

geckordp: https://jpramosi.github.io/geckordp/index.html