Meerkat Cloud Browser

Control Browser Server via Web API

Backend

• Web Server: https://github.com/hansonkd/puff#puff--asyncio
• Proxy Server: Develop new proxy that can stream video data written in Rust

Web app: FastAPI

Frontend:

• Data Extraction GUI: https://github.com/xyflow/xyflow
• Dashboard: Clean Admin GUI

Database:

• Main DB: Postgres
• Cache: Consider using SlateDB - https://docs.rs/slatedb/latest/slatedb/

Batch:

• Use Python RQ to handle batch runner and execute Firefox ESR to crawl data
• Browser: Firefox ESR to support login and cookies

Drivers

• geckordp: https://jpramosi.github.io/geckordp/index.html