Spinneret is a web spider and dynamic load generator for testing, benchmarking, and warming caches.
git clone [email protected]:steder/spinneret.git
cd spinneret
mkdir -p ~/.virtualenvs
virtualenv ~/.virtualenvs/spinneret
python setup.py developspinneret spider --base_url http://localhost:8000Because this can take a while I've included a sample sitemap.yaml that can be used to immediately generate traffic.
spinneret traffic --base_url http://localhost:8000 --inflight 100