A node.js powered scrapper 🔥 that iterates trough all the internal links of the specified url.

It works on CSR pages (React, Angular) with dynamic urls.

Once it is done it generates a sitemap.xml file with all the urls found, ready to be uploaded to Google Search Console.

Usage:

$ sitemap https://vvlog.dev

Params:

Parameter	type	default	description
--wait	integer	1500	Specify the time (milliseconds) to wait (So the fetches are completed) before starting to parse the page.
--limit	integer	999999	Specify the limit of urls to parse before stopping the scrapper.

Todo:

• Make it a NPM package.
• Make wait time dynamic in response of fetches inside url.
• New params that lets you specify how deep you want to go inside the url.
• Integrate it as part of build process of a create-react-app.
• Clean old code.