Skip to content

🏫 Scrape news from my school site and send them trough Discord!

License

Notifications You must be signed in to change notification settings

ElektronPlus/zseis-news

Repository files navigation

News scraper and publisher for zseis.zgora.pl


📐 Technical info

  • You can't get id of entry in easy way (to my knowledge other than brute-forcing), so script generates a checksum (md5(title + dateModified)) to check what news were already sent to Discord
  • Script has hard-coded that there are 4 news per page and will throw error if will find more, this is due to not well structured HTML of website. This also leads to news extracter seem to be a bit unintuitive.
  • It uses Discord Webhooks and requires function in other bot to crosspost on announcement channel.

🔨 Developing

git clone https://github.com/konhi/zseis-news.git
cd zseis-news
echo WEBHOOK_URL="<https://discord.com/api/webhooks/xxxxxxxxx/xxxxxxxxxxxxxxxxxxxxxxxxx" > .env
npm install
  • npm run start: builds and runs the script
  • npm run test: code testing using jest
  • npm run lint: automatically fix problems and show style errors
  • npm run dev: TypeScript compiler watch for changes
  • node dist/main.js: run main program (without compiling)