RSS Renderer Slowing down system

[mariobehling]

With regards to speed, the culprits are happy to present themselves.
From quickly tailing the prod log app@voicerepublic:~/app/current$ tail -f log/production.log, I see many RSS requests happen.
Those are not efficiently implemented (and very old code).

[munen]

I'll elaborate a little on the RSS feed scaling issue: VR is an open platform and features multiple podcasts (for example per user or venue). The feature itself was implemented in May 2014 when scaling wasn't an issue at VR. Back then, it was required to have a running version fast and not a fast version.

The current implementation aggregates all necessary information for every podcast/rss request. This is a CPU and DB intensive task which - depending on the number of talks - takes seconds up to minutes.

Since VR has gotten more popular, many peoples feeds have been subscribed to with podcatchers. Those in turn make recurring requests to see if there's new content.

The mitigation has already been started in PR #845. In that, we're introducing a message broker and work queue (RabbitMQ). Feeds will pre-emptively be rendered whenever a user makes changes to his content and then be cached. So they will only be rendered once and the actual request from the podcatcher will be trivial and fast.