Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement pure NodeJS jobs queue backed by Bull #73

Open
1 of 11 tasks
joseconstela opened this issue Apr 28, 2020 · 0 comments
Open
1 of 11 tasks

Implement pure NodeJS jobs queue backed by Bull #73

joseconstela opened this issue Apr 28, 2020 · 0 comments

Comments

@joseconstela
Copy link
Member

joseconstela commented Apr 28, 2020

The current implementation of the jobs queue (the engine in charge of running and orchestrating workflows executions) is built in top of msavin:sjobs-ui-blaze, which at the same time is built in top of MeteorJS.

The new Jobs queue service should be created from the scratch, not using either msavin:sjobs-ui-blaze or MeteorJS. This will allow to:

  • Run the queue either as a separate process, or within the same process as Tideflow's
  • Run the queue in a separate machine or server
  • Let the jobs and Tideflow itself to use separate database connections
  • Do not depend in possible obsolete or unmaintained MeteorJs packages.
  • Possiblity to fork queues processes for blazzing fast performance.
  • To have a separate dashboard to see metrics on the queue.

To be discussed:

  • How to keep modularity? as the services will run on the queue, but they also have visual elements that are rendered on the main application.

Subtasks

  • Skeleton with MongodDB connection
  • Rewrite job workflow-start
  • Rewrite job workflow-step
  • Rewrite Eliminate job workflow-execution-notify-email
  • Rewrite job workflow-execution-finished
  • Rewrite job _executionLogsRun
  • Rewrite job _executionLogsSendEmail
  • Rewrite job s-cron-runOne
  • Rewrite job s-rss-runOne
  • Rewrite job s-rss-schedule
  • Move the service's server/service to the new queue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant