-
Notifications
You must be signed in to change notification settings - Fork 5
Concepts
In Kronos, everything belongs to a namespace. It allows the user to segregate workflows across namespaces for the multi-tenant environment.
In Kronos, a workflow is a sequence of tasks necessary to complete a job. Tasks can be dependant on other tasks.
For example, a simple Workflow could consist of three tasks: TaskA, TaskB, and TaskC. It could say that TaskA has to run successfully before TaskB can run, but TaskC can run anytime. The workflow is to be scheduled to run everyday midnite.
In this way, a workflow defines how you want to carry out the execution of these (in)dependent tasks. When we say tasks, they could be anything from a simple Java job to Spark job. Maybe TaskA prepares data for TaskB to analyze where TaskC simply sends an email. The important thing is that the workflow isn’t concerned with what its constituent tasks do; it is responsible to make sure that whatever they do happens in the right order, with the right handling of any unexpected issues.
The way a workflow defines how to carry out the execution of tasks, a workflow trigger defines when to execute a workflow. A trigger contains an optional start and end time with a mandatory schedule. In absence of a start and end time, the trigger is active until detached from the workflow. This allows users to defines workflow separately and then attach triggers to it to schedule it for execution. The separation also allows a user to attach multiple triggers to the same workflow.
A Job represents a specific run of a workflow at a point in time scheduled through the trigger. The job has a unique identifier and an indicative state, which could be "CREATED", "RUNNING", "FAILED", "SUCCESSFUL".
Tasks in a workflow are specified while defining (creating) a workflow. A task is composed of a set of attributes required by the Kronos during execution like the type of task (used to determine the handler used for its execution), policy to apply in case of timeout, max execution time etc.
A task instance represents a specific instance of a workflow task in a job at a point in time. Task instances also have a unique identifier and an indicative state, which could be "CREATED", "WAITING", "UP_FOR_RETRY", "SCHEDULED", "RUNNING", "SUCCESSFUL", "SKIPPED", "FAILED", "TIMED_OUT", "ABORTED".
While workflow defines the execution order of tasks, a task handler defines how the task is executed.
A Task handler is configured to execute a specific type of task.
Kronos provides the following task handlers as part of the distribution.
- Shell Command Handler- executes a bash command.
- Spark Handler - submits a Spark job to a Spark cluster for execution and tracks the status.
In addition to these pre-built handlers, Kronos allows a user to plug in a custom handler to execute a specific type of task.