[flytekit/async] Merge loop usage & do proper signal handling #6068
Labels
backlogged
For internal use. Reserved for contributor team workflow.
Epic: Async Flytekit
Enable asynchronous behavior in flytekit
flytekit
FlyteKit Python related issue
Describe the issue
Multiple loops and threads are used in the course of flytekit's async handling. Eager tasks themselves rely on a worker to launch executions of sub-flyte-entities against the control plane. Currently this latter loop is run outside of the main loop manager that flytekit uses. We should consider merging, and then implement proper loop lifecycle management for all loops, running on any thread. (i.e. calling
.cancel()
on all running tasks of each loop and handling canceled errors correctly in the relevant coroutines).What if we do not do this?
Signal handling and async loop lifecycle management remains messy.
Are you sure this issue hasn't been raised already?
Have you read the Code of Conduct?
The text was updated successfully, but these errors were encountered: