Feature/configure kernel launch terminate on events #1383
Conversation
…omparing the container events to the configuration events.
…ngKernelManager`.
for more information, see https://pre-commit.ci
|
Discussed in #1357 |
…ttps://github.com/OrenZ1/enterprise_gateway into feature/configure_kernel_launch_terminate_on_events
kevin-bates
added
enhancement
kubernetes
kernels
docker
gateway-provisioners
There was a problem hiding this comment.
Thanks @OrenZ1 - this looks good. I'm wondering if this particular option shouldn't be documented - at which point we could add references to expected event types and reasons.
Also, once merged, it would be awesome to get this into the similar location within gateway_provisioners.
| kernel_pod_events = self.get_container_events() | ||
| for event in kernel_pod_events: | ||
| if ( | ||
| event.type in self.kernel_launch_terminate_on_events |
There was a problem hiding this comment.
Since the event type and reason need to be matches, is there a doc reference we could add that enumerates the various values that could be expected? That said, I suppose folks will need to be reactive (vs. proactive) in this sense and will have the event information when troubleshooting, but it still might be helpful to have a link or two, or add the enumeration directly.
There was a problem hiding this comment.
Added a docstring to handle_pending_kernel method in container.py, as well as a more detailed configuration help. I found it hard to find a detailed list of Kubernetes events by type and reason, so I added an enumeration which may differ in different platforms (or even versions of K8S itself). I agree that folks will need to be reactive, but I did also add a link to a guide I found helpful with finding the correct event type and reason.
…le events reasons and types.
…n `kernel_launch_terminate_on_events` help.
for more information, see https://pre-commit.ci
|
Hi! Any update on this? |
There was a problem hiding this comment.
I think this option should have a presence in the Operator's Guide somewhere but can't really find a good location. The only way folks will know about this right now is looking at the code. Thinking we should at least add it into Command-line Options in the RemoteMappingKernelManager section.
…ingKernelManager` cli options.
…ttps://github.com/OrenZ1/enterprise_gateway into feature/configure_kernel_launch_terminate_on_events
for more information, see https://pre-commit.ci
|
Added presence in the |
This pull request adds the option to configure kernel container events which should cause a termination of the kernel launch process.
The user can now configure "unresolvable" events, to prevent a long wait for the timeout when such events occur.
It also allows the users to set a different timeout for some events, in example when a user knows an event may resolve after a small wait, but if it didn't, they want to terminate the kernel launch process.