-
Notifications
You must be signed in to change notification settings - Fork 7
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Model aware scheduling #96
Comments
/kind feature |
I see that the built-in plugins of scheduler currently include: volumebinding volumerestrictions volumeszone plugins. Do we have scoring plugins similar to imagelocality plugins (e.g. volumelocality) to cover this scenario? |
Yes, basically the idea is we have models located at different nodes, and we should be aware of which node is the best candidate. However, right now, I'm developing a P2P based model distribution project, you can take it as another lightweight dragonfly but most works for model weights. See https://github.com/InftyAI/Manta, once this finished, the model aware is not that urgent, because models will be transmitted across nodes, but still available. |
What would you like to be added:
Right now, model management is a tricky problem in the cluster, it's big, so we need to cache them in the node just like images, however, kubelet will take over the image lifecycle management but files, so that's a problem, and will not be tacked in the near future, so maybe we need to manage the models manually and make it aware by the scheduler to make pod placement decisions.
Why is this needed:
Efficient pod scheduling with models
Completion requirements:
This enhancement requires the following artifacts:
The artifacts should be linked in subsequent comments.
The text was updated successfully, but these errors were encountered: