Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make XPK Handle multiple slice sizes #17

Open
rwitten opened this issue Nov 17, 2023 · 3 comments
Open

Make XPK Handle multiple slice sizes #17

rwitten opened this issue Nov 17, 2023 · 3 comments

Comments

@rwitten
Copy link
Collaborator

rwitten commented Nov 17, 2023

N queues, 1 per slice size, 1 cluster.

(This is complicated!)

@Obliviour
Copy link
Collaborator

From my understanding I believe this would be multiple resource flavors with their respective chip sizes still in 1 ClusterQueue. https://kubernetes.io/blog/2022/10/04/introducing-kueue/#example-use-case

@Obliviour
Copy link
Collaborator

maybe looks something like this:

xpk cluster create --cluster=my-cluster --tpu-types v5p-128,5 v5p-256,5   # type, num_slices
  • make sure that cluster create when rerunning the command is aware of the heterogenousness of the cluster

@Obliviour
Copy link
Collaborator

One aspect to this is that the overall goal here is to allow the gke cluster to figure out the needed slice types based on a set of user provided options: chip-budget and what incoming requests are.

Probably can also set some minimum / starting points.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants