Skip to content

Commit

Permalink
Merge pull request #9 from run-ai/support_podpool
Browse files Browse the repository at this point in the history
support podPool
  • Loading branch information
lliranbabi authored Jul 10, 2024
2 parents 5fa6ff9 + 4c96c39 commit a5fd521
Show file tree
Hide file tree
Showing 21 changed files with 9,292 additions and 139 deletions.
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
# To re-generate a bundle for another specific version without changing the standard setup, you can:
# - use the VERSION as arg of the bundle target (e.g make bundle VERSION=0.0.2)
# - use environment variables to overwrite this value (e.g export VERSION=0.0.2)
VERSION ?= 0.0.2
VERSION ?= 0.0.3

# CHANNELS define the bundle channels used in the bundle.
# Add a new line here if you would like to change its default config. (E.g CHANNELS = "candidate,fast,stable")
Expand Down
9 changes: 9 additions & 0 deletions PROJECT
Original file line number Diff line number Diff line change
Expand Up @@ -29,4 +29,13 @@ resources:
kind: DeploymentPool
path: github.com/run-ai/kwok-operator/api/v1beta1
version: v1beta1
- api:
crdVersion: v1
namespaced: true
controller: true
domain: run-ai.com
group: kwok.sigs
kind: PodPool
path: github.com/run-ai/kwok-operator/api/v1beta1
version: v1beta1
version: "3"
194 changes: 115 additions & 79 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,54 +54,58 @@ To install Kwok CRDs and the Kwok Operator, follow these steps:
```
or
```shell
kubectl apply --server-side -f https://github.com/run-ai/kwok-operator/releases/download/0.0.2/kwok-operator.yaml
kubectl apply --server-side -f https://github.com/run-ai/kwok-operator/releases/download/0.0.3/kwok-operator.yaml
```
## Usage

To use the Kwok Operator to provision nodes, follow these steps:

1. Define a NodePool custom resource (CR) with your desired configuration. Example:

```yaml
apiVersion: kwok.sigs.k8s.io/v1beta1
kind: NodePool
```yaml
apiVersion: kwok.sigs.run-ai.com/v1beta1
kind: NodePool
metadata:
labels:
app.kubernetes.io/name: nodepool
app.kubernetes.io/instance: nodepool-sample
app.kubernetes.io/part-of: kwok-operator
app.kubernetes.io/managed-by: kustomize
app.kubernetes.io/created-by: kwok-operator
name: nodepool-sample
spec:
nodeCount: 15
nodeTemplate:
apiVersion: v1
metadata:
labels:
app.kubernetes.io/instance: nodepool-sample
name: nodepool-sample
spec:
nodeCount: 15
nodeTemplate:
apiVersion: v1
metadata:
annotations:
node.alpha.kubernetes.io/ttl: "0"
labels:
kubernetes.io/hostname: kwok-node
kubernetes.io/role: agent
type: kwok
spec: {}
status:
allocatable:
cpu: 32
memory: 256Gi
pods: 110
capacity:
cpu: 32
memory: 256Gi
pods: 110
nodeInfo:
architecture: amd64
bootID: ""
containerRuntimeVersion: ""
kernelVersion: ""
kubeProxyVersion: fake
kubeletVersion: fake
machineID: ""
operatingSystem: linux
osImage: ""
systemUUID: ""
phase: Running
annotations:
node.alpha.kubernetes.io/ttl: "0"
labels:
kubernetes.io/hostname: kwok-node
kubernetes.io/role: agent
type: kwok
spec: {}
status:
allocatable:
cpu: 32
memory: 256Gi
pods: 110
capacity:
cpu: 32
memory: 256Gi
pods: 110
nodeInfo:
architecture: amd64
bootID: ""
containerRuntimeVersion: ""
kernelVersion: ""
kubeProxyVersion: fake
kubeletVersion: fake
machineID: ""
operatingSystem: linux
osImage: ""
systemUUID: ""
phase: Running
```
2. Apply the NodePool CR to your Kubernetes cluster:
Expand All @@ -127,51 +131,83 @@ To use the Kwok Operator to manage deployments and run the pods on top the nodes
1. ensure the namespace is exist
2. Define a DeploymentPool custom resource (CR) with your desired configuration. Example:
```yaml
apiVersion: kwok.sigs.run-ai.com/v1beta1
kind: DeploymentPool
metadata:
labels:
app.kubernetes.io/name: deploymentpool
app.kubernetes.io/instance: deploymentpool-sample
app.kubernetes.io/part-of: kwok-operator
app.kubernetes.io/managed-by: kustomize
app.kubernetes.io/created-by: kwok-operator
name: deploymentpool-sample
namespace: default # -----> change if needed
spec:
deploymentTemplate:
apiVersion: apps/v1
metadata:
name: kwok-operator
labels:
app.kubernetes.io/name: deployment
app.kubernetes.io/instance: deployment-sample
app.kubernetes.io/part-of: kwok-operator
app.kubernetes.io/managed-by: kustomize
app.kubernetes.io/created-by: kwok-operator
spec:
replicas: 3
selector:
matchLabels:
apiVersion: kwok.sigs.run-ai.com/v1beta1
kind: DeploymentPool
metadata:
labels:
app.kubernetes.io/name: deploymentpool
app.kubernetes.io/instance: deploymentpool-sample
app.kubernetes.io/part-of: kwok-operator
app.kubernetes.io/managed-by: kustomize
app.kubernetes.io/created-by: kwok-operator
name: deploymentpool-sample
namespace: default
spec:
deploymentTemplate:
apiVersion: apps/v1
metadata:
name: kwok-operator
labels:
app.kubernetes.io/name: deployment
app.kubernetes.io/instance: deployment-sample
app.kubernetes.io/part-of: kwok-operator
app.kubernetes.io/managed-by: kustomize
app.kubernetes.io/created-by: kwok-operator
spec:
replicas: 3
selector:
matchLabels:
app.kubernetes.io/name: deployment
app.kubernetes.io/instance: deployment-sample
app.kubernetes.io/part-of: kwok-operator
app.kubernetes.io/managed-by: kustomize
app.kubernetes.io/created-by: kwok-operator
template:
metadata:
labels:
app.kubernetes.io/name: deployment
app.kubernetes.io/instance: deployment-sample
app.kubernetes.io/part-of: kwok-operator
app.kubernetes.io/managed-by: kustomize
app.kubernetes.io/created-by: kwok-operator
template:
metadata:
labels:
app.kubernetes.io/name: deployment
app.kubernetes.io/instance: deployment-sample
app.kubernetes.io/part-of: kwok-operator
app.kubernetes.io/managed-by: kustomize
app.kubernetes.io/created-by: kwok-operator
spec:
containers:
- image: nginx
name: nginx
restartPolicy: Always
spec:
containers:
- image: nginx
name: nginx
restartPolicy: Always
```
---
To use the Kwok Operator to manage pods on top the nodes you provisioned above, follow these steps:
1. ensure the namespace is exist
2. Define a PodPool custom resource (CR) with your desired configuration. Example:
```yaml
apiVersion: kwok.sigs.run-ai.com/v1beta1
kind: PodPool
metadata:
labels:
app.kubernetes.io/name: podpool
app.kubernetes.io/instance: podpool-sample
app.kubernetes.io/part-of: kwok-operator
app.kubernetes.io/created-by: kwok-operator
name: podpool-sample
namespace: default
spec:
podCount: 5
podTemplate:
metadata:
name: kwok-operator
labels:
app.kubernetes.io/name: pod
app.kubernetes.io/instance: pod-sample
app.kubernetes.io/part-of: kwok-operator
app.kubernetes.io/managed-by: kustomize
app.kubernetes.io/created-by: kwok-operator
spec:
containers:
- image: nginx
name: nginx
restartPolicy: Always
```
## Troubleshooting
Expand Down
59 changes: 59 additions & 0 deletions api/v1beta1/podpool_types.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
/*
Copyright 2024.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package v1beta1

import (
corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)

// PodPoolSpec defines the desired state of PodPool
type PodPoolSpec struct {
PodCount int32 `json:"podCount"`
PodTemplate corev1.Pod `json:"podTemplate"`
}

// PodPoolStatus defines the observed state of PodPool
type PodPoolStatus struct {
Conditions []metav1.Condition `json:"conditions,omitempty" patchStrategy:"merge" patchMergeKey:"type" protobuf:"bytes,1,rep,name=conditions"`
ObservedGeneration int64 `json:"observedGeneration,omitempty" protobuf:"varint,2,opt,name=observedGeneration"`
}

//+kubebuilder:object:root=true
//+kubebuilder:subresource:status

// PodPool is the Schema for the podpools API
type PodPool struct {
metav1.TypeMeta `json:",inline"`
metav1.ObjectMeta `json:"metadata,omitempty"`

Spec PodPoolSpec `json:"spec,omitempty"`
Status PodPoolStatus `json:"status,omitempty"`
}

//+kubebuilder:object:root=true

// PodPoolList contains a list of PodPool
type PodPoolList struct {
metav1.TypeMeta `json:",inline"`
metav1.ListMeta `json:"metadata,omitempty"`
Items []PodPool `json:"items"`
}

func init() {
SchemeBuilder.Register(&PodPool{}, &PodPoolList{})
}
Loading

0 comments on commit a5fd521

Please sign in to comment.