[Bug] Deleting EFS Addon Removes CSIDriver Resource

[CrisNevares]

What were you trying to accomplish?

Add, then remove the aws-efs-csi-driver addon in a Fargate-only cluster.

Because the efs.csi.aws.com CSIDriver resource existed prior to the creation of the aws-efs-csi-driver addon, I would expect that CSIDriver resource to persist in the cluster if the aws-efs-csi-driver is added and subsequently removed.

What happened?

The efs.csi.aws.com CSIDriver resource was deleted

How to reproduce it?

Create a Fargate-only cluster.
Check that the efs.csi.aws.com CSIDriver already exists.
Create the aws-efs-csi-driver addon.
Delete the aws-efs-csi-driver addon.
Notice the efs.csi.aws.com CSIDriver has been deleted.

Logs

Create an eks-test1 cluster, in order to check default csi drivers:

---

$ cat eks-test1.yml
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig

metadata:
  name: eks-test1
  region: us-east-1
  version: "1.27"
  tags:
    environment: dev

kubernetesNetworkConfig:
  ipFamily: IPv4

fargateProfiles:
  - name: fp-default
    selectors:
      - namespace: default
  - name: fp-kube
    selectors:
      - namespace: kube-system


$ eksctl create cluster -f eks-test1.yml

Check csidriver:

---

$ kubectl get csidriver
NAME              ATTACHREQUIRED   PODINFOONMOUNT   STORAGECAPACITY   TOKENREQUESTS   REQUIRESREPUBLISH   MODES        AGE
efs.csi.aws.com   false            false            false             <unset>         false               Persistent   34m

$ kubectl get csidriver -o yaml
apiVersion: v1
items:
- apiVersion: storage.k8s.io/v1
  kind: CSIDriver
  metadata:
    annotations:
      kubectl.kubernetes.io/last-applied-configuration: |
        {"apiVersion":"storage.k8s.io/v1","kind":"CSIDriver","metadata":{"annotations":{},"name":"efs.csi.aws.com"},"spec":{"attachRequired":false}}
    creationTimestamp: "2024-01-27T09:30:56Z"
    name: efs.csi.aws.com
    resourceVersion: "272"
    uid: 534abd7e-55c2-45d6-94e9-40b80f5b26df
  spec:
    attachRequired: false
    fsGroupPolicy: ReadWriteOnceWithFSType
    podInfoOnMount: false
    requiresRepublish: false
    storageCapacity: false
    volumeLifecycleModes:
    - Persistent
kind: List
metadata:
  resourceVersion: ""

Install aws-efs-csi-driver addon adding this to eks-test1.yml:

---

addons:
  - name: aws-efs-csi-driver
    resolveConflicts: overwrite

$ eksctl create addon -f eks-test1.yml

I now have efs-csi-controller pods stuck in pending, due to node requirement. But also note the efs csi driver remains intact:

---

$ kubectl get csidriver
NAME              ATTACHREQUIRED   PODINFOONMOUNT   STORAGECAPACITY   TOKENREQUESTS   REQUIRESREPUBLISH   MODES        AGE
efs.csi.aws.com   false            false            false             <unset>         false               Persistent   38m

$ kubectl get csidriver -o yaml
apiVersion: v1
items:
- apiVersion: storage.k8s.io/v1
  kind: CSIDriver
  metadata:
    annotations:
      kubectl.kubernetes.io/last-applied-configuration: |
        {"apiVersion":"storage.k8s.io/v1","kind":"CSIDriver","metadata":{"annotations":{},"name":"efs.csi.aws.com"},"spec":{"attachRequired":false}}
    creationTimestamp: "2024-01-27T09:30:56Z"
    name: efs.csi.aws.com
    resourceVersion: "6373"
    uid: 534abd7e-55c2-45d6-94e9-40b80f5b26df
  spec:
    attachRequired: false
    fsGroupPolicy: ReadWriteOnceWithFSType
    podInfoOnMount: false
    requiresRepublish: false
    storageCapacity: false
    volumeLifecycleModes:
    - Persistent
kind: List
metadata:
  resourceVersion: ""

I now remove the addon:

---

$ eksctl delete addon --name aws-efs-csi-driver --cluster eks-test1

Removes the csidriver, not unexpectedly in retrospect:

---

$ kubectl get csidriver
No resources found

Anything else we need to know?

Versions

$ eksctl info
eksctl version: 0.167.0
kubectl version: v1.28.2
OS: linux

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[CrisNevares]

Bump. This is still an issue.

[TiberiuGC]

Hi @CrisNevares , you just need to use the --preserve flag for the delete command, this way you'll delete the addon, but preserve the underlying K8s resources.

$ eksctl delete addon --name aws-efs-csi-driver --cluster eks-test1 --preserve

I'll make a small amendment to the docs so this functionality is explained there as-well.