Unable to install StackStorm packs via volume method in K8s deployment

[ektasaxena04]

Hi,

I have created all the 3 pvc's and updated values.yaml as per https://github.com/StackStorm/stackstorm-k8s#install-custom-st2-packs-in-the-cluster

volumes:
    enabled: true
    packs:
      persistentVolumeClaim:
        claim-name: pvc-st2-packs
    virtualenvs:
      persistentVolumeClaim:
        claim-name: pvc-st2-virtualenvs
    configs:
      persistentVolumeClaim:
        claim-name: pvc-st2-pack-configs

claim for virtualnvs is not getting bind to the required pods. following mounts have been created.

ubuntu@Bastion:/opt/stackstorm$ ll
total 24
drwxr-xr-x 6 root root 4096 Apr 21 05:53 ./
drwxr-xr-x 9 root root 4096 Apr 20 10:50 ../
drwxrwxrwx 2 root root 4096 Apr 20 11:23 configs/
drwxrwxrwx 2 root root 4096 Apr 21 05:53 configs-helm/
drwxrwxrwx 2 root root 4096 Apr 20 10:51 packs/
drwxrwxrwx 2 root root 4096 Apr 20 11:22 virtualenvs/

Error :

Normal  WaitForPodScheduled  3m19s (x22 over 72m)  persistentvolume-controller  waiting for pods stackstorm-ha-job-st2-register-content-52rn9,stackstorm-ha-st2sensorcontainer-f4796d8b5-qm5hd,stackstorm-ha-st2actionrunner-7966ddc66d-svmgb,stackstorm-ha-st2actionrunner-7966ddc66d-hv6jv,stackstorm-ha-st2actionrunner-7966ddc66d-7gjqs,stackstorm-ha-st2client-5756b856d6-vzs75,stackstorm-ha-st2api-649489f4c9-lnz7b to be schedule.

All the 3 pods mentioned in the above error message are also in pending state and unable to bind the volume.

yaml for pvc.virtualenvs

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pvc-st2-virtualenvs
spec:
  accessModes:
  - ReadOnlyMany
  resources:
    requests:
      storage: 100Mi
status: {}

Please help.

[arm4b]

@cognifloyd Any pointers to help with the initial debugging and configuration?

[cognifloyd]

Why is your PVC using ReadOnlyMany?

[cognifloyd]

What about ReadWriteMany?

And if that doesn't work, can you create a dummy pod with your PVC to test that the PersistentVolume is working?

[ektasaxena04]

Thanks, Volume got bound to the required pods with accessmodes - ReadWriteOnce. It worked by adding labels to the pvc

This is the yaml

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  labels:
    io.kompose.service: stackstorm-virtualenvs
  name: pvc-st2-virtualenvs
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 100Mi

Thanks a lot :)

[cognifloyd]

So, is it working now?

[cognifloyd]

Note: I believe you will run into issues using st2 pack install with ReadWriteOnce if any of the actionrunner pods run on different nodes. If they all happen to be on the same node, then all the pods on that node will be able to write, which is required to install packs. So, if you can, I suggest using ReadWriteMany.

What about the Jobs - did the jobs work or were they stuck pending as well? the Jobs also need write access.

[ektasaxena04]

i am unable to change now, i tried to delete the pvc to reapply the changes(i.e. to ReadWriteMany) but the pvc for virtualenvs is in terminating status since many hours now.

This is what i got from the describe command

Capacity:      1Gi
Access Modes:  RWO
VolumeMode:    Filesystem
Used By:       stackstorm-ha-st2actionrunner-7966ddc66d-4c5h6
               stackstorm-ha-st2actionrunner-7966ddc66d-6gtvh
               stackstorm-ha-st2actionrunner-7966ddc66d-mztgr
               stackstorm-ha-st2actionrunner-7966ddc66d-p8q5h
               stackstorm-ha-st2actionrunner-7966ddc66d-srrzg
               stackstorm-ha-st2api-649489f4c9-2zrgc
               stackstorm-ha-st2api-649489f4c9-xqmnb
               stackstorm-ha-st2client-5756b856d6-b92f6

               stackstorm-ha-st2sensorcontainer-f4796d8b5-qm5hd
Events:        <none>