Test opnfv/functest-kubernetes-security:v1.23 run_tests -t kube_bench_master not working on K8S cluster where PSP configured

[sultetveny]

Overview

I executed the kube_bench_master test suite on a K8S cluster where strict PSP defined and the test failed.

How did you run kube-bench?

podman run -it --env-file ~/opnfv/env \
-v ~/opnfv/ca.pem:/home/opnfv/functest/ca.pem:Z \
-v ~/opnfv/config:/root/.kube/config:Z \
-v ~/opnfv/results:/home/opnfv/functest/results:Z \
-v ~/opnfv/repositories.yml:/home/opnfv/functest/repositories.yml:Z \
-v ~/opnfv/cluster-admin.pem:/home/opnfv/functest/cluster-admin.pem:Z \
-v ~/opnfv/cluster-admin-key.pem:/home/opnfv/functest/cluster-admin-key.pem:Z \
opnfv/functest-kubernetes-security:v1.23 /bin/bash

run_tests -t kube_bench_master

What happened?

Test case failed. For more information please check attached file
functest-kubernetes.log
functest-kubernetes.debug.log

Log from the cluster:

kubectl get events -A --watch

NAMESPACE          LAST SEEN   TYPE      REASON         OBJECT                  MESSAGE
kube-bench-mzk6x   3s          Warning   FailedCreate   job/kube-bench-master   Error creating: pods "kube-bench-master-" is forbidden: PodSecurityPolicy: unable to admit pod: [spec.securityContext.hostPID: Invalid value: true: Host PID is not allowed to be used spec.volumes[0].hostPath.pathPrefix: Invalid value: "/var/lib/etcd": is not allowed to be used spec.volumes[1].hostPath.pathPrefix: Invalid value: "/etc/kubernetes": is not allowed to be used spec.volumes[2].hostPath.pathPrefix: Invalid value: "/usr/bin": is not allowed to be used]

What did you expect to happen:

I expected the test case executed successfully.

Environment

[What is your version of Kubernetes? (run kubectl version or oc version on OpenShift.)]

Client Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.15", GitCommit:"b84cb8ab29366daa1bba65bc67f54de2f6c34848", GitTreeState:"clean", BuildDate:"2022-12-08T10:49:13Z", GoVersion:"go1.17.13", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.15", GitCommit:"b84cb8ab29366daa1bba65bc67f54de2f6c34848", GitTreeState:"clean", BuildDate:"2022-12-08T10:42:57Z", GoVersion:"go1.17.13", Compiler:"gc", Platform:"linux/amd64"}

Running processes

[Please include the output from running ps -eaf | grep kube on the affected node. This will allow us to check what Kubernetes processes are running, and how this compares to what kube-bench detected.]

Configuration files

Anything else you would like to add:

Ufortunately

• no configurable parameter (like namespace) available for execution, to make sure the proper PSP will be allocated for these PODs
• because only the half of the namespace are predictable the other half is randomly generated, it's not possible to prepare the environment (pre create ns, sa, roler, rolebinding, psp)
  The only way I found is to disable PSP on cluster level, but it's not so sophisticated. Maybe sa/role/rolebinding/psp should be created automatically, for this purpose.