Kubernetes scan skips Pods due to file size

[MrSecure]

I've been testing the Kubernetes scanning feature on a cluster which has ~325 pods, many which are in the Completed state, and awaiting cleanup. However, because the Pods.yaml is larger than 5MB, ALL Pods get skipped.

Scan Command:

kics scan -q /usr/local/opt/kics/share/kics/assets/queries  \
  -p "kuberneter::*:*:*" -v --no-progress -o kicsout \
  --report-formats all --type Kubernetes

... omitted for clarity ...

Error: failed to read sources: failed to get file content: kicsout/kics-extract-kuberneter-11-03-2022/core/v1/Pods.yaml: file size limit exceeded

Partial output from ls -l:

6352203 Nov  3 14:22 kicsout/kics-extract-kuberneter-11-03-2022/core/v1/Pods.yaml

Pod Count

grep -c ^kind kicsout/kics-extract-kuberneter-11-03-2022/core/v1/Pods.yaml
323

5 MB Restriction Code:

kics/pkg/kics/service.go

Lines 132 to 145 in a52e458

	func getContent(rc io.Reader, data []byte) (*Content, error) {
	maxSizeMB := 5 // Max size of file in MBs
	var content []byte
	countLines := 0
	c := &Content{
	Content: &[]byte{},
	CountLines: 0,
	}
	for {
	if maxSizeMB < 0 {
	return c, errors.New("file size limit exceeded")
	}

Has anyone else been hindered by this limit? It looks like the intent is to prevent a resource exhaustion attack (see: #2156), however, there's no user control to adjust the limit.

Is there a way to specify a Kuberneter path which excludes pods in specific states (or includes only pods in desired states)?

What would be a better approach to handle large files - Process the first 5MB, rather than discarding the entire file because it's too large? Present a configuration option to the user to set a higher limit? Adjust Kuberneter library to split large files into smaller pieces?

Can anyone lend a hand with creating a PR to adjust the limit in a sustainable manner? (I'd prefer to avoid changing one arbitrary fixed limit for another)

[YuvalFireFly]

I've encountered a similar issue when using KICS to scan Terraform plan files. These plan files can grow significantly in size, especially for complex infrastructure deployments. While the 5MB limit is a protective measure, it can be quite restrictive for scenarios like Terraform plans.

[gabriel-cx]

Hi @MrSecure @YuvalFireFly ,

Thank you for your inputs!
We are working on this PR that will tackle the situation you mentioned. We will merge it asap.

[gabriel-cx]

Hi @MrSecure @YuvalFireFly ,

FYI, a new KICS version was released just now!
With KICS v1.7.12 you can now use the flag --max-file-size to set the max file size permitted for scanning.

Check this info for more details about the flag.

[YuvalFireFly]

@gabriel-cx Thank you very much!