Add configuration for secret related checks

[pr3l14t0r]

New feature motivation

All checks related to secret detection lead to a massive amount of false positives. This could get prevented if there would be a possibility to configure those checks, since most of them use the detect-secrets package.

Use case:
The checks awslambda_function_no_secrets_in_variables and ecs_task_definitions_no_environment_secrets do use the detect-secrets package.

Unfortunately there is no possibility (at least no documented one) that allows to configure the checks done with that package. The repository documents how you can use a customized settings file, which can for example adjust the threshold of Base64HighEntropyString etc.

Could it be possible to add the JSON like config to the prowler's config.yaml?

Furthermore it would be nice to have an option to filter out environment variable names.

Example:
In an AWS TaskDefinition i use a variable named GCP_SECRET_ARN. This contains an ARN to AWS_SECRETS_MANAGER service which will be used during runtime of a script to retrieve a JSON, since it is not really an option to mount a JSON with newline characters into an environment variable as secret. Thus i would like to filter out for name patterns like "*SECRET_ARN".
Would this make sense to you, too?

As documented for detect-secrets, one could use the following parameter:

  --exclude-secrets EXCLUDE_SECRETS
                        If secrets match this regex, it will be ignored.

This could already help me out for my use-case.

Solution Proposed

Add an option to the config.yaml that would apply to all scans which use detect-secrets.

My pseudo example:

aws:
  # configure entropy threshold
  awslambda_function_no_secrets_in_variables_Base64HighEntropyString_limit: 7.0
  # Configure a  regex-based ignore-pattern for variable names
  ecs_task_definitions_no_environment_secrets_ignore_patterns:
    - ".*SECRET_ARN"
    - ".*PASSWORD_ARN"

And then fetch those values in the corresponding python scripts...

I know: configuring this is more a global option than a "per-check" option. So maybe it makes even more sense to create a default config-file for detect-secrets which someone could adjust. Just as with the yaml files.

Describe alternatives you've considered

I did consider to overwrite the python scripts for the mentioned checks, but this is somewhat unfeasible to have it within a deployment CI-CD deployment.. It would be more easy if i could just specify or change a global settings file for the detect-secrets package.

Additional context

Maybe i have overseen something in the documentation or something in the code itself. So if there already is a solution for my presented use cases, i would already like to apologize.

[toniblyx]

Thanks @pr3l14t0r, this is a good idea to enhance our Mutelist feature! what do you think @jfagoagas @sergargar ?

[jfagoagas]

Hi @pr3l14t0r, I'm going to answer you inline:

• Regarding the detect-secrets Python package configuration, as far as we know there is no way to configure BASE64_LIMIT and HEX_LIMIT but we can find a workaround to do that.
• About the environment variables I think the Mutelist should be the way to filter out those environment variables but if the value within is an ARN, detect-secrets shouldn't report this as a secret.

Thanks for using Prowler 🚀

[pr3l14t0r]

[..]but if the value within is an ARN, detect-secrets shouldn't report this as a secret.

@jfagoagas Well... the check ecs_task_definitions_no_environment_secrets detects a secret in my task-definition config with the following STATUS_EXTENDED explanation:

Potential secret found in variables of ECS task definition REDACTED-name-of-taskdefinition-resource with revision 1 -> Secret Keyword on line 2.

I don't know what exactly the Secret Keyword on line 2 is referring to, but my environment variables are the following:

            "environment": [
                {
                    "name": "GCP_SECRET_ARN",
                    "value": "arn:aws:secretsmanager:<aws-region>:<aws-account-id>:secret:<name-of-my-secret>"
                },
                {
                    "name": "S3_BUCKET",
                    "value": "<REDACTED-name-of-my-output-bucket>"
                },
                {
                    "name": "AWS_STACK",
                    "value": "<name-of-my-aws-stack>"
                }
            ]

does it get flagged because if have named the environment variable GCP_SECRET_ARN - containing the term *secret* ?

[pr3l14t0r]

About the environment variables I think the Mutelist should be the way to filter out those environment variables

That's not possible at the moment... In my example from above the ResourceName is the name of the TaskDefinition, which makes sense. But this won't allow me to filter for the specific variable name

[jfagoagas]

Hi @pr3l14t0r,

I don't know what exactly the Secret Keyword on line 2 is referring to, but my environment variables are the following:

That line number comes from the AWS ECS API response, sometimes there are differences between the format returned by the API and how is shown in the AWS Console.

does it get flagged because if have named the environment variable GCP_SECRET_ARN - containing the term secret ?

I need to review the detect-secrets engine but it seems to be the reason and also due to the format of the value.

That's not possible at the moment... In my example from above the ResourceName is the name of the TaskDefinition, which makes sense. But this won't allow me to filter for the specific variable name

You are right, we have identified some other scenarios like this and we are thinking about ways to improve the mutelist to allow combinations but for that we need to include more in the finding object, like the secret name for this use case.

The only I can recommend you now is to move that secret from the environment object in the ECS Task Definition to the secrets key. With that, ECS will get the secret value and inject it into the container's runtime as an environment variable but that would need you to change a little bit your logic since I suppose you are calling the AWS Secrets Manager API from your code to retrieve that secret value. You can read more about here.

Another option we are evaluating is to include the configuration you suggested with ecs_task_definitions_no_environment_secrets_ignore_patterns or something similar.

Thanks for using Prowler 🚀