🐛 [BUG] - KubeMonitoring chart produces a diff on every helm upgrade

[IvoGoman]

Priority

(Medium) I'm annoyed but I'll live

Description

Related to #546

The kube-monitoring plugin creates a diff on every helm diff, this leads to reoccurring reconciliations.
The problematic file is Secret/tls-prometheus-alertmanager-auth which looks to be static in the chart: https://github.com/cloudoperators/greenhouse-extensions/blob/main/kube-monitoring/charts/templates/alertmanager-tls-secret.yaml.

The secret values used are mounted via secretRef into the Plugin. In this case from tls-greenhouse-prometheus-auth, which is the secret for Certificate greenhouse-prometheus-auth.

greenhouse-extensions/alerts/charts/templates/ca-secret-issuer-cert.yaml

Lines 38 to 39 in c8fbfb1

	annotations:
	"helm.sh/hook": pre-install, pre-upgrade

Since the Certificate uses helm lifecycle hooks and no "helm.sh/hook-delete-policy" is defined, it is recreated on every helm deploy If no hook deletion policy annotation is specified, the before-hook-creation behavior applies by default.

"Secret/tls-prometheus-alertmanager-auth: [\n {\n \"value\": \"***** - after\",\n \"op\": \"replace\",\n \"path\": \"/data/tls.crt\"\n },\n {\n \"value\": \"***** - after\",\n \"op\": \"replace\",\n \"path\": \"/data/tls.key\"\n }\n]"} 

Reproduction steps

1. Go to the organisation namespace 
2. `helm history kube-monitoring`
3. see frequent upgrades of the plugin

Screenshots

![DESCRIPTION](LINK.png)

[richardtief]

@IvoGoman I think this is solved by keeping the cert in greenhouse-prometheus-auth generated by the alerts plugin static.

[IvoGoman]

Ah yes, I overlooked that both certs are enclosed by the same if-condition. Can be closed in favour of #546