pgbackrest backup fails with "primary pod not found"

[infestdead]

Report

Operator is 2.5.0, images used are main-ppg16-postgres. Since a day ago we are getting an error that "primary pod is not found".

Seems now label : postgres-operator.crunchydata.com/role=primary is used,

but the operator in the go code looks for the value of master:

./percona/watcher/wal.go here line:

func getPrimaryPod(ctx context.Context, cli client.Client, cr *pgv2.PerconaPGCluster) (*corev1.Pod, error) {
        podList := &corev1.PodList{}
        err := cli.List(ctx, podList, &client.ListOptions{
                Namespace: cr.Namespace,
                LabelSelector: labels.SelectorFromSet(map[string]string{
                        "app.kubernetes.io/instance":             cr.Name,
                        "postgres-operator.crunchydata.com/role": "master",
                }),
        })

on line 179

However none of the pods have labels have master, only primary.

More about the problem

│ 2024-12-06T12:11:18.453Z ERROR WALWatcher get latest commit timestamp {"controller": "perconapgcluster", "controllerGroup": "pgv2.percona.com", "controllerKind": "PerconaPGCluster", "PerconaPGCluster": {"name":"postgresql- │
│ cluster","namespace":"postgresql-system"}, "namespace": "postgresql-system", "name": "postgresql-cluster", "reconcileID": "6caad6f9-e3a6-45ab-ae12-e948b7870b87", "error": "primary pod not found", "errorVerbose": "primary pod not found │
│ \ngithub.com/percona/percona-postgresql-operator/percona/watcher.init\n\t:1\nruntime.doInit1\n\t/usr/local/go/src/runtime/proc.go:7176\nruntime.doInit\n\t/usr/local/go/src/runtime/proc.go:7143\nruntime.main\n\t/usr/loca │
│ l/go/src/runtime/proc.go:253\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1695"} │
│ runtime.goexit │

Steps to reproduce

1. Deploy standard cluster.
2. Setup cluster.
3. Run backup.

Versions

1. Kubernetes.1.28
2. Operator 2.5.0
3. Database 16

Anything else?

No response

[hors]

Hi @infestdead, please do not use images with main tags :( These images are unstable (auto daily builds)

[infestdead]

@hors thanks for the heads up, what are considered stable build tags?

[hors]

E.g. for PGO 2.5 we have the following list of images: https://docs.percona.com/percona-operator-for-postgresql/2.0/images.html