K8SPG-549: merge upstream changes (#720) · percona/percona-postgresql-operator@71d7735

Commit

K8SPG-549: merge upstream changes (#720)

* Update internal/pgbackrest/config.go

Co-authored-by: Chris Bandy <[email protected]>

* Changed code to use strings.EqualFold() for case-insensitive comparison.

* Update pgBackRest repo option logic

When taking a backup, PGO tries to help by not allowing the user
to pass the "--repo" option. However, the current method for catching
this results in catching any option that begins with "--repo", which
prevents users from passing in perfectly valid options.

This commit corrects the flag check to only block on exact matches of
"--repo".

Issue: [sc-16128]

* Bumping kubebuilder:validation:Maximum for major PostgresVersion to 15.

* Add constants for services registered with the IANA

The PostgreSQL and pgBackRest protocols are both registered with the
IANA according to RFC 6335.

See: https://www.iana.org/assignments/service-names-port-numbers

* Get primary name after waiting for redeploy

* Update kuttl tests for Postgres 15 public schema updates

With Postgres 15, the removal of PUBLIC creation permisson on the
public schema requires updates to our kuttl test logic. This
commit allows the tests to perform as expected with these new
changes by creating/referencing new schemas as needed. Note that
these changes should not impact Postgres versions < 15.

Issue: [sc-16289]

* Alter make generate-kuttl to quiet output (#3442)

* Pass the upgrade-check URL as an argument

The global value is now a constant and somewhat easier to reason about.

* Handle upgrade-check panics in a single place

* Start and stop upgrade-check using controller-runtime

Blocking functions can be added to a controller-runtime Manager so that
they start after caches have started and synced. They also stop before
caches have stopped.

* Added namespace limiters to all client.List() calls in pgbackrest and volumes files in the controller. Changed List calls to consistently use ListOptions struct or individual ListOption arguments, but not a mixture of both.

Issue: [sc-13871]
Issue: [sc-16139]
Issue: CrunchyData/postgres-operator#3058
Issue: CrunchyData/postgres-operator#3364

* updated urls from github.io to the access portal ensuring
users are looking at the latest documentation
Issue: [sc-16478]

* Move environment logging into main()

* controller-runtime Source that emits a constant Event periodically

* Single-method implementations of controller-runtime Client

* Bridge API client

Issue: [sc-16285]

* Bridge installation reconciler

Issue: [sc-16285]

* Use optimistic concurrency and log retries

The Kubernetes clients provided by controller-runtime Manager fetch from
a cache. When fetching then writing back a single object, one should use
the object's resourceVersion to avoid races and lost updates.

Issue: [sc-16285]
See: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency

* Hide the progress bar when calling curl in tests

* Migration assistance (#3445)

* Log errors when the PostgreSQL data directory is wrong

The postgres-startup container now reports when it finds the installed
PostgreSQL binaries do not match the specified PostgreSQL version.

Some storage providers do not mount the PostgreSQL data volume with
correct ownership or permissions. The postgres-startup container now
prints those attributes of parent directories when it cannot create or
modify a needed file or directory.

Issue: [sc-11804]
Issue: CrunchyData/postgres-operator#2870

Co-authored-by: @cbandy

* Change owner of the PostgreSQL directory at startup
PostgreSQL won't to start unless it owns the data directory. Kubernetes
sets the group according to fsGroup but not the owner.

The postgres-startup container now recreates the data directory to give
it a new owner when permissions are sufficient to do so. It now raises
an error when the owner is incorrect and cannot be changed.

Issue: [sc-15909]
See: https://docs.k8s.io/tasks/configure-pod-container/security-context/

Co-authored-by: @cbandy

* Add KUTTL test for migration from third-party PGSQL

Issue: [sc-15909]

* Add concurrencyPolicy to backup CronJobs

Only one pgBackRest backup can run at a time. A scheduled backup that
runs too long can cause the next scheduled backup to fail and retry
multiple times. Skip that next one instead.

Co-authored-by: Scott Zelenka <[email protected]>
Issue: CrunchyData/postgres-operator#3439

* Require SCRAM authentication of the monitoring user

The PostgreSQL STIG requires that password authentication be done using
scram-sha-256.

Co-authored-by: Scott Zelenka <[email protected]>
Issue: CrunchyData/postgres-operator#3424
See: https://www.stigviewer.com/stig/crunchy_data_postgresql/2022-06-13/finding/V-233519

* Limit the monitoring user to local connections

Issue: [sc-12218]

* Remove disable exporter tls test

Checking that tls has been disabled on a cluster (where it was
previously enabled) is difficult. This is because we need to wait for
the instance pod to be redeployed without tls configuration. We are
removing case from the kuttl test with plans to ensure we have the same
coverage in go tests in the future.

Issue: [sc-16572]

* Pin GitHub actions to Ubuntu 20.04

The Ubuntu 22.04 runners include ShellCheck v0.8 which has new rules.

Issue: [sc-13394]

* Added a warning noticed ot the pgadmin 4 architecture docs
to let users know there are compatibility issues with pgAdmin 4
and pg15

Issue: [sc-16516]

* Adding uniqueness to cluster names when testing service type changes to work around race condition that is causing these tests to flake.

[sc-16571]

* Moving PG Major Upgrades API to postgres-operator repo.

[SC-16347]

* Add PGUpgrades to the controller-gen TODO hack

Issue: [sc-16347]

* Do not configure JIT for the monitoring user

PostgreSQL 10 does not have a "jit" parameter. The current release of
pgMonitor includes this fix and correctly applies it to specific
versions of PostgreSQL.

This partially reverts commit df492f1.

Issue: [sc-15755]
See: CrunchyData/pgmonitor#295

* Update security context kuttl test for OCP 4.11

Adjusts the SCC check to support the 'restricted-v2' SCC in addition
to the 'restricted' SCC.

* Make the TTL of pgBackRest backups configurable

The default retention of one failed backup Job can leave a Job and its
Pods in a failed state indefinitely. The TTL setting lets someone choose
how long they want Jobs, Pods, and their logs to be available.

This field is functional in Kubernetes 1.21 and OpenShift 4.8 where the
TTLAfterFinished feature gate is enabled by default.

Issue: [sc-14014]
Issue: CrunchyData/postgres-operator#3444

* Bumping pgMonitor to v4.8.0.

[SC-16701]

* Update Version 5.2.0 to 5.3.0

Update PGO and Postgres versions for 5.3.0.

Issue: [sc-16943]

* Add Postgres 15 RELATED_IMAGE environment variable

This commit adds the Postgres 15 RELATED_IMAGE environment variable
to manager.yaml

Issue: [sc-16943]

* Add entries to bundle.relatedImages.yaml

Add entries for Postgres 15, Postgres 14 with GIS 3.3 and
Postgres 15 with GIS 3.3 images to the bundle.relatedImages.yaml
file.

Issue: [sc-16943]

* Update the minimum Kubernetes and OCP OLM versions

PGO 5.3.0 will support, per the documentation, Kubernetes 1.22-1.25
and OpenShift 4.8-4.11. However, the OLM bundle minKubeVersion must
match the minimum OCP's included Kubernetes version, which is 1.21 per
https://access.redhat.com/solutions/4870701.

Therefore, this commit sets 'com.redhat.openshift.versions' to v4.8 and
'minKubeVersion' to 1.21.0 for our OLM bundle generation.

Issue: [sc-16943]

* Helm OCI Release Notes

Issue: [sc-16943]

* Add docs for helm oci (#3493)

Co-authored-by: Chris Bandy <[email protected]>
Issue: [sc-16938]

Co-authored-by: Chris Bandy <[email protected]>

* Update Postgres version 15.0 to 15.1

* Update comment for Metadata (#3496)

Metadata is used by postgrescluster and pgupgrade

* pgMonitor v4.8.0 Release Note

Issue: [sc-16943]

* Bump Build Number for PG 14 PostGIS 3.3

* Fix Typo for CLI in Release Notes

* Update the default Postgers image used for Kuttl tests

* Document Postgres 15 recovery_target_action behavior

Postgres 15 behaves the same as Postgres 14 in this regard.

* Remove the note about language in the pgBackRest docs

The pgBackRest documentation seems clear enough to me now.

* Integrating Major PG Upgrades controller logic and testing into PGO.

[sc-16348]

Co-authored-by: Tony Landreth <[email protected]>

* Set operator image tag to release v5.4.0

After pulling major-upgrades into postgres-operator, a new
image will be needed to install a fully functional operator.
This commit bumps the tag on the operator image to the
presently unreleased v5.4.0.

Issue: [sc-16349]

* Adds KUTTL_PG_UPGRADE_TO_VERSION parameter

A new parameter is added to decouple settings
between operator tests and upgrade tests.

Issue: [sc-17416]

* Update README.md

Fix installation, otherwise it is not working.

* Bumping min OCP version (#3509)

* Pin checks to Kube 1.25

* Simplify Makefile

A help target has been added that describes each target and groups them
by category.

Remove targets to push/pull images from gcr - now that we only have two
images in this repo manually running the podman commands will be fine

Remove option to push to docker daemon or build with sudo - with buildah
and podman we don't typically need these options

Update build targets - we had some logic in our image and binary
build targets that was overly complicated now that we only have two
images in this repo. Each binary and image has a single target used to
build that particular resource. The names of these targets have been
updated to improve readability.

Random cleanup
- Add phony targets
- Remove relics of the past
- remove images var that is now unused

* Simplify postgres-operator dockerfiles

This change simplifies the dockerfiles used to build our
postgres-operator and crunchy-postgres-exporter images. We remove the
concept of a base image and put all required layers in its own image.

The postgres-operator image is now build from ubi8-micro and the
exporter image is built using ubi8-micro.

Remove setup scripts used to gather pgmonitor resources. This logic has
been moved to the make get-pgmonitor and get-postgres-exporter targets

* Add a GeoJSON assertion to the PostGIS Kuttl test

Issue: [sc-13236]

* Update PGO upgrade docs

When upgrading to v5.4, Kustomize installations will require
deletion of the pgo-upgrade deployment.

Issue: [sc-16349]

* Update Copyright notices for 2023

* Add trivy action to catch CVEs (#3544)

Note: cron is set for testing purposes at the moment

Issue: [sc-17241]

* New generic function to dereference a non-nil pointer

* Stop using the k8s.io/utils module directly

The few functions we used were already available in an internal package.

* Ensure go.mod is tidy during pull request checks

We imported the "k8s.io/utils" module directly a few commits ago but
neglected to update the "go.mod" file.

* Update go.mod to avoid CVEs (#3548)

Issue: [sc-17241]

* Remove backup assertions from exporter test

This test is not interested in backups and completes faster without
those assertions.

Issue: [sc-17016]

* Remove backup assertions from streaming standby test

This test completes faster without those assertions.

Issue: [sc-17016]

* Correct the comments on CodeQL actions

CodeQL has changed and our Make targets have changed.

* Update OLM bundle generation logic for postgres major upgrade

This updates the OLM bundle generation logic to allow for the
inclusion of the 'postgres-operator-upgrade' controller, the
'crunchy-upgrade' image and related PGUpgrade CRD and functionality.
Related examples and documentation have been updated and all current
images are included as required.

Issue: [sc-17486]

* updated pgaudit extension upgrade directions

Issue: [sc-17351]

* Update docs/content/guides/major-postgres-version-upgrade.md

Co-authored-by: tjmoore4 <[email protected]>

* Update docs/content/guides/major-postgres-version-upgrade.md

Co-authored-by: tjmoore4 <[email protected]>

* Update docs/content/guides/major-postgres-version-upgrade.md

Co-authored-by: tjmoore4 <[email protected]>

* updated paragraph for clarity and grammar mistakes

* Update docs/content/guides/major-postgres-version-upgrade.md

Co-authored-by: tjmoore4 <[email protected]>

* Adding GitHub Actions Job for E2E testing. Refactoring kubernetes-k3d Job to use new K3d action for setting up k3d. Adjusting root-cert-ownership kuttl test to work with POSIX shell used in Github Actions.

[sc-17404]

* Adjusting create-kubeconfig.sh script to avoid race condition where the service-account-token secret was created, but the .data.token has not yet been populated.

* Fix tests to work on macOS Ventura

Shell utilities included in Ventura do not behave the same as GNU core
utilities, and OpenSSL has been replaced with LibreSSL.

* Updated go.mod

Issue [sc-17837]

* PGO will now turn "huge_pages" to "try" or "off" based on whether huge pages have been requested in the resource spec.

[sc-17766]

* Update docs/content/guides/huge-pages.md

Co-authored-by: Tony Landreth <[email protected]>

* Update standby configuration documentation

Update the docs to better reflect required value types in tutorial
documentation.

Issue: [sc-17928]

* Bump github.com/onsi/ginkgo to v2

Recent versions of "sigs.k8s.io/controller-runtime" have switched to "github.com/onsi/ginkgo/v2"
and dropped the "sigs.k8s.io/controller-runtime/pkg/envtest/printer" package.
This change to tests should make updating controller-runtime easier in the future.

* Update k3d and k3s URLs

Things have moved away from the Rancher domain and organization.
The URLs we were using redirect to these.

* Add tablespace alpha functionality (#3575)

* Adds the tablespaceVolumes field to the CRD;
* Adds basic tablespace functionality: mounts the volumes and preps them with correct permissions;
* Adds option for restoring with tablespaces (needs more testing);
* Adds docs/content/guides/tablespaces
* Adds a basic KUTTL test for creating a cluster with tablespaces;
* Updates the github test to add the feature gate

Issue: [sc-17759]

* Regularize kubebuilder RBAC annotations (#3586)

* Improvements to feature gate handling (#3599)

a) improve deploy-dev to allow user to easily set
b) print feature gates on startup

* Update docs (#3604)

Issue: [sc-18286]

* Breaks out trivy-scheduled-scans

Runs scheduled Trivy scans on the main and REL_4_7 branches.

Issue: [sc-17407]

* Removed Postgres 13 from RELATED_IMAGES.
Now that we've had 2 patch releases of Postgres 15 we are dropping postgres 13.

Issue [sc-17907]

* Updated the github actions works flow with latest container images

* changed kuttl pg version back to pg 14

* Fix e2e-other/postgis-cluster KUTTL (#3628)

Problem: PostGIS < v3.1 had trouble parsing result from
ST_AsGeoJSON with ST_AsText function.
Solution: Remove ST_AsText and check JSON directly

Issue: [sc-18159]

* Updated images to the latest versions and updated to postgres 15

Issue [sc-17991]

* Update examples/postgrescluster/postgrescluster.yaml

Co-authored-by: Chris Bandy <[email protected]>

* Clarifications to docs about restoring individual databases, plus additional links to CRD and cross-linking to improve readbility

* Changed Individual Databases paragraph into a warning, as per Andrew's suggestion.

* Add extra comma, as per bblattberg

* Refactor looping tests

Instead of setting an amount of time that these loops are allowed to
run, we can use an infinite loop that will fail when Kuttl hits its
timeout.

Issue [sc-18801]

* Clarify custom tls documentation (#3629)

* Add documentation about custom TLS secrets, clarifying replication secret common name
* Bump streaming standby test secrets to have 10y expiration

Issue: [sc-14645]

* document that wal files are not deleted

* typo in pgdata path

* more verbose wording

Co-authored-by: Drew Sessler <[email protected]>

* Change buildah for new build process (#3646)

Issue: [sc-19532]
Issue: [sc-18718]

* Update kustomization: patches (#3658)

* Update kustomization.yaml (#3655)

Update kyverno URI

* Update component page info

Issue: [sc-16032]

* Updating Keycloak example documentation

* Add warning blocks to hugepages doc.

[sc-18155]

* Renew Bridge installations

Issue: [sc-16285]

* Update exporter release target to build exporter

* Revamp demoting active to standby (#3661)

Issue: [sc-20085]

* Update depguard configuration for golangci-lint v1.53

The depguard v2 linter allows different rules to be applied to different
sets of files.

See: golangci/golangci-lint#3795
See: https://github.com/OpenPeeDeeP/depguard#config
See: https://golangci-lint.run/usage/linters/#depguard

* Update HA Architecture Doc

Revises the High Availability Algorithm section to bring it into
alignment with our current configuration.

Issue: [sc-20086]

* adding Postgres primary & replica cert to Secret

* Adding fix for hugepages/restore issue.

[sc-20758]

* Revise pgbouncer kuttl test to debug (#3683)

Issue: [sc-21015]

Co-authored-by: Chris Bandy <[email protected]>

* Fix README Links

* Latest updates

* Remove redundant trivy scans (#3695)

* Update test workflow

Issue: [sc-20728]

* Refactor Delete Namespace test

- Allow runner to define a namespace to delete through the makefile. This
will be useful if two sets of kuttl tests are running in the same env
- Move from 2 replicas to 1 to speed up the test
- Use single line volume claim specs

* Update Postgres Exporter version to 0.12.1 PGO-42

* Stop PostgresCluster reconciliation when required image not set

This update prevents empty image values from impacting the
reconciliation of a PostgresCluster. With this change, the impacted
cluster will not be updated until the necessary images are defined
and a corresponding warning event will be created. PostgresClusters
with images properly defined will reconcile normally.

Issue: [sc-21130]

* Update the PGUpgrade logic for missing image scenario

Adjusts the PGUpgrade logic to allow for easier recovery from a
missing image scenario. Specific Conditions are more clearly defined
and checking is added for the 'crunchy-upgrade' image.

A Kuttl test scenario is also added.

Issue: [sc-21130]

* Latest updates

* Adjust major upgrade kuttl tests

Move major-upgrade-missing-image test to e2e-other and create
shorter version, empty-image-upgrade.

* Add Discord Info to README

* Update Invite Code

* Quiet issues detected by golangci-lint v1.54.2

gosec v2.17.0 detects more cases of pointers to loop variables.

* Update apply_test to handle changes for Kubernetes 1.28+

Prior to 1.28.0, certain no-op server-side apply updates bumped
the resourceVersion value. For new Kubernetes versions this behavior
has been adjusted so that resourceVersion is not bumped.

This change adds an additional check for the server version to allow
the correct test to be executed.

* Remove kubectl '--short' flag from Github actions

The 'short' flag is now deprecated. The default output for
kubectl is now equivalent to the previous shortened output.

- https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.28.md#deprecation

* Exporter refactor. Remove all of crunchy-postgres-exporter from this repo. Refactor postgres-operator to hold the setup.sql and queries.yml files used by the postgres_exporter. Add logic to postgres-operator to replace the functionality that was in the start.sh script that will be removed from the exporter image. Adjust testing accordingly.

* Version updates

* This commit adds the ability to append custom queries to the default exporter queries. This ability is feature gated with the AppendCustomQueries flag. The exporter kuttl tests have been adjusted. Tests that need the feature gate turned on have been added to e2e-other. This commit also moves some things from the postgrescluster package to the pgmonitor package.

* Update workflow Kubernetes test versions

* Make PGO backwards compatible with older crunchy-postgres-exporter im… (#3728)

* Make PGO backwards compatible with older crunchy-postgres-exporter images.

* Github action kuttl tests are failing due to recent changes to PGO. Get pgmonitor files and set QUERIES_CONFIG_DIR.

Co-authored-by: Chris Bandy <[email protected]>

---------

Co-authored-by: Drew Sessler <[email protected]>
Co-authored-by: Chris Bandy <[email protected]>

* Ability to set custom ccp_monitoring pass

With this change users can update the
<cluster>-monitoring secret with a password, in either the `stringData`
or `data` secret fields, and remove the verifier to update the
ccp_monitoring password in postgres. After this users will need to
restart the exporter process by deleting the instance pods (a solution
that doesn't require full pod restarts is coming).

This change is to support monitoring for standby clusters. Before this a
standby cluster would be created without having access to the
ccp_monitoring user password that was replicated from the primary
cluster.

Test to ensure that the postgres_exporter can scrape postgres using a
custom ccp_monitoring password.

The tests will:
1. create a cluster with exporter enabled and ensure metrics can be
collected
2. Update the password and restart the pod
3. ensure that metrics can still be collected with the new password

Tests now require jq to run

- Refactor existing exporter tests

- Split out the tls and no-tls tests into separate directories.
- Update the tests to check the containers ready conditions
- Add collectors for test failures
- Include a test where we deploy a postgres-cluster with monitoring
enabled on a replica. It will then check that the exporter on the
replica can hit the query the database

- Update exporter to use pass file

- The exporter container now provides the ccp_monitoring password to
postgres_exporter using a password file instead of an environment
variable. With this, the password can be updated without requiring a
container restart. The path to the password file has also been added to
the exporter watcher logic meaning that the postgres_exporter process
will be restarted when either the queries directory or password file
change.

- The password change test is updated to check that the postgres_exporter
pid has been updated before trying to re-connect.

- Update pgmonitor 4.9

- Update to pull pgMonitor 4.9 queries. The new version has a specific
file for the global_dbsize metric that needs to be included when
generating the default queries

- Standby metrics testing

- Now that the password for the monitoring user is configurable, users can
configure a standby cluster to allow the exporter to query postgres
using the ccp_monitoring user. This change implements testing to
validate this use case. This test is included in e2e-other because it
requires more work. We need to ensure a backup is complete before
attempting to curl metrics. See note below*

Note: Move standby and replica tests to e2e-other

These two test can fail because of a scrape_error if a backup has not
completed. They need to be updated to check that a backup is complete
before attempting to collect metrics. There is a related story in our
backlog.

Due to the race condition, backup not being complete, they could pass or
fail. After a backup chack is in place they should be able to move back
into the e2e directory.

* Exporter tests overwrite annotation

As part of testing, kuttl will add annotations to instances pods before
and after a change. Kuttl tests will continue to loop through the script
until various conditions are met. This means that the annotation may be
run more than once. If this happens, kubectl will complain that the
annotation is already set and needs to be overwritten. This change adds
the overwrite flag to kubectl annotate commands.

* Replace most empty interface with "any"

The "any" type has been available since Go 1.18 and is easier to read.

* Add some badges to the README

Co-authored-by: Greg Nokes <[email protected]>

* Change release announcements from mailing list to discord

Co-authored-by: Greg Nokes <[email protected]>

* Initial scaffolding for standalone pgAdmin implementation

Includes basic implementation for CRD, installation, and a dummy
reconciliation loop. Reconciliation is feature-gated.

* add initial changes for PG16 compatibility and pgMonitor 4.10.0 bump

* Create and use a schema in kuttl tests to work around the change to CREATE permissions in public schema in pg15.

* Reconcile a pgAdmin StatefulSet, Pod PVC and ConfigMap

Add the reconciliation logic for the main initial elements for
pgAdmin. Includes initial configuration options for the StatefulSet
and example implementations for the ConfigMap, PVC and Status block

* Update PostgresCluster example to RWO Volumes

* pgAdmin use and discovery (#3739)

* Reconcile admin secret
* Working pgadmin with new image
* Cluster config reconcile and load
* add KUTTL test, fix item sorting

* PR feedback and adding comments (#3741)

* Remove admin as user-settable field (#3742)

* Normalize our kustomize files and get rid of deprecation warnings.

* Remove pgAdmin feature gate

PGO-558

* Update OLM installer bundles for pgAdmin API updates

This commit adds a new OLM example for the pgAdmin CRD and updates
the bundle description file.

* Update pgAdmin RBAC to match feature requirements

Adjust kubebuilder markers and regenerate RBAC for current
pgAdmin feature.

PGO-565

* add pgadmin config (#3747)

* remove unused kerberos env vars for now
* add init container to write script to read from mounted config file

Issue: PGO-547

* update service name (#3753)

* update kuttl admin username

* Remove pgAdmin Service

The option to allow a user to create a service through the pgAdmin spec
was pulled over from the v4.30 implentation. We are reevaluating how we
want to handle services and have decided to remove service creation
entirely for this round of pgAdmin changes.

* Update naming to use pgAdmin CR instance UID

Update all relevant names and impacted tests.

Issue: PGO-591

* add username to pgadmin secret (#3757)

* Grammar fix for README.md

Issue: PGO-550

* Defines registration and encumbrance PGO-217

* Adds registration to unencumber PGO-431

* When requested, load the "citus" library first

Issue: PGO-284
Issue: CrunchyData/postgres-operator#3194

* update versions

* updated gis version

* Add semantic versioning for registration.

* updated github workflow
Issue: PGO-353

* Update SSA expectations for recent versions of Kubernetes

The fix for https://issue.k8s.io/116861 was backported a few times.

* Relax an expectation in the Ticker tests

The test is racy and occasionally fails with "4 (int) != 3 (int)".

* update dependencies

* Set SecurityContext for standalone pgAdmin to avoid filesystem permission issues.

* Add DEFAULT_BINARY_PATH to standalone pgadmin startup script.

* Simplify exporter tests using MarshalMatches

Issue: PGO-635

* Move permanent flags into ExporterStartCommand

Issue: PGO-635

* Move exporter TLS logic into generate method

Issue: PGO-635

* Allow disabling of postgres_exporter defaults

postgres_exporter provides default metrics, settings, and collectors.
This change creates an annotation to allow disabling all of the
postgres_exporter defaults.

Co-authored-by: jmckulk <[email protected]>
Issue: PGO-635

* Update default upgrade test versions

* Update olm bundling (#3786)

Update olm bundling
Issue: PGO-430

* Update Versions

* Add Parallel Pod Management Policy to standalone pgadmin StatefulSet.

* Add Parallel Pod Management Policy and RollingUpdate UpdateStrategy to repohost StatefulSet. This allows the Pod to recover from a bad rollout.

* Add Parallel Pod Management Policy to postgrescluster-scoped pgadmin StatefulSet. This allows the Pod to recover from a bad rollout.

* Upgrade opm version

Issue: PGO-429

* Update OLM bundle description

Issue: PGO-728

* Remove Docs

* Reduces line length < 180 chars

The RH Certified bundle GitHub pipeline requires lines of
yaml to be less than 180 characters.

Issue: PGO-728

* updated makefile and linter for doc deletions

* Add Discord link to intro section of readme.

* Update restore configuration file behavior

Currently, the restore Pod will load both the pgBackRest
configuration objects from `spec.datasource` and `spec.backups`
sections. This commit updates that behavior so that only the
`datasource.pgbackrest.configuration` is loaded when performing
a cloud based restore.

Issue: PGO-260

* Make standalone pgAdmin controller the owner of objects that it creates.

* Update watchPods() behavior for Patroni role change

Add an additional check to the existing watchPods function to
queue an event when an instance Pod is first given the 'master'
role.

Issue: PGO-190

* Replace KUTTL 'empty-image-upgrade' with 'major-upgrade-missing-image'

Now that the bug fix is in place, move 'major-upgrade-missing-image'
back to the main testing folder in place of the subset test,
'empty-image-upgrade'

Issue: PGO-190

* Use the queries collected in queries dir

* Move standalone pgadmin test from e2e-other to e2e.

* Add standalone pgadmin related image to github action test.yaml.

* Update manager.yaml (#3816)

* Tweak restart logic for exporter (#3817)

* Tweak restart logic for exporter

This separates out the kill/restart logic for the exporter;
previously, if a file changed, we would kill/restart. This
led to some test flakes where the restart would happen too
quickly (hypothesis) before the port was free. This PR separates
the logic:

If the watched files change, kill the exporter;
If no exporter is running, start the exporter.

This also adds a check to the start_postgres_exporter func:
save the PID to a file only if that proc is running.

Issue: [PGO-420]

* Update PGO configurations to support TDE

This update builds on the exist custom configuration capabilities
of PGO to allow a Postgres cluster to be configured to support of
Transparent Data Encryption (TDE) in Postgres.

Issue: PGO-779

* Revise cluster-pause/start tests (#3821)

* Revise cluster-pause test

* Use files for legibility
* Add describe/log collectors to every assert
* Change cluster-pause change from replica to service to speed up tests

* pgadmin configuration keys can be alphanumeric see config.py docs

* Additional TDE configuration for Patroni pg_rewind

Provide a wrapper script to allow Patroni to invoke pg_rewind as
required for TDE and update configuration to enable pg_rewind in
Patroni for all versions > 10.

Issue: PGO-785

* pgbackrest-restore KUTTL test namespace flag usage updates

* Restores logo to README

* Add additional configuration files to the restore Job Pod

To support necessary TDE configurations, this commit adds any
configured items, such as ConfigMaps or Secrets, from the `config`
field to be mounted at `/etc/postgres` in the restore Job Pod's
'pgbackrest-restore' Container.

Issue: PGO-909

* Bridge via PGO MVP

Issue: [PGO-814]

Co-authored-by: Benjamin Blattberg <[email protected]>

* Update Copyright (#3818)

Issue: [PGO-812]

* Update pgMonitor version (#3837)

* Update pgMonitor version

Issue: [PGO-319]

* Change name of bridge crd and files
Issue: [PGO-915]

* Update internal/bridge/crunchybridgecluster/crunchybridgecluster_controller.go

Co-authored-by: Drew Sessler <[email protected]>

* Update internal/bridge/crunchybridgecluster/crunchybridgecluster_controller.go

Co-authored-by: Drew Sessler <[email protected]>

* Bump golang.org/x/crypto to quiet an SSH CVE

We use the PBKDF2 functions of this module for SCRAM verifiers.
We do not use any of its SSH functionality.

Issue: PGO-938
See: CVE-2023-48795
See: GHSA-45x7-px36-x8w8

* Bind pgAdmin to every IPv4 address by default

The upstream default is "127.0.0.1", the IPv4 loopback address, which
allows only local connections.

Issue: CrunchyData/postgres-operator#3809
Issue: PGO-842

* Pull the trademarks forward

* Add Crunchy Bridge Cluster adoption annotation logic.

* Add params to bridge client methods. Add team_id to ListClusters params.

* Cleanup EnvTest binaries more aggressively

I had trouble running "make clean" while switching between Linux and
macOS in the same working directory.

* Pin the "controller-gen" workflow to Go 1.21

The tool panics in Go 1.22, and we don't want to bump to a compatible
version just yet.

* Pin test coverage workflows to Go 1.21

The Go 1.22 "go test" command fails when using the "-coverpkg" flag.

See: https://go.dev/issue/65653

* Remove dependency licenses during the "clean" target

While I was switching between old branches, these directories were left
with Go code that breaks the "check" targets.

* updated API fields to look kube native, and updated some API fields
to names match the spec and status
Issue: [PGO-910]

* Updates to the Readme (#3855)

* Updates to the Readme

* Allow configuration of replica service through spec

This change adds the ReplicaService field to the spec that gives users
the same configuration options as other services (primary and
pgbouncer).

Notes:
- go and kuttl tests have been added to confirm that the spec type is
configured correctly
- check, check-envtest, and check-envtest-existing are passing
- check-kuttl tests are passing
- connected to LoadBalancer service using GKE loadbalancer

* Spelling (#3856)

* spelling: adopt