override replace on the systemd rpm does not replace /usr/bin/systemctl

[prydom]

Describe the bug

When using rpm-ostree override replace to upgrade packages in a container overlay, upgrading systemd does not replace the /usr/bin/systemctl binary.

Reproduction steps

1. podman run -it -v ~/rpms:/tmp/rpms:ro,z "$BASE_IMAGE:$BASE_TAG" /bin/bash --login
2. rpm-ostree override replace /tmp/rpms/systemd-255.5-1.fc41.x86_64.rpm (upgrading from systemd-255.4)
3. rpm -V --nomtime systemd

Expected behavior

/usr/bin/systemctl should match the version of the replaced RPM. The RPM should verify as expected below.

# rpm -V --nomtime systemd
.M.......  c /etc/machine-id
missing     /var/lib/systemd
missing     /var/lib/systemd/catalog

Actual behavior

/usr/bin/systemctl is the version from the base image. If libsystemd was also upgraded then /usr/bin/systemctl will fail to start due to missing shared libraries.

# rpm -V --nomtime systemd
.M.......  c /etc/machine-id
S.5......    /usr/bin/systemctl
missing     /var/lib/systemd
missing     /var/lib/systemd/catalog

System details

# rpm-ostree --version
rpm-ostree:
 Version: '2024.5'
 Git: f9c588bf86cc2ff00034bd7ea846392351aa5e47
 Features:
  - rust
  - compose
  - container
  - fedora-integration
# cat /etc/os-release
NAME="Fedora Linux"
VERSION="Rawhide.20240422.n.0 (Kinoite Prerelease)"
ID=fedora
VERSION_ID=41
VERSION_CODENAME=""
PLATFORM_ID="platform:f41"
PRETTY_NAME="Fedora Linux Rawhide.20240422.n.0 (Kinoite Prerelease)"
ANSI_COLOR="0;38;2;60;110;180"
LOGO=fedora-logo-icon
CPE_NAME="cpe:/o:fedoraproject:fedora:41"
DEFAULT_HOSTNAME="fedora"
HOME_URL="https://kinoite.fedoraproject.org"
DOCUMENTATION_URL="https://docs.fedoraproject.org/en-US/fedora-kinoite/"
SUPPORT_URL="https://ask.fedoraproject.org/"
BUG_REPORT_URL="https://pagure.io/fedora-kde/SIG/issues"
REDHAT_BUGZILLA_PRODUCT="Fedora"
REDHAT_BUGZILLA_PRODUCT_VERSION=rawhide
REDHAT_SUPPORT_PRODUCT="Fedora"
REDHAT_SUPPORT_PRODUCT_VERSION=rawhide
SUPPORT_END=2025-05-13
VARIANT="Kinoite"
VARIANT_ID=kinoite
OSTREE_VERSION='Rawhide.20240422.n.0'

Additional information

This can be worked around by rebuilding the base image from a treefile. My base image uses Fedora Rawhide (quay.io/fedora/fedora-kinoite). In order to reduce the number of layers changed from day-to-day I use override replace in a container overlay to apply package updates using this script and Containerfile.

I rebase the entire container at least once a week to reduce but not eliminate hysteresis.

cliwrap is enabled.

[root@a7f254fd6474 /]# ls -l /usr/libexec/rpm-ostree/wrapped/
total 188
-rwxr-xr-x. 2 root root 95282 Dec 31  1969 dracut
-rwxr-xr-x. 2 root root 66000 Dec 31  1969 kernel-install
-rwxr-xr-x. 2 root root 20624 Dec 31  1969 rpm

[prydom]

This seems likely to be caused by the wrapping of systemctl and the various user modification tools at

rpm-ostree/rust/src/core.rs

Lines 143 to 151 in e0cf3f7

	impl FilesystemScriptPrep {
	/// Filesystem paths that we rename out of the way if present
	const OPTIONAL_PATHS: &'static [&'static str] = &[SSS_CACHE_PATH];
	const REPLACE_OPTIONAL_PATHS: &'static [(&'static str, &'static [u8])] = &[
	(GROUPADD_PATH, GROUPADD_WRAPPER),
	(SYSTEMCTL_PATH, SYSTEMCTL_WRAPPER),
	(USERADD_PATH, USERADD_WRAPPER),
	(USERMOD_PATH, USERMOD_WRAPPER),
	];

and so this probably also impacts rpm-ostree override replace of the shadow-utils package.