[BUG] Podman REST API return a list of containers with partial/missing "Mounts" attribute values

[D3vil0p3r]

Issue Description

Podman REST API container list returns a list of containers with partial/missing Mounts attribute values.

Steps to reproduce the issue

To simplify the reproduction, let's use Podman py:

import podman
from podman import PodmanClient

client = podman.from_env()
client.ping()
docker_create_function = client.containers.create
docker_args = {"image": "athenaos/base:latest",
               "name": "exegol-default",
               "mounts": [
                    {'target': '/opt/resources', 'source': '/home/athena/Exegol/exegol-resources', 'type': 'bind'},
                    {'target': '/workspaces', 'source': '/home/athena/.exegol/workspaces', 'type': 'bind'}
                ]}

container = docker_create_function(**docker_args)
print(container.attrs.get("Mounts", []))

print("Now use client.containers.list:")
docker_containers = client.containers.list(all=True, filters={"name": "exegol-"})

for container in docker_containers:
    print(container.attrs.get("Mounts", []))

Run this script. It will create a exegol-default container with different mounts elements.

One note: when the container is created, the container attribute mounts has the right values. When the container is listed (so when the /containers/json API is invoked), no.

Then, enable API service:

podman system service tcp:localhost:8080 --time=0 &

and run:

curl -k http://localhost:8080/v4.0.0/libpod/containers/json?all=true | jq

It will return only target info of mounts, instead of other information:

...
    "Labels": null,
    "Mounts": [
      "/opt/resources",
      "/workspaces"
    ],
    "Names": [
      "exegol-default"
    ],
...

Describe the results you received

...
    "Labels": null,
    "Mounts": [
      "/opt/resources",
      "/workspaces"
    ],
    "Names": [
      "exegol-default"
    ],
...

Describe the results you expected

I expect that Podman API return all the elements of Mounts, so not only target but also source, type and so on. I expect a result as occurs by docker API when running:

curl --cacert ca.pem --cert client-cert.pem --key client-key.pem https://localhost:1234/containers/json?all=true | jq

By docker API the output is correct:

...
    "Mounts": [
      {
        "Type": "bind",
        "Source": "/home/athena/Exegol/exegol-resources",
        "Destination": "/opt/resources",
        "Mode": "",
        "RW": true,
        "Propagation": "rprivate"
      },
      {
        "Type": "bind",
        "Source": "/home/athena",
        "Destination": "/workspaces",
        "Mode": "",
        "RW": true,
        "Propagation": "rprivate"
      }
    ]
...

podman info output

host:
  arch: amd64
  buildahVersion: 1.38.0
  cgroupControllers:
  - cpu
  - memory
  - pids
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-1:2.1.12-1
    path: /usr/bin/conmon
    version: 'conmon version 2.1.12, commit: e8896631295ccb0bfdda4284f1751be19b483264'
  cpuUtilization:
    idlePercent: 82.38
    systemPercent: 6.71
    userPercent: 10.91
  cpus: 4
  databaseBackend: sqlite
  distribution:
    distribution: athena
    version: Rolling release
  eventLogger: journald
  freeLocks: 2047
  hostname: athenaos
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
  kernel: 6.6.65-1-lts
  linkmode: dynamic
  logDriver: journald
  memFree: 264495104
  memTotal: 3966132224
  networkBackend: netavark
  networkBackendInfo:
    backend: netavark
    dns:
      package: aardvark-dns-1.13.1-1
      path: /usr/lib/podman/aardvark-dns
      version: aardvark-dns 1.13.1
    package: netavark-1.13.1-1
    path: /usr/lib/podman/netavark
    version: netavark 1.13.1
  ociRuntime:
    name: runc
    package: runc-1.2.3-1
    path: /usr/bin/runc
    version: |-
      runc version 1.2.3
      spec: 1.2.0
      go: go1.23.4
      libseccomp: 2.5.5
  os: linux
  pasta:
    executable: /usr/bin/pasta
    package: passt-2024_11_27.c0fbc7e-1
    version: |
      pasta 2024_11_27.c0fbc7e
      Copyright Red Hat
      GNU General Public License, version 2 or later
        <https://www.gnu.org/licenses/old-licenses/gpl-2.0.html>
      This is free software: you are free to change and redistribute it.
      There is NO WARRANTY, to the extent permitted by law.
  remoteSocket:
    exists: true
    path: /run/user/1000/podman/podman.sock
  rootlessNetworkCmd: pasta
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /etc/containers/seccomp.json
    selinuxEnabled: false
  serviceIsRemote: false
  slirp4netns:
    executable: ""
    package: ""
    version: ""
  swapFree: 6514012160
  swapTotal: 8053059584
  uptime: 7h 45m 8.00s (Approximately 0.29 days)
  variant: ""
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries: {}
store:
  configFile: /home/athena/.config/containers/storage.conf
  containerStore:
    number: 1
    paused: 0
    running: 0
    stopped: 1
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /home/athena/.local/share/containers/storage
  graphRootAllocated: 631544741888
  graphRootUsed: 30339878912
  graphStatus:
    Backing Filesystem: btrfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Supports shifting: "false"
    Supports volatile: "true"
    Using metacopy: "false"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 3
  runRoot: /run/user/1000/containers
  transientStore: false
  volumePath: /home/athena/.local/share/containers/storage/volumes
version:
  APIVersion: 5.3.1
  Built: 1732225906
  BuiltTime: Thu Nov 21 22:51:46 2024
  GitCommit: 4cbdfde5d862dcdbe450c0f1d76ad75360f67a3c
  GoVersion: go1.23.3
  Os: linux
  OsArch: linux/amd64
  Version: 5.3.1

Podman in a container

No

Privileged Or Rootless

Rootless

Upstream Latest Release

Yes

Additional environment details

Arch Linux

Additional information

Initial issue reported to containers/podman-py#488

[rhatdan]

This is calling into the podman ps api.

$ podman ps -a
CONTAINER ID  IMAGE                            COMMAND     CREATED        STATUS      PORTS       NAMES
c42e7d2b4e7a  docker.io/library/alpine:latest  /bin/sh     4 minutes ago  Created                 exegol-default
$ podman ps -a --format '{{ .Mounts }}'
[/opt/resources /workspaces]

Not sure if that is correct, but this is the way it is coded.

[rhatdan]

As opposed to the podman inspect api.

podman inspect exegol-default   --format '{{ .Mounts }}'
[{bind  /tmp/foobar /opt/resources   [nosuid nodev rbind] true rprivate } {bind  /tmp/workspaces /workspaces   [nosuid nodev rbind] true rprivate }]

[D3vil0p3r]

I guess that, unlike podman inspect, podman ps invokes the same problematic /containers/json API. Do you know what is the source file in the repository that implements /containers/json API?

[rhatdan]

Sorry, perhaps @Luap99 or @mheon can help.

[D3vil0p3r]

I just tested docker API by the container list call and docker API works well:

curl --cacert ca.pem --cert client-cert.pem --key client-key.pem https://localhost:1234/containers/json?all=true | jq

Output:

...
    "Mounts": [
      {
        "Type": "bind",
        "Source": "/home/athena/Exegol/exegol-resources",
        "Destination": "/opt/resources",
        "Mode": "",
        "RW": true,
        "Propagation": "rprivate"
      },
      {
        "Type": "bind",
        "Source": "/home/athena",
        "Destination": "/workspaces",
        "Mode": "",
        "RW": true,
        "Propagation": "rprivate"
      }
    ]
...

I think podman container listing API should be fixed to show the same consistent information.

[rhatdan]

I tend to agree with you.

It looks like the Docker API does the correct thing and looks up the mounts on the server side.

[mheon]

Should be a simple enough fix, but most of us are on PTO so I doubt it'll be touched until the new year.

[D3vil0p3r]

Should be a simple enough fix, but most of us are on PTO so I doubt it'll be touched until the new year.

In terms of time effort, in your opinion how much time is it needed to work on the code to fix it?

[mheon]

Probably a few hours? From what I can see, the results from podman inspect at the command line are correct and have a fully populated mounts struct, and the REST API uses the same underlying code, so the problem is likely somewhere in the handler code and is stripping the full Mount structs

[rhatdan]

The containers.list function is calling

docker_containers = client.containers.list(all=True, filters={"name": "exegol-"})

Which is the same thing as podman --remote ps --all would call. This looks like in Docker and Podman the ps command only outputs the Destination of Mounts, which is probably why the API only returns it in the libpod API. The Docker API for the same call returns the entire mount object. I think it would be smarter to merge the two Calls together into a signle function in code. Since the only difference I easily see is around additional filters that libpod provides. I don't really know what the compat API and the libpod API required different implementations.