pulling image with zstd:chunked compression is returning manifest too big error

[miabbott]

Issue Description

I regularly build a "pet" container via Github Actions and push it to quay.io as a zstd:chunked compressed image. See the GHA definition here - https://github.com/miabbott/pet/blob/main/.github/workflows/build.yml

When I tried pulling the latest image today, I got the following error:

$ podman pull quay.io/miabbott/pet                                                                                                                                        
Trying to pull quay.io/miabbott/pet:latest...                                                                                                                                                  
Getting image source signatures                                                                                                                                                                
Copying blob 4ff7216921d7 done  34.5KiB / 34.5KiB (skipped: 240.0b = 0.68%)                                                                                                                    
Copying blob 1753f747eed5 done  339.1MiB / 339.1MiB (skipped: 6.6KiB = 0.00%)                                                                                                                  
Error: partial pull of blob sha256:133f7c600c0bfb06bb109c35d10339ee36e223727510624b0c538d6dd5aba990: read zstd:chunked manifest: manifest too big 

I suspected this might be a problem pushing the image with an older version of podman that is used in the Ubuntu runner for GHA, but I was able to reproduce the same error when building/pushing on Fedora 40 with podman 5.3.1

Steps to reproduce the issue

Steps to reproduce the issue

1. Build image from https://github.com/miabbott/pet/tree/main
2. Push as zstd:chunked compressed image to quay.io
3. Pull the compressed image from quay.io

Describe the results you received

$ podman pull quay.io/miabbott/pet                                                                                                                                        
Trying to pull quay.io/miabbott/pet:latest...                                                                                                                                                  
Getting image source signatures                                                                                                                                                                
Copying blob 4ff7216921d7 done  34.5KiB / 34.5KiB (skipped: 240.0b = 0.68%)                                                                                                                    
Copying blob 1753f747eed5 done  339.1MiB / 339.1MiB (skipped: 6.6KiB = 0.00%)                                                                                                                  
Error: partial pull of blob sha256:133f7c600c0bfb06bb109c35d10339ee36e223727510624b0c538d6dd5aba990: read zstd:chunked manifest: manifest too big 

Describe the results you expected

Successful retrieval of the image.

podman info output

From the client pulling the image:

$ podman info
host:                   
  arch: amd64           
  buildahVersion: 1.38.0
  cgroupControllers:                
  - cpu                                     
  - io         
  - memory                     
  - pids                                              
  cgroupManager: systemd
  cgroupVersion: v2              
  conmon:              
    package: conmon-2.1.12-2.fc40.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.12, commit: '  
  cpuUtilization:
    idlePercent: 95.34
    systemPercent: 0.96
    userPercent: 3.7
  cpus: 8   
  databaseBackend: sqlite
  distribution:
    distribution: fedora
    variant: silverblue
    version: "40"
  eventLogger: journald
  freeLocks: 2048
  hostname: meatwad
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000           
      size: 1                 
    - container_id: 1
      host_id: 524288
      size: 65536                                               
    uidmap:      
    - container_id: 0
      host_id: 1000
      size: 1 
    - container_id: 1
      host_id: 524288     
      size: 65536 
  kernel: 6.12.4-100.fc40.x86_64                               
  linkmode: dynamic               
  logDriver: journald        
  memFree: 11606368256
  memTotal: 33336758272    
  networkBackend: netavark     
  networkBackendInfo:      
    backend: netavark         
    dns:                     
      package: aardvark-dns-1.13.1-1.fc40.x86_64
      path: /usr/libexec/podman/aardvark-dns
      version: aardvark-dns 1.13.1
    package: netavark-1.13.0-1.fc40.x86_64
    path: /usr/libexec/podman/netavark
    version: netavark 1.13.0
  ociRuntime:                                                           
    name: crun
    package: crun-1.18.2-1.fc40.x86_64
    path: /usr/bin/crun
    version: |-                      
      crun version 1.18.2
      commit: 00ab38af875ddd0d1a8226addda52e1de18339b5
      rundir: /run/user/1000/crun
      spec: 1.0.0    
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +LIBKRUN +WASM:wasmedge +YAJL
  os: linux
  pasta:                                                         
    executable: /usr/bin/pasta
    package: passt-0^20241127.gc0fbc7e-1.fc40.x86_64
    version: |
      pasta 0^20241127.gc0fbc7e-1.fc40.x86_64
      Copyright Red Hat
      GNU General Public License, version 2 or later
        <https://www.gnu.org/licenses/old-licenses/gpl-2.0.html>
      This is free software: you are free to change and redistribute it.
      There is NO WARRANTY, to the extent permitted by law.
  remoteSocket:
    exists: true
    path: /run/user/1000/podman/podman.sock
  rootlessNetworkCmd: pasta
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.3.1-1.fc40.x86_64
    version: |-
      slirp4netns version 1.3.1
      commit: e5e368c4f5db6ae75c2fce786e31eef9da6bf236
      libslirp: 4.7.0
      SLIRP_CONFIG_VERSION_MAX: 4
      libseccomp: 2.5.5
  swapFree: 4707753984
  swapTotal: 8589930496
  uptime: 23h 0m 1.00s (Approximately 0.96 days)
  variant: ""
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries:
  search:
  - registry.fedoraproject.org
  - registry.access.redhat.com
  - docker.io
store:
  configFile: /var/home/miabbott/.config/containers/storage.conf
  containerStore:
    number: 0
    paused: 0
    running: 0
    stopped: 0
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /var/home/miabbott/.local/share/containers/storage
  graphRootAllocated: 433944612864
  graphRootUsed: 355216216064
  graphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Supports shifting: "false"
    Supports volatile: "true"
    Using metacopy: "false"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 72
  runRoot: /run/user/1000/containers
  transientStore: false
  volumePath: /var/home/miabbott/.local/share/containers/storage/volumes
version:
  APIVersion: 5.3.1
  Built: 1732147200
  BuiltTime: Wed Nov 20 19:00:00 2024
  GitCommit: ""
  GoVersion: go1.22.7
  Os: linux
  OsArch: linux/amd64
  Version: 5.3.1

Podman in a container

No

Privileged Or Rootless

Rootless

Upstream Latest Release

Yes