Podman Fails to Pull Image on macOS (EOF error)

[yanmxa]

Issue Description

When attempting to pull an image (e.g., busybox) using Podman on macOS, the operation fails with an EOF error while attempting to ping the Docker registry (registry-1.docker.io)

Steps to reproduce the issue

1. Run the following command:

   podman pull busybox --log-level debug

2. The process fails with the following error:

   ❯ podman pull busybox --log-level debug
    INFO[0000] podman filtering at log level debug
    DEBU[0000] Called pull.PersistentPreRunE(podman pull busybox --log-level debug)
    DEBU[0000] SSH Ident Key "/Users/yanmeng/.local/share/containers/podman/machine/machine" SHA256:2tNPfhe0u0iBoEuom9k/OYNf05MMFe4+02X0X4dqlSs ssh-ed25519
    DEBU[0000] DoRequest Method: GET URI: http://d/v5.2.5/libpod/_ping
    DEBU[0000] Loading registries configuration "/Users/yanmeng/.config/containers/registries.conf"
    DEBU[0000] No credentials matching registry.stage.redhat.io found in /Users/yanmeng/.config/containers/auth.json
    DEBU[0000] No credentials matching registry.stage.redhat.io found in /Users/yanmeng/.config/containers/auth.json
    DEBU[0000] Found an empty credential entry "registry.stage.redhat.io" in "/Users/yanmeng/.docker/config.json" (an unhandled credential helper marker?), moving on
    DEBU[0000] No credentials matching registry.stage.redhat.io found in /Users/yanmeng/.dockercfg
    DEBU[0000] No credentials for registry.stage.redhat.io found
    DEBU[0000] Found credentials for quay.io in credential helper containers-auth.json in file /Users/yanmeng/.config/containers/auth.json
    DEBU[0000] Found credentials for docker.io in credential helper containers-auth.json in file /Users/yanmeng/.config/containers/auth.json
    DEBU[0000] DoRequest Method: POST URI: http://d/v5.2.5/libpod/images/pull
    Resolved "busybox" as an alias (/etc/containers/registries.conf.d/000-shortnames.conf)
    Trying to pull docker.io/library/busybox:latest...
    Error: initializing source docker://busybox:latest: pinging container registry registry-1.docker.io: Get "https://registry-1.docker.io/v2/": EOF
    DEBU[0006] Shutting down engines

Describe the results you received

The pull operation fails with an EOF error.

Describe the results you expected

Podman should successfully pull the image from the Docker registry.

podman info output

❯ podman info
host:
  arch: arm64
  buildahVersion: 1.37.5
  cgroupControllers:
  - cpuset
  - cpu
  - io
  - memory
  - pids
  - rdma
  - misc
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.1.12-2.fc40.aarch64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.12, commit: '
  cpuUtilization:
    idlePercent: 99.93
    systemPercent: 0.04
    userPercent: 0.03
  cpus: 5
  databaseBackend: sqlite
  distribution:
    distribution: fedora
    variant: coreos
    version: "40"
  eventLogger: journald
  freeLocks: 2048
  hostname: localhost.localdomain
  idMappings:
    gidmap: null
    uidmap: null
  kernel: 6.11.3-200.fc40.aarch64
  linkmode: dynamic
  logDriver: journald
  memFree: 1558630400
  memTotal: 2042908672
  networkBackend: netavark
  networkBackendInfo:
    backend: netavark
    dns:
      package: aardvark-dns-1.12.2-2.fc40.aarch64
      path: /usr/libexec/podman/aardvark-dns
      version: aardvark-dns 1.12.2
    package: netavark-1.12.2-1.fc40.aarch64
    path: /usr/libexec/podman/netavark
    version: netavark 1.12.2
  ociRuntime:
    name: crun
    package: crun-1.17-1.fc40.aarch64
    path: /usr/bin/crun
    version: |-
      crun version 1.17
      commit: 000fa0d4eeed8938301f3bcf8206405315bc1017
      rundir: /run/crun
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +LIBKRUN +WASM:wasmedge +YAJL
  os: linux
  pasta:
    executable: /usr/bin/pasta
    package: passt-0^20240906.g6b38f07-1.fc40.aarch64
    version: |
      pasta 0^20240906.g6b38f07-1.fc40.aarch64-pasta
      Copyright Red Hat
      GNU General Public License, version 2 or later
        <https://www.gnu.org/licenses/old-licenses/gpl-2.0.html>
      This is free software: you are free to change and redistribute it.
      There is NO WARRANTY, to the extent permitted by law.
  remoteSocket:
    exists: true
    path: /run/podman/podman.sock
  rootlessNetworkCmd: pasta
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: false
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: true
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.2.2-2.fc40.aarch64
    version: |-
      slirp4netns version 1.2.2
      commit: 0ee2d87523e906518d34a6b423271e4826f71faf
      libslirp: 4.7.0
      SLIRP_CONFIG_VERSION_MAX: 4
      libseccomp: 2.5.5
  swapFree: 0
  swapTotal: 0
  uptime: 68h 29m 5.00s (Approximately 2.83 days)
  variant: v8
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries:
  search:
  - docker.io
store:
  configFile: /usr/share/containers/storage.conf
  containerStore:
    number: 0
    paused: 0
    running: 0
    stopped: 0
  graphDriverName: overlay
  graphOptions:
    overlay.imagestore: /usr/lib/containers/storage
    overlay.mountopt: nodev,metacopy=on
  graphRoot: /var/lib/containers/storage
  graphRootAllocated: 106769133568
  graphRootUsed: 4783685632
  graphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Supports shifting: "true"
    Supports volatile: "true"
    Using metacopy: "true"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 0
  runRoot: /run/containers/storage
  transientStore: false
  volumePath: /var/lib/containers/storage/volumes
version:
  APIVersion: 5.2.5
  Built: 1729209600
  BuiltTime: Fri Oct 18 08:00:00 2024
  GitCommit: ""
  GoVersion: go1.22.7
  Os: linux
  OsArch: linux/arm64
  Version: 5.2.5

Podman in a container

No

Privileged Or Rootless

None

Upstream Latest Release

Yes

Additional environment details

1. MacOS version

sw_vers

ProductName:            macOS
ProductVersion:         15.1.1
BuildVersion:           24B91

2. Podman machin info

❯ podman machine info
host:
    arch: arm64
    currentmachine: podman-machine-default
    defaultmachine: ""
    eventsdir: /var/folders/4t/fhhxz9q56vd0mk9t7ytjpvbm0000gn/T/storage-run-501/podman
    machineconfigdir: /Users/yanmeng/.config/containers/podman/machine/applehv
    machineimagedir: /Users/yanmeng/.local/share/containers/podman/machine/applehv
    machinestate: Running
    numberofmachines: 1
    os: darwin
    vmtype: applehv
version:
    apiversion: 5.2.5
    version: 5.2.5
    goversion: go1.23.2
    gitcommit: 10c5aa720d59480bc7edad347c1f5d5b75d4424f
    builttime: Thu Oct 24 01:53:09 2024
    built: 1729705989
    osarch: darwin/arm64
    os: darwin

3. Podman connection

❯ podman system connection list
Name                         URI                                                         Identity                                                       Default     ReadWrite
podman-machine-default       ssh://[email protected]:64043/run/user/501/podman/podman.sock  /Users/yanmeng/.local/share/containers/podman/machine/machine  false       true
podman-machine-default-root  ssh://[email protected]:64043/run/podman/podman.sock           /Users/yanmeng/.local/share/containers/podman/machine/machine  true        true

Additional information

Additional information like issue happens only occasionally or issue happens with a particular architecture or on a particular setting

[vrothberg]

Such EOFs tend to happen when going through a proxy. Are you behind (coorporate) proxies in your environment?

[yanmxa]

Yes, I need to configure the corporate proxies to connect to the registry(network issue), and It works with Docker. Is there a way to make it work with Podman?

[vrothberg]

Yes, I need to configure the corporate proxies to connect to the registry(network issue), and It works with Docker. Is there a way to make it work with Podman?

Can you access the logs of the proxy? I have seen cases where proxies would block any other client than docker or behave different.

[yanmxa]

Yes, I need to configure the corporate proxies to connect to the registry(network issue), and It works with Docker. Is there a way to make it work with Podman?

Can you access the logs of the proxy? I have seen cases where proxies would block any other client than docker or behave different.

I use Viscosity to proxy my network, But there aren't echo any logs when I try to use the podman pull image