debugging do-not-track test failures

[alexcb]

No description provided.

[alexcb]

I got a bind error, and the netstat didn't show anything listening on 443:

./t/do-not-track+test-do-not-track *failed* | Active Internet connections (only servers)
./t/do-not-track+test-do-not-track *failed* | Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
./t/do-not-track+test-do-not-track *failed* | waiting for stub-server to start
./t/do-not-track+test-do-not-track *failed* | stub-server ready

[alexcb]

this version did a netstat before and after running the stub server, but it doesn't show any services listening on 443:

./t/do-not-track+test-do-not-track *failed* | Repeating the failure error...
./t/do-not-track+test-do-not-track *failed* | DOCKERHUB_MIRROR=registry-1.docker.io.mirror.corp.earthly.dev DOCKERHUB_MIRROR_AUTH=true GLOBAL_WAIT_END=
./t/do-not-track+test-do-not-track *failed* | --> RUN --privileged /bin/sh /tmp/earthly-script
./t/do-not-track+test-do-not-track *failed* | running earthly with EARTHLY_EXEC_CMD=/tmp/test-earthly-script
./t/do-not-track+test-do-not-track *failed* | + export 'EARTHLY_VERBOSE=1'
./t/do-not-track+test-do-not-track *failed* | + true
./t/do-not-track+test-do-not-track *failed* | + export 'EARTHLY_TMP_DIR=/tmp/earthly-tmpfs'
./t/do-not-track+test-do-not-track *failed* | + '[' -n  ]
./t/do-not-track+test-do-not-track *failed* | + '[' -n /tmp/test-earthly-script ]
./t/do-not-track+test-do-not-track *failed* | + '[' /tmp/test-earthly-script '=' /tmp/earthly-script ]
./t/do-not-track+test-do-not-track *failed* | + '[' +all '!=' +all ]
./t/do-not-track+test-do-not-track *failed* | + export 'EARTHLY_EXEC_CMD=/tmp/test-earthly-script'
./t/do-not-track+test-do-not-track *failed* | + echo running earthly with 'EARTHLY_EXEC_CMD=/tmp/test-earthly-script'
./t/do-not-track+test-do-not-track *failed* | + set +e
./t/do-not-track+test-do-not-track *failed* | + tee earthly.output
./t/do-not-track+test-do-not-track *failed* | + /bin/sh /usr/bin/earthly-entrypoint.sh
./t/do-not-track+test-do-not-track *failed* | detected cgroups v2; earthly-entrypoint.sh running under pid=14 with controllers "cpuset cpu io memory hugetlb pids rdma misc" in group 0::/
./t/do-not-track+test-do-not-track *failed* | loading config values from "/etc/.earthly/config.yml"
./t/do-not-track+test-do-not-track *failed* |             frontend | Running command: docker info
./t/do-not-track+test-do-not-track *failed* |             frontend | Running command: podman info --format={{.Host.Security.Rootless}}
./t/do-not-track+test-do-not-track *failed* |             frontend | auto frontend initialization failed due to failed to autodetect a supported frontend: 2 errors occurred:
./t/do-not-track+test-do-not-track *failed* |             frontend | 	* docker-shell frontend failed to initialize: command failed: docker info: exit status 1: Client:
./t/do-not-track+test-do-not-track *failed* |             frontend |  Context:    default
./t/do-not-track+test-do-not-track *failed* |             frontend |  Debug Mode: false

./t/do-not-track+test-do-not-track *failed* |             frontend | Server:
./t/do-not-track+test-do-not-track *failed* |             frontend | ERROR: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
./t/do-not-track+test-do-not-track *failed* |             frontend | errors pretty printing info: exit status 1
./t/do-not-track+test-do-not-track *failed* |             frontend | 	* podman-shell frontend failed to initialize: command failed: podman info --format={{.Host.Security.Rootless}}: exec: "podman": executable file not found in $PATH: : exec: "podman": executable file not found in $PATH

./t/do-not-track+test-do-not-track *failed* |            bootstrap | Bootstrapping successful.
./t/do-not-track+test-do-not-track *failed* | == netstat before ==
./t/do-not-track+test-do-not-track *failed* | Active Internet connections (only servers)
./t/do-not-track+test-do-not-track *failed* | Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
./t/do-not-track+test-do-not-track *failed* | == done ==
./t/do-not-track+test-do-not-track *failed* | waiting for stub-server to start
./t/do-not-track+test-do-not-track *failed* | stub-server ready
./t/do-not-track+test-do-not-track *failed* | == netstat after stub server ==
./t/do-not-track+test-do-not-track *failed* | Active Internet connections (only servers)
./t/do-not-track+test-do-not-track *failed* | Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
./t/do-not-track+test-do-not-track *failed* | tcp        0      0 :::8372                 :::*                    LISTEN      90/buildkitd
./t/do-not-track+test-do-not-track *failed* | tcp        0      0 :::8371                 :::*                    LISTEN      90/buildkitd
./t/do-not-track+test-do-not-track *failed* | == done ==
./t/do-not-track+test-do-not-track *failed* | loading config values from "/etc/.earthly/config.yml"
./t/do-not-track+test-do-not-track *failed* |             frontend | Running command: docker info
./t/do-not-track+test-do-not-track *failed* |             frontend | Running command: podman info --format={{.Host.Security.Rootless}}
./t/do-not-track+test-do-not-track *failed* |             frontend | Local registry host is specified while using remote buildkit. Local registry will not be used.
./t/do-not-track+test-do-not-track *failed* |             frontend | auto frontend initialization failed due to failed to autodetect a supported frontend: 2 errors occurred:
./t/do-not-track+test-do-not-track *failed* |             frontend | 	* docker-shell frontend failed to initialize: command failed: docker info: exit status 1: Client:
./t/do-not-track+test-do-not-track *failed* |             frontend |  Context:    default
./t/do-not-track+test-do-not-track *failed* |             frontend |  Debug Mode: false

./t/do-not-track+test-do-not-track *failed* |             frontend | Server:
./t/do-not-track+test-do-not-track *failed* |             frontend | ERROR: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
./t/do-not-track+test-do-not-track *failed* |             frontend | errors pretty printing info: exit status 1
./t/do-not-track+test-do-not-track *failed* |             frontend | 	* podman-shell frontend failed to initialize: command failed: podman info --format={{.Host.Security.Rootless}}: exec: "podman": executable file not found in $PATH: : exec: "podman": executable file not found in $PATH

./t/do-not-track+test-do-not-track *failed* | loading config values from "/etc/.earthly/config.yml"
./t/do-not-track+test-do-not-track *failed* |             frontend | Running command: docker info
./t/do-not-track+test-do-not-track *failed* |             frontend | Running command: podman info --format={{.Host.Security.Rootless}}
./t/do-not-track+test-do-not-track *failed* |             frontend | Local registry host is specified while using remote buildkit. Local registry will not be used.
./t/do-not-track+test-do-not-track *failed* |             frontend | auto frontend initialization failed due to failed to autodetect a supported frontend: 2 errors occurred:
./t/do-not-track+test-do-not-track *failed* |             frontend | 	* docker-shell frontend failed to initialize: command failed: docker info: exit status 1: Client:
./t/do-not-track+test-do-not-track *failed* |             frontend |  Context:    default
./t/do-not-track+test-do-not-track *failed* |             frontend |  Debug Mode: false

./t/do-not-track+test-do-not-track *failed* |             frontend | Server:
./t/do-not-track+test-do-not-track *failed* |             frontend | ERROR: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
./t/do-not-track+test-do-not-track *failed* |             frontend | errors pretty printing info: exit status 1
./t/do-not-track+test-do-not-track *failed* |             frontend | 	* podman-shell frontend failed to initialize: command failed: podman info --format={{.Host.Security.Rootless}}: exec: "podman": executable file not found in $PATH: : exec: "podman": executable file not found in $PATH

./t/do-not-track+test-do-not-track *failed* |  Init 🚀
./t/do-not-track+test-do-not-track *failed* | ————————————————————————————————————————————————————————————————————————————————

./t/do-not-track+test-do-not-track *failed* |            buildkitd | Connecting to tcp://buildkitsandbox:8372...
./t/do-not-track+test-do-not-track *failed* |            buildkitd | ...Done
./t/do-not-track+test-do-not-track *failed* |            buildkitd | Version github.com/earthly/buildkit dev-HEAD 086f60eecf5a2261bbd53299614f88b3def12746
./t/do-not-track+test-do-not-track *failed* |            buildkitd | Platforms: linux/amd64 (native) linux/amd64/v2 linux/amd64/v3 linux/arm64 linux/riscv64 linux/ppc64le linux/s390x linux/386 linux/mips64le linux/mips64 linux/arm/v7 linux/arm/v6
./t/do-not-track+test-do-not-track *failed* |            buildkitd | Utilization: 0 other builds, 0/20 op load
./t/do-not-track+test-do-not-track *failed* |            buildkitd | GC stats: 0 B cache, avg GC duration 0s, all-time GC duration 0s, last GC duration 0s, last cleared 0 B

./t/do-not-track+test-do-not-track *failed* |  Build 🔧
./t/do-not-track+test-do-not-track *failed* | ————————————————————————————————————————————————————————————————————————————————

./t/do-not-track+test-do-not-track *failed* |       registry-proxy | Registry proxy disabled via --disable-remote-registry-proxy
./t/do-not-track+test-do-not-track *failed* | Logbus: setting organization "" and project "" at 2024-01-26T17:44:51.035966514Z
./t/do-not-track+test-do-not-track *failed* | earthfile2llb building +true with OverridingVars=map[]
./t/do-not-track+test-do-not-track *failed* | earthfile2llb building +base with OverridingVars=map[]
./t/do-not-track+test-do-not-track *failed* |          alpine:3.18 | --> Load metadata alpine:3.18 linux/amd64
./t/do-not-track+test-do-not-track *failed* | using *authprovider.authProvider for registry-1.docker.io.mirror.corp.earthly.dev
./t/do-not-track+test-do-not-track *failed* |                +true | --> FROM alpine:3.18
./t/do-not-track+test-do-not-track *failed* |                +true | [----------] 100% FROM alpine:3.18
./t/do-not-track+test-do-not-track *failed* |                +true | [          ]   0% FROM alpine:3.18
./t/do-not-track+test-do-not-track *failed* |                +true | [----------] 100% FROM alpine:3.18
./t/do-not-track+test-do-not-track *failed* |                +true | --> RUN true
./t/do-not-track+test-do-not-track *failed* |               output | --> exporting outputs

./t/do-not-track+test-do-not-track *failed* |  Push Summary ⏫ (disabled)
./t/do-not-track+test-do-not-track *failed* | ————————————————————————————————————————————————————————————————————————————————

./t/do-not-track+test-do-not-track *failed* | To enable pushing use earthly --push

./t/do-not-track+test-do-not-track *failed* |  Local Output Summary 🎁
./t/do-not-track+test-do-not-track *failed* | ————————————————————————————————————————————————————————————————————————————————



./t/do-not-track+test-do-not-track *failed* | ========================== 🌍 Earthly Build  ✅ SUCCESS ==========================

./t/do-not-track+test-do-not-track *failed* | 🛰️ Reuse cache between CI runs with Earthly Satellites! 2-20X faster than without cache. Generous free tier https://cloud.earthly.dev/

./t/do-not-track+test-do-not-track *failed* | == shutting down api-earthly-stub-server ==
./t/do-not-track+test-do-not-track *failed* | ERROR: stub server malfunction; failed to connect to stub server: {e}

./t/do-not-track+test-do-not-track *failed* | == api-earthly-stub-server log ==
./t/do-not-track+test-do-not-track *failed* | creating socket
./t/do-not-track+test-do-not-track *failed* | binding socket to 127.0.0.1:443
./t/do-not-track+test-do-not-track *failed* | unexpected exception [Errno 98] Address in use
./t/do-not-track+test-do-not-track *failed* | Traceback (most recent call last):
./t/do-not-track+test-do-not-track *failed* |   File "/bin/api-earthly-stub-server", line 92, in <module>
./t/do-not-track+test-do-not-track *failed* |     s.bind((host, port))
./t/do-not-track+test-do-not-track *failed* | OSError: [Errno 98] Address in use

./t/do-not-track+test-do-not-track *failed* | During handling of the above exception, another exception occurred:

./t/do-not-track+test-do-not-track *failed* | Traceback (most recent call last):
./t/do-not-track+test-do-not-track *failed* |   File "/bin/api-earthly-stub-server", line 105, in <module>
./t/do-not-track+test-do-not-track *failed* |     os.write(ready_pipe_w, f'unexpected exception while starting server: {e}'.encode('utf8'))
./t/do-not-track+test-do-not-track *failed* | OSError: [Errno 9] Bad file descriptor