Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

git clone hangs with curl-8.7.1 #13474

Closed
thesamesam opened this issue Apr 25, 2024 · 7 comments
Closed

git clone hangs with curl-8.7.1 #13474

thesamesam opened this issue Apr 25, 2024 · 7 comments

Comments

@thesamesam
Copy link
Contributor

thesamesam commented Apr 25, 2024 •
edited

I did this

Originally reported downstream in Gentoo at https://bugs.gentoo.org/930633 by @hlein, although he reproduced it on EndeavourOS (an Arch derivative) too.

$ git --version
git version 2.44.0

$ git clone --origin=origin --mirror https://git.rockbox.org/cgit/rockbox.git rockbox
Cloning into bare repository 'rockbox'... # <-- hangs

$ GIT_TRACE=1 git clone --origin=origin --mirror https://git.rockbox.org/cgit/rockbox.git rockbox
11:36:58.598401 git.c:463               trace: built-in: git clone --origin=origin --mirror https://git.rockbox.org/cgit/rockbox.git rockbox
Cloning into bare repository 'rockbox'...
11:36:58.599803 run-command.c:657       trace: run_command: git remote-https origin https://git.rockbox.org/cgit/rockbox.git
11:36:58.601069 git.c:750               trace: exec: git-remote-https origin https://git.rockbox.org/cgit/rockbox.git
11:36:58.601174 run-command.c:657       trace: run_command: git-remote-https origin https://git.rockbox.org/cgit/rockbox.git

Full output of GIT_TRACE=1 GIT_TRACE_CURL=1 git clone --origin=origin --mirror https://git.rockbox.org/cgit/rockbox.git rockbox, killed after a few minutes: curl-trace-git-hang.txt.gz

Attaching gdb after it is stuck for a few minutes:

0x00007fffc78fb880 in clock_gettime ()
(gdb) bt
#0  0x00007fffc78fb880 in clock_gettime ()
#1  0x00007f5629b8b589 in clock_gettime () from /usr/lib64/libc.so.6
#2  0x00007f5629dc3d54 in Curl_now () at /usr/src/debug/net-misc/curl-8.7.1-r3/curl-8.7.1/lib/timeval.c:96
#3  curl_multi_perform (multi=0x557a2b757800, running_handles=running_handles@entry=0x7fffc788bc50) at /usr/src/debug/net-misc/curl-8.7.1-r3/curl-8.7.1/lib/multi.c:2679
#4  0x0000557a29712edf in step_active_slots () at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http.c:1551
#5  fetch_alternates (walker=0x557a2c1f1dc0, base=0x557a2c1f1e30 "https://git.rockbox.org/cgit/rockbox.git") at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http-walker.c:356
#6  0x0000557a29711db5 in process_object_response (callback_data=0x557a2bcb2a70) at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http-walker.c:107
#7  0x0000557a29707c89 in finish_active_slot (slot=0x557a2b75c2d0) at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http.c:329
#8  process_curl_messages () at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http.c:353
#9  0x0000557a29713075 in step_active_slots () at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http.c:1554
#10 step_active_slots () at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http.c:1545
#11 fetch_alternates (walker=0x557a2c1f1dc0, base=0x557a2c1f1e30 "https://git.rockbox.org/cgit/rockbox.git") at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http-walker.c:356
#12 0x0000557a29711db5 in process_object_response (callback_data=0x557a2c21afc0) at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http-walker.c:107
#13 0x0000557a29707c89 in finish_active_slot (slot=0x557a2c21b320) at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http.c:329
#14 process_curl_messages () at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http.c:353
#15 0x0000557a29713075 in step_active_slots () at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http.c:1554
#16 step_active_slots () at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http.c:1545
#17 fetch_alternates (walker=0x557a2c1f1dc0, base=0x557a2c1f1e30 "https://git.rockbox.org/cgit/rockbox.git") at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http-walker.c:356
#18 0x0000557a29711db5 in process_object_response (callback_data=0x557a2bcb30b0) at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http-walker.c:107
#19 0x0000557a29707c89 in finish_active_slot (slot=0x557a2c210010) at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http.c:329
#20 process_curl_messages () at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http.c:353
#21 0x0000557a29713075 in step_active_slots () at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http.c:1554
#22 step_active_slots () at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http.c:1545
#23 fetch_alternates (walker=0x557a2c1f1dc0, base=0x557a2c1f1e30 "https://git.rockbox.org/cgit/rockbox.git") at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http-walker.c:356
#24 0x0000557a29711db5 in process_object_response (callback_data=0x557a2c21b830) at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http-walker.c:107
#25 0x0000557a29707c89 in finish_active_slot (slot=0x557a2b8409e0) at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http.c:329
#26 process_curl_messages () at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http.c:353
#27 0x0000557a29713075 in step_active_slots () at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http.c:1554
#28 step_active_slots () at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http.c:1545
#29 fetch_alternates (walker=0x557a2c1f1dc0, base=0x557a2c1f1e30 "https://git.rockbox.org/cgit/rockbox.git") at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http-walker.c:356
#30 0x0000557a29711db5 in process_object_response (callback_data=0x557a2c2154a0) at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http-walker.c:107
#31 0x0000557a29707c89 in finish_active_slot (slot=0x557a2c20f260) at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http.c:329
#32 process_curl_messages () at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http.c:353
#33 0x0000557a29707e15 in get_active_slot () at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http.c:1411
#34 0x0000557a29712f36 in fetch_alternates (walker=0x557a2c1f1dc0, base=0x557a2c1f1e30 "https://git.rockbox.org/cgit/rockbox.git")
    at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http-walker.c:375
#35 0x0000557a29711db5 in process_object_response (callback_data=0x557a2c20fef0) at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http-walker.c:107
#36 0x0000557a29707c89 in finish_active_slot (slot=0x557a2c21b320) at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http.c:329
#37 process_curl_messages () at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http.c:353
#38 0x0000557a29713075 in step_active_slots () at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http.c:1554
#39 step_active_slots () at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http.c:1545
#40 fetch_alternates (walker=0x557a2c1f1dc0, base=0x557a2c1f1e30 "https://git.rockbox.org/cgit/rockbox.git") at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http-walker.c:356
#41 0x0000557a29711db5 in process_object_response (callback_data=0x557a2c210060) at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http-walker.c:107
#42 0x0000557a29707c89 in finish_active_slot (slot=0x557a2c210010) at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http.c:329
#43 process_curl_messages () at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http.c:353
#44 0x0000557a29713075 in step_active_slots () at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http.c:1554
#45 step_active_slots () at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http.c:1545
#46 fetch_alternates (walker=0x557a2c1f1dc0, base=0x557a2c1f1e30 "https://git.rockbox.org/cgit/rockbox.git") at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http-walker.c:356
#47 0x0000557a29711db5 in process_object_response (callback_data=0x557a2b75adc0) at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http-walker.c:107
#48 0x0000557a29707c89 in finish_active_slot (slot=0x557a2c20f260) at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http.c:329
#49 process_curl_messages () at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http.c:353
#50 0x0000557a29713075 in step_active_slots () at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http.c:1554
#51 step_active_slots () at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http.c:1545
#52 fetch_alternates (walker=0x557a2c1f1dc0, base=0x557a2c1f1e30 "https://git.rockbox.org/cgit/rockbox.git") at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http-walker.c:356
#53 0x0000557a29711db5 in process_object_response (callback_data=0x557a2c1f5060) at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http-walker.c:107
#54 0x0000557a29707c89 in finish_active_slot (slot=0x557a2b8409e0) at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http.c:329
#55 process_curl_messages () at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http.c:353
#56 0x0000557a29708355 in step_active_slots () at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http.c:1554
#57 step_active_slots () at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http.c:1545
#58 run_active_slot (slot=0x557a2b75c2d0) at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http.c:1570
#59 0x0000557a29712fe1 in fetch_alternates (walker=<optimized out>, base=0x557a2c1f1e30 "https://git.rockbox.org/cgit/rockbox.git")
    at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http-walker.c:391
#60 0x0000557a29711db5 in process_object_response (callback_data=0x557a2c1f2180) at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http-walker.c:107
#61 0x0000557a29707c89 in finish_active_slot (slot=0x557a2b75c2d0) at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http.c:329
#62 process_curl_messages () at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http.c:353
#63 0x0000557a29708275 in step_active_slots () at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http.c:1554
#64 step_active_slots () at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http.c:1545
#65 prefetch (walker=0x557a2c1f1dc0, sha1=0x557a2c344dec "k\356\275u\347yR+\374\231\354W\v\263\227\004\037\336", <incomplete sequence \364\234>)
    at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/http-walker.c:164
#66 0x0000557a297a5e2e in process.isra.0 (walker=0x557a2c1f1dc0, obj=0x557a2c344de8) at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/walker.c:158
#67 0x0000557a2970d0db in walker_fetch (write_ref=0x0, write_ref_log_details=0x0, walker=0x557a2c1f1dc0, targets=690040849, target=0x7f5629213010)
    at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/walker.c:306
#68 fetch_dumb (nr_heads=nr_heads@entry=19931, to_fetch=to_fetch@entry=0x7f5629e3b010) at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/remote-curl.c:1158
#69 0x0000557a2970ff5c in fetch (nr_heads=<optimized out>, to_fetch=0x7f5629e3b010) at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/remote-curl.c:1238
#70 parse_fetch (buf=0x7fffc788d4a0) at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/remote-curl.c:1284
#71 cmd_main (argc=<optimized out>, argv=<optimized out>) at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/remote-curl.c:1579
#72 0x0000557a2970433a in main (argc=3, argv=0x7fffc788da68) at /usr/src/debug/dev-vcs/git-2.44.0/git-2.44.0/common-main.c:62

I expected the following

Successful git clone, but instead, git clone hanged for ~7.5 hours until killed.

curl/libcurl version

$ curl -V
curl 8.7.1 (x86_64-pc-linux-gnu) libcurl/8.7.1 OpenSSL/3.2.1 zlib/1.3.1 c-ares/1.28.1 libpsl/0.21.5 nghttp2/1.61.0
Release-Date: 2024-03-27
Protocols: dict file ftp ftps http https imap imaps ipfs ipns mqtt pop3 pop3s rtsp smtp smtps tftp
Features: alt-svc AsynchDNS HSTS HTTP2 HTTPS-proxy IPv6 Largefile libz NTLM PSL SSL threadsafe TLS-SRP UnixSockets

operating system

Gentoo on:

Linux mop 6.6.28 #1 SMP PREEMPT_DYNAMIC Mon Apr 22 23:31:55 BST 2024 x86_64 AMD Ryzen 9 3950X 16-Core Processor AuthenticAMD GNU/Linux
@thesamesam
Copy link
Contributor Author

thesamesam commented Apr 25, 2024 •
edited

I tried to bisect for the fix on master, as master at ac7670b works, and got 5c59f91, but I'm not sure if it's right or not (because the bug it mentions is kind of different - #13411).

It doesn't apply cleanly to 8.7.1 unfortunately and I think it's beyond me to try backport it myself as it touches a few different areas that earlier commits modified too.

@thesamesam
Copy link
Contributor Author

cc @icing

@icing
Copy link
Contributor

icing commented Apr 25, 2024

How do we proceed here? Are you asking for a patch version of 5c59f91 that can be applied to 8.7.1?

@bagder how do we want to handle this?

@thesamesam
Copy link
Contributor Author

How do we proceed here? Are you asking for a patch version of 5c59f91 that can be applied to 8.7.1?

I think so, yes please, if that commit sounds plausible as a fix for it and I'm not talking nonsense :)

I tried checking out 5c59f91427c6e14036070d4e1426360393458b25~1 which hanged and 5c59f91427c6e14036070d4e1426360393458b25 which worked.

@icing
Copy link
Contributor

icing commented Apr 26, 2024

@thesamesam could you check if https://gist.github.com/icing/873a18d24230cf1ee18963305e167c09 works for you?

@thesamesam
Copy link
Contributor Author

@icing That works, thank you very much!

gentoo-bot pushed a commit to gentoo/gentoo that referenced this issue Apr 26, 2024
@thesamesam
net-misc/curl: backport fix for hanging git clone over HTTP2
9e15b31 
Thank you to Stefan for giving us a bespoke backport, as the original
fix didn't apply cleanly to 8.7.1.

Bug: curl/curl#13474
Closes: https://bugs.gentoo.org/930633
Thanks-to: Stefan Eissing <[email protected]>
Signed-off-by: Sam James <[email protected]>
@bagder
Copy link
Member

bagder commented Apr 26, 2024

Fixed in master, and now also backported to gentoo.

@bagder bagder closed this as completed Apr 26, 2024
archlinux-github pushed a commit to archlinux/aur that referenced this issue Apr 28, 2024
Added to makedepends: 'curl>8.7.1' # Because of https://bugs.gentoo.o…
ca9b8a8 
…rg/930633 / curl/curl#13474
archlinux-github pushed a commit to archlinux/aur that referenced this issue Apr 28, 2024
Added to makedepends: 'curl>8.7.1' # Because of https://bugs.gentoo.o…
e01aa60 
…rg/930633 / curl/curl#13474
archlinux-github pushed a commit to archlinux/aur that referenced this issue Apr 28, 2024
Added to makedepends: 'curl>8.7.1' # Because of https://bugs.gentoo.o…
d54825e 
…rg/930633 / curl/curl#13474
archlinux-github pushed a commit to archlinux/aur that referenced this issue Apr 28, 2024
Added to makedepends: 'curl>8.7.1' # Because of https://bugs.gentoo.o…
6f1697a 
…rg/930633 / curl/curl#13474
archlinux-github pushed a commit to archlinux/aur that referenced this issue Apr 28, 2024
Added to makedepends: 'curl>8.7.1' # Because of https://bugs.gentoo.o…
24315af 
…rg/930633 / curl/curl#13474
archlinux-github pushed a commit to archlinux/aur that referenced this issue Apr 28, 2024
Added to makedepends: 'curl>8.7.1' # Because of https://bugs.gentoo.o…
84734d8 
…rg/930633 / curl/curl#13474
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@icing @bagder @thesamesam