http3 quiche: QUIC connection is draining

[mb]

I did this

Running curl3 (curl with quiche) from AUR: https://aur.archlinux.org/packages/curl-http3
Connecting to the mozilla-central http3 test server using the first port outputted by the http3server.

$ SSLKEYLOGFILE=~/tmp/test.keys curl3 -i --http3-only https://127.0.0.1:41996/hello --insecure -vv
* processing: https://127.0.0.1:41996/hello
*   Trying 127.0.0.1:41996...
* Connected to 127.0.0.1 (127.0.0.1) port 41996
* using HTTP/3
* Using HTTP/3 Stream ID: 0
> GET /hello HTTP/3
> Host: 127.0.0.1:41996
> User-Agent: curl/8.2.1
> Accept: */*
> 
* QUIC connection is draining
* Connection #0 to host 127.0.0.1 left intact
curl: (95) QUIC connection is draining

The error is coming from:

• curl/lib/vquic/curl_quiche.c

  Line 927 in 5ee0b9d

  failf(data, "QUIC connection is draining");

• https://docs.rs/quiche/0.18.0/quiche/struct.Connection.html#method.is_draining

  Returns true if the connection is draining.

  If this returns true, the connection object cannot yet be dropped, but no new application data can be sent or received. An application should continue calling the recv(), timeout(), and on_timeout() methods as normal, until the is_closed() method returns true.

  In contrast, once is_draining() returns true, calling send() is not required because no new outgoing packets will be generated.

draining is not an error to drop the connection. As far as I can tell the connection is terminated on that error.

Test server binaries and wireshark capture including the SSLKEYLOGFILE: test-setup.zip

I expected the following

I would expect the connection not to be terminated, but to be continued. I hope I understand it correctly and this is really a bug here, sorry if not.

curl/libcurl version

$ curl3 --version
curl 8.2.1 (x86_64-pc-linux-gnu) libcurl/8.2.1 BoringSSL zlib/1.3 brotli/1.1.0 zstd/1.5.5 libidn2/2.3.4 libpsl/0.21.2 (+libidn2/2.3.4) libssh2/1.11.0 nghttp2/1.56.0 quiche/0.17.2 librtmp/2.3
Release-Date: 2023-07-26
Protocols: dict file ftp ftps gopher gophers http https imap imaps mqtt pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS brotli GSS-API HSTS HTTP2 HTTP3 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM NTLM_WB PSL SPNEGO SSL threadsafe UnixSockets zstd

operating system

$ uname -a
Linux archlinux 6.5.5-arch1-1 #1 SMP PREEMPT_DYNAMIC Sat, 23 Sep 2023 22:55:13 +0000 x86_64 GNU/Linux

[bagder]

Can you reproduce this against a publicly available URL?

[bagder]

I have not been able to reproduce, because I get reproducible crashes in quiche all the time: cloudflare/quiche#1633

[mb]

I don't know any publicly available server with that error. I tried to setup the http3server on my test server. Unfortunately, I can't reproduce there https://curl-neqo.neon.rocks:4433. I don't get to the stage where I connect and haven't been able to send out the GET / request, because the connection gets reset for some reason. curl3 -vv --http3-only https://curl-neqo.neon.rocks:4433

 curl3 -vv --http3-only https://curl-neqo.neon.rocks:4433
* processing: https://curl-neqo.neon.rocks:4433
*   Trying 89.58.41.211:4433...
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: none
*   Trying 89.58.41.211:4433...
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: none
*   Trying 89.58.41.211:4433...
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: none
*   Trying 89.58.41.211:4433...
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: none
*   Trying 89.58.41.211:4433...
*  CAfile: /etc/ssl/certs/ca-certificates.crt
[...]

curl3 -vv --http3-only https://curl-neqo.neon.rocks:4433 --insecure
* processing: https://curl-neqo.neon.rocks:4433
*   Trying 89.58.41.211:4433...
*   Trying 89.58.41.211:4433...
*   Trying 89.58.41.211:4433...
*   Trying 89.58.41.211:4433...
[...]

[icing]

@mb could you try the current master? We have made changes in the retry behaviour when connecting to servers that immediately go into DRAINING state.

[bagder]

@mb ?

I can't repro that problem with --http3-only https://curl-neqo.neon.rocks:4433 just now:

$ curl --http3-only https://curl-neqo.neon.rocks:4433
curl: (8) Failed to connect to curl-neqo.neon.rocks port 4433 after 104 ms: Weird server reply

[mb]

Hm, locally I still get the connection is draining error:

$ curl3 --version
curl 8.5.0 (x86_64-pc-linux-gnu) libcurl/8.5.0 BoringSSL zlib/1.3 brotli/1.1.0 zstd/1.5.5 libidn2/2.3.4 libpsl/0.21.2 (+libidn2/2.3.4) libssh2/1.11.0 nghttp2/1.58.0 quiche/0.20.0 librtmp/2.3
Release-Date: 2023-12-06
Protocols: dict file ftp ftps gopher gophers http https imap imaps mqtt pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS brotli GSS-API HSTS HTTP2 HTTP3 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM PSL SPNEGO SSL threadsafe UnixSockets zstd

$ curl3 https://127.0.0.1:60930 --http3-only --insecure -vv -i
*   Trying 127.0.0.1:60930...
* Connected to 127.0.0.1 (127.0.0.1) port 60930
* using HTTP/3
* [HTTP/3] [0] OPENED stream for https://127.0.0.1:60930/
* [HTTP/3] [0] [:method: GET]
* [HTTP/3] [0] [:scheme: https]
* [HTTP/3] [0] [:authority: 127.0.0.1:60930]
* [HTTP/3] [0] [:path: /]
* [HTTP/3] [0] [user-agent: curl/8.5.0]
* [HTTP/3] [0] [accept: */*]
> GET / HTTP/3
> Host: 127.0.0.1:60930
> User-Agent: curl/8.5.0
> Accept: */*
> 
* QUIC connection is draining
* Connection #0 to host 127.0.0.1 left intact
curl: (95) QUIC connection is draining

Sorry for long reply time. I can confirm that I get the same error you see on https://curl-neqo.neon.rocks:4433. That is weird. I have a hard time testing, because installing curl with http3 support on the server is harder than on my dev machine. It's probably the different hostname that produces the weird server reply. So the hosted test server might be useless, unless I can modify it to ignore the sent hostname.

[icing]

You can send a URL to localhost with curl --resolve curl-nego.neon.rocks:4433:127.0.0.1. Then the hostname the server sees may work better for you.