NTLM authentication failing on Linux unless --http2 is specified #13291

andreiaugustin · 2024-04-05T09:34:02Z

I did this

curl http://192.168.0.234/ -v --ntlm -u andrei:password

I expected the following

IIS index page to be returned, instead I get back a 401, not authorized:

*   Trying 192.168.0.234:80...
* Connected to 192.168.0.234 (192.168.0.234) port 80
* Server auth using NTLM with user 'andrei'
> GET / HTTP/1.1
> Host: 192.168.0.234
> Authorization: NTLM [redacted]
> User-Agent: curl/8.6.0
> Accept: */*
> 
< HTTP/1.1 401 Unauthorized
< Content-Type: text/html; charset=us-ascii
< Server: Microsoft-HTTPAPI/2.0
< WWW-Authenticate: NTLM [redacted]
< Date: Fri, 05 Apr 2024 09:19:31 GMT
< Connection: close
< Content-Length: 341
* Connection closure while negotiating auth (HTTP 1.0?)
< 
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN""http://www.w3.org/TR/html4/strict.dtd">
<HTML><HEAD><TITLE>Not Authorized</TITLE>
<META HTTP-EQUIV="Content-Type" Content="text/html; charset=us-ascii"></HEAD>
<BODY><h2>Not Authorized</h2>
<hr><p>HTTP Error 401. The requested resource requires user authentication.</p>
</BODY></HTML>
* Closing connection

This works fine on both Windows 11 and macOS.

It seems to be failing at the last step in NTLM authentication, where client should be reattempting auth with the challenge received from the server?

If I instead do curl http://192.168.0.234/ -v --ntlm -u andrei:password --http2, it works:

./curl http://192.168.0.234/ -v --ntlm -u andrei:password--http2

*   Trying 192.168.0.234:80...
* Connected to 192.168.0.234 (192.168.0.234) port 80
* Server auth using NTLM with user 'andrei'
> GET / HTTP/1.1
> Host: 192.168.0.234
> Authorization: NTLM [redacted]
> User-Agent: curl/8.6.0
> Accept: */*
> Connection: Upgrade, HTTP2-Settings
> Upgrade: h2c
> HTTP2-Settings: AAMAAABkAAQAoAAAAAIAAAAA
> 
< HTTP/1.1 401 Unauthorized
< Content-Type: text/html; charset=us-ascii
< Server: Microsoft-HTTPAPI/2.0
< WWW-Authenticate: NTLM [redacted]
< Date: Fri, 05 Apr 2024 09:23:00 GMT
< Content-Length: 341
< 
* Ignoring the response-body
* Connection #0 to host 192.168.0.234 left intact
* Issue another request to this URL: 'http://192.168.0.234/'
* Found bundle for host: 0x56424d0ae350 [serially]
* Can not multiplex, even if we wanted to
* Re-using existing connection with host 192.168.0.234
* Server auth using NTLM with user 'andrei'
> GET / HTTP/1.1
> Host: 192.168.0.234
> Authorization: NTLM [redacted]
> User-Agent: curl/8.6.0
> Accept: */*
> Connection: Upgrade, HTTP2-Settings
> Upgrade: h2c
> HTTP2-Settings: AAMAAABkAAQAoAAAAAIAAAAA
> 
< HTTP/1.1 200 OK
< Content-Type: text/html
< Last-Modified: Fri, 05 Apr 2024 08:25:45 GMT
< Accept-Ranges: bytes
< ETag: "28f37db3287da1:0"
< Server: Microsoft-IIS/10.0
< Persistent-Auth: true
< Date: Fri, 05 Apr 2024 09:23:00 GMT
< Content-Length: 696
< 
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
<title>IIS Windows</title>
<style type="text/css">
<!--
body {
        color:#000000;
        background-color:#0072C6;
        margin:0;
}

#container {
        margin-left:auto;
        margin-right:auto;
        text-align:center;
        }

a img {
        border:none;
}

-->
</style>
</head>
<body>
<div id="container">
<a href="http://go.microsoft.com/fwlink/?linkid=66138&amp;clcid=0x409"><img src="iisstart.png" alt="IIS" width="960" height="600" /></a>
</div>
</body>
* Connection #0 to host 192.168.0.234 left intact

Trying any other option such as --http1.0 or --http1.1 results in the same 401, only --http2 works. Also tried with the default curl build that comes on the system, here's that one's -V:

curl 7.88.1 (x86_64-pc-linux-gnu) libcurl/7.88.1 OpenSSL/3.0.11 zlib/1.2.13 brotli/1.0.9 zstd/1.5.4 libidn2/2.3.3 libpsl/0.21.2 (+libidn2/2.3.3) libssh2/1.10.0 nghttp2/1.52.0 librtmp/2.3 OpenLDAP/2.5.13
Release-Date: 2023-02-20, security patched: 7.88.1-10+deb12u5
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS brotli GSS-API HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM NTLM_WB PSL SPNEGO SSL threadsafe TLS-SRP UnixSockets zstd

I don't know whether this is worth fixing since Microsoft has deprecated NTLM, so I'm raising this issue with the goal to perhaps getting it documented, in case it's replicable for others too?

curl/libcurl version

On Linux:

curl 8.6.0 (x86_64-pc-linux-gnu) libcurl/8.6.0 mbedTLS/3.5.2 (OpenSSL/3.0.11) zlib/1.3.1 brotli/1.0.9 zstd/1.5.4 libidn2/2.3.3 libssh2/1.11.0 nghttp2/1.59.0 OpenLDAP/2.5.13
Release-Date: 2024-01-31
Protocols: file ftp ftps http https imap imaps ipfs ipns ldap ldaps mqtt pop3 pop3s scp sftp smtp smtps
Features: alt-svc AsynchDNS brotli HSTS HTTP2 HTTPS-proxy IDN IPv6 Largefile libz MultiSSL NTLM SSL threadsafe TLS-SRP UnixSockets zstd

On Windows:

curl 8.6.0 (x86_64-pc-win32) libcurl/8.6.0 mbedTLS/3.5.2 (Schannel) zlib/1.3.1 WinIDN libssh2/1.11.0 nghttp2/1.59.0
Release-Date: 2024-01-31
Protocols: file ftp ftps http https imap imaps ipfs ipns ldap ldaps mqtt pop3 pop3s scp sftp smtp smtps
Features: alt-svc AsynchDNS HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz MultiSSL NTLM SPNEGO SSL SSPI threadsafe UnixSockets

operating system

Linux f67e3e8643fb 5.15.146.1-microsoft-standard-WSL2 #1 SMP Thu Jan 11 04:09:03 UTC 2024 x86_64 GNU/Linux

On Windows, running with Windows 11 23H2 OS Build 22631.3296 with IIS Version 10.0.22621.1

The text was updated successfully, but these errors were encountered:

bagder · 2024-04-08T09:09:57Z

I don't know whether this is worth fixing since Microsoft has deprecated NTLM

NTLM has been an inferior solution since forever but people still seem to use it. I don't think Microsoft's intention will serve as a strong guidance for us.

I think more importantly: if we can't get a way to reproduce this case, I don't see how we can debug/fix it. We can however cheer for your attempts in doing so.

andreiaugustin · 2024-04-08T09:33:18Z

Thanks! I totally agree - I just thought it might be worth adding it as a known issue, at least until NTLM support is removed, granted it can be replicated on other machines as well of course.

Here's the steps I went thought to replicate it, first I set up IIS on my Windows machine:

Settings > System > Optional features > More Windows features
Check: Internet Information Services
Expand Internet Information services, expand Security and finally check Windows Authentication
Click OK to start installing the new features

Once installed, IIS should be available like it is on a Windows server.

Enable Windows Authentication:

Open IIS and select the Default Web Site
Select Authentication
Disable Anonymous Authentication
Enable Windows Authentication
In my case, I also selected Providers... and moved NTLM up, so it is first. I don't think this will make a difference?

Using cURL on a Linux machine (in my case I was using Docker which seems to run on the WSL), GET the index page on IIS using NTLM authentication:

curl http://[your ISS IP address]/ -v --ntlm -u windowsUsername:windowsPassword

I would love to debug this, but I doubt I will get the time to... On the grand scheme of things I believe it's very low priority - at least in my tests, asking cURL to attempt upgrading to http2 solved the issue, even if the server decided to keep using http1.1 during the authentication steps. Considering NTLM's inferiority as a security solution and its deprecation from Microsoft, even if it can be replicated, it's probably not worth the time...

My end goal was to at least have this documented somewhere, in case anybody else ever runs into the same issue 😃

edmcln · 2024-04-13T19:56:56Z

I can't reproduce your results.

root@debian:~# curl -Lv 192.168.x.x --ntlm -u xxx:xxx
*   Trying 192.168.x.x:80...
* Connected to 192.168.x.x (192.168.x.x) port 80 (#0)
* Server auth using NTLM with user 'xxx'
> GET / HTTP/1.1
> Host: 192.168.x.x
> Authorization: NTLM [REDACTED]
> User-Agent: curl/7.88.1
> Accept: */*
>
< HTTP/1.1 401 Unauthorized
< Content-Type: text/html
< Server: Microsoft-IIS/10.0
< WWW-Authenticate: NTLM
< WWW-Authenticate: Negotiate
< Date: Sat, 13 Apr 2024 18:41:59 GMT
< Content-Length: 1293
<
* Ignoring the response-body
* Connection #0 to host 192.168.x.x left intact
* Issue another request to this URL: 'http://192.168.x.x/'
* Found bundle for host: 0x5e73e0 [serially]
* Can not multiplex, even if we wanted to
* Re-using existing connection #0 with host 192.168.x.x
* Server auth using NTLM with user 'xxx'
> GET / HTTP/1.1
> Host: 192.168.x.x
> Authorization: NTLM [REDACTED]
> User-Agent: curl/7.88.1
> Accept: */*
>
< HTTP/1.1 401 Unauthorized
< Content-Type: text/html; charset=us-ascii
< Server: Microsoft-HTTPAPI/2.0
< WWW-Authenticate: NTLM [REDACTED]
< Date: Sat, 13 Apr 2024 18:41:59 GMT
< Content-Length: 341
<
* Ignoring the response-body
* Connection #0 to host 192.168.x.x left intact
* Issue another request to this URL: 'http://192.168.x.x/'
* Found bundle for host: 0x5e73e0 [serially]
* Can not multiplex, even if we wanted to
* Re-using existing connection #0 with host 192.168.x.x
* Server auth using NTLM with user 'xxx'
> GET / HTTP/1.1
> Host: 192.168.x.x
> Authorization: NTLM [REDACTED]
> User-Agent: curl/7.88.1
> Accept: */*
>
< HTTP/1.1 200 OK
< Content-Type: text/html
< Last-Modified: Sat, 13 Apr 2024 16:26:07 GMT
< Accept-Ranges: bytes
< ETag: "e2a66449bf8dda1:0"
< Server: Microsoft-IIS/10.0
< Persistent-Auth: true
< Date: Sat, 13 Apr 2024 18:41:59 GMT
< Content-Length: 696
<
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
<title>IIS Windows</title>
<style type="text/css">
<!--
body {
        color:#000000;
        background-color:#0072C6;
        margin:0;
}

#container {
        margin-left:auto;
        margin-right:auto;
        text-align:center;
        }

a img {
        border:none;
}

-->
</style>
</head>
<body>
<div id="container">
<a href="http://go.microsoft.com/fwlink/?linkid=66138&amp;clcid=0x409"><img src="iisstart.png" alt="IIS" width="960" height="600" /></a>
</div>
</body>
* Connection #0 to host 192.168.x.x left intact
</html>root@debian:~#

root@debian:~# curl -V
curl 7.88.1 (i686-pc-linux-gnu) libcurl/7.88.1 OpenSSL/3.0.11 zlib/1.2.13 brotli/1.0.9 zstd/1.5.4 libidn2/2.3.3 libpsl/0.21.2 (+libidn2/2.3.3) libssh2/1.10.0 nghttp2/1.52.0 librtmp/2.3 OpenLDAP/2.5.13
Release-Date: 2023-02-20, security patched: 7.88.1-10+deb12u5
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS brotli GSS-API HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM NTLM_WB PSL SPNEGO SSL threadsafe TLS-SRP UnixSockets zstd

andreiaugustin · 2024-04-17T08:04:03Z

Very strange it works without problems on your end? I've created a new Docker container to test in, just in case my build environment was messed up, but I can still recreate the problem:

FROM --platform=linux/amd64 debian:12.1-slim
ARG DEBIAN_FRONTEND=noninteractive

RUN apt-get update && apt-get install -y curl

Built with docker build . -t test and run with docker run -it test /bin/bash.

Inside the container, curl version:

root@9a2e4404422e:/# curl -V
curl 7.88.1 (x86_64-pc-linux-gnu) libcurl/7.88.1 OpenSSL/3.0.11 zlib/1.2.13 brotli/1.0.9 zstd/1.5.4 libidn2/2.3.3 libpsl/0.21.2 (+libidn2/2.3.3) libssh2/1.10.0 nghttp2/1.52.0 librtmp/2.3 OpenLDAP/2.5.13
Release-Date: 2023-02-20, security patched: 7.88.1-10+deb12u5
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS brotli GSS-API HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM NTLM_WB PSL SPNEGO SSL threadsafe TLS-SRP UnixSockets zstd

And executing the request:

root@9a2e4404422e:/# curl -Lv 192.168.0.234 --ntlm -u andrei:[redacted]
*   Trying 192.168.0.234:80...
* Connected to 192.168.0.234 (192.168.0.234) port 80 (#0)
* Server auth using NTLM with user 'andrei'
> GET / HTTP/1.1
> Host: 192.168.0.234
> Authorization: NTLM TlRM[redacted]
> User-Agent: curl/7.88.1
> Accept: */*
>
< HTTP/1.1 401 Unauthorized
< Content-Type: text/html; charset=us-ascii
< Server: Microsoft-HTTPAPI/2.0
< WWW-Authenticate: NTLM TlRM[redacted]
< Date: Wed, 17 Apr 2024 08:00:46 GMT
< Connection: close
< Content-Length: 341
* Connection closure while negotiating auth (HTTP 1.0?)
<
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN""http://www.w3.org/TR/html4/strict.dtd">
<HTML><HEAD><TITLE>Not Authorized</TITLE>
<META HTTP-EQUIV="Content-Type" Content="text/html; charset=us-ascii"></HEAD>
<BODY><h2>Not Authorized</h2>
<hr><p>HTTP Error 401. The requested resource requires user authentication.</p>
</BODY></HTML>
* Closing connection 0

But then with --http2:

root@9a2e4404422e:/# curl -Lv 192.168.0.234 --ntlm -u andrei:[redacted] --http2
*   Trying 192.168.0.234:80...
* Connected to 192.168.0.234 (192.168.0.234) port 80 (#0)
* Server auth using NTLM with user 'andrei'
> GET / HTTP/1.1
> Host: 192.168.0.234
> Authorization: NTLM TlRM[redacted]
> User-Agent: curl/7.88.1
> Accept: */*
> Connection: Upgrade, HTTP2-Settings
> Upgrade: h2c
> HTTP2-Settings: AAMAAABkAAQCAAAAAAIAAAAA
> 
< HTTP/1.1 401 Unauthorized
< Content-Type: text/html; charset=us-ascii
< Server: Microsoft-HTTPAPI/2.0
< WWW-Authenticate: NTLM TlRM[redacted]
< Date: Wed, 17 Apr 2024 08:02:21 GMT
< Content-Length: 341
< 
* Ignoring the response-body
* Connection #0 to host 192.168.0.234 left intact
* Issue another request to this URL: 'http://192.168.0.234/'
* Found bundle for host: 0x55f7e73401a0 [serially]
* Can not multiplex, even if we wanted to
* Re-using existing connection #0 with host 192.168.0.234
* Server auth using NTLM with user 'andrei'
> GET / HTTP/1.1
> Host: 192.168.0.234
> Authorization: NTLM TlRM[redacted]
> User-Agent: curl/7.88.1
> Accept: */*
> Connection: Upgrade, HTTP2-Settings
> Upgrade: h2c
> HTTP2-Settings: AAMAAABkAAQCAAAAAAIAAAAA
> 
< HTTP/1.1 200 OK
< Content-Type: text/html
< Last-Modified: Fri, 05 Apr 2024 08:25:45 GMT
< Accept-Ranges: bytes
< ETag: "28f37db3287da1:0"
< Server: Microsoft-IIS/10.0
< Persistent-Auth: true
< Date: Wed, 17 Apr 2024 08:02:21 GMT
< Content-Length: 696
< 
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
<title>IIS Windows</title>
<style type="text/css">
<!--
body {
        color:#000000;
        background-color:#0072C6;
        margin:0;
}

#container {
        margin-left:auto;
        margin-right:auto;
        text-align:center;
        }

a img {
        border:none;
}

-->
</style>
</head>
<body>
<div id="container">
<a href="http://go.microsoft.com/fwlink/?linkid=66138&amp;clcid=0x409"><img src="iisstart.png" alt="IIS" width="960" height="600" /></a>
</div>
</body>
* Connection #0 to host 192.168.0.234 left intact

edmcln · 2024-04-30T12:29:52Z

It works fine on my end. I built the latest curl from scratch on Fedora 40 and here is the result:

[root@fedora ~]# curl -Lv 192.168.x.x --ntlm -u xxx:xxx
*   Trying 192.168.x.x:80...
* Connected to 192.168.x.x (192.168.x.x) port 80
* Server auth using NTLM with user 'xxx'
> GET / HTTP/1.1
> Host: 192.168.x.x
> Authorization: NTLM [REDACTED]
> User-Agent: curl/8.7.1
> Accept: */*
>
* Request completely sent off
< HTTP/1.1 401 Unauthorized
< Content-Type: text/html
< Server: Microsoft-IIS/10.0
< WWW-Authenticate: NTLM
< WWW-Authenticate: Negotiate
< Date: Tue, 30 Apr 2024 11:06:33 GMT
< Content-Length: 1293
<
* Ignoring the response-body
* Connection #0 to host 192.168.x.x left intact
* Issue another request to this URL: 'http://192.168.x.x/'
* Found bundle for host: 0x919580 [serially]
* Re-using existing connection with host 192.168.x.x
* Server auth using NTLM with user 'xxx'
> GET / HTTP/1.1
> Host: 192.168.x.x
> Authorization: NTLM [REDACTED]
> User-Agent: curl/8.7.1
> Accept: */*
>
* Request completely sent off
< HTTP/1.1 401 Unauthorized
< Content-Type: text/html; charset=us-ascii
< Server: Microsoft-HTTPAPI/2.0
< WWW-Authenticate: NTLM [REDACTED]
< Date: Tue, 30 Apr 2024 11:06:33 GMT
< Content-Length: 341
<
* Ignoring the response-body
* Connection #0 to host 192.168.x.x left intact
* Issue another request to this URL: 'http://192.168.x.x/'
* Found bundle for host: 0x919580 [serially]
* Re-using existing connection with host 192.168.x.x
* Server auth using NTLM with user 'xxx'
> GET / HTTP/1.1
> Host: 192.168.x.x
> Authorization: NTLM [REDACTED]
> User-Agent: curl/8.7.1
> Accept: */*
>
* Request completely sent off
< HTTP/1.1 200 OK
< Content-Type: text/html
< Last-Modified: Sat, 13 Apr 2024 16:26:07 GMT
< Accept-Ranges: bytes
< ETag: "e2a66449bf8dda1:0"
< Server: Microsoft-IIS/10.0
< Persistent-Auth: true
< Date: Tue, 30 Apr 2024 11:06:33 GMT
< Content-Length: 696
<
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
<title>IIS Windows</title>
<style type="text/css">
<!--
body {
        color:#000000;
        background-color:#0072C6;
        margin:0;
}

#container {
        margin-left:auto;
        margin-right:auto;
        text-align:center;
        }

a img {
        border:none;
}

-->
</style>
</head>
<body>
<div id="container">
<a href="http://go.microsoft.com/fwlink/?linkid=66138&amp;clcid=0x409"><img src="iisstart.png" alt="IIS" width="960" height="600" /></a>
</div>
</body>
* Connection #0 to host 192.168.x.x left intact
</html>

[root@fedora ~]# curl -V
curl 8.7.1 (x86_64-pc-linux-gnu) libcurl/8.7.1 OpenSSL/3.2.1 zlib/1.3.0.zlib-ng libpsl/0.21.5
Release-Date: 2024-03-27
Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS HSTS HTTPS-proxy IPv6 Largefile libz NTLM PSL SSL threadsafe TLS-SRP UnixSockets

bagder added HTTP authentication labels Apr 5, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

NTLM authentication failing on Linux unless --http2 is specified #13291

NTLM authentication failing on Linux unless --http2 is specified #13291

andreiaugustin commented Apr 5, 2024

bagder commented Apr 8, 2024

andreiaugustin commented Apr 8, 2024

edmcln commented Apr 13, 2024

andreiaugustin commented Apr 17, 2024 •

edited

edmcln commented Apr 30, 2024

NTLM authentication failing on Linux unless --http2 is specified #13291

NTLM authentication failing on Linux unless --http2 is specified #13291

Comments

andreiaugustin commented Apr 5, 2024

I did this

I expected the following

curl/libcurl version

operating system

bagder commented Apr 8, 2024

andreiaugustin commented Apr 8, 2024

edmcln commented Apr 13, 2024

andreiaugustin commented Apr 17, 2024 • edited

edmcln commented Apr 30, 2024

andreiaugustin commented Apr 17, 2024 •

edited