JWKClient.get_jwk_set() raises http.client.IncompleteRead

[scotchneat]

When trying to integrate into an internal JWKS endpoint, PyJWKClient.get_jwk_set() returns http.client.IncompleteRead.

I can fetch the content via curl.

As an aside, I also successfully get the response using both htpx.get() and requests.get(). I can convert that result into a jwt.PyJWKSet (using jwt.PyJWKSet.from_dict(response.json()). Then the rest of the validation works.

Additionally, I'm able to get the same results just using urllib.request.Request (as done here) directly.

Expected Result

The call should return the key set. I've verified. Ideally, the JWKClient would handle retrying this within the PyJWKClient.fetch_data method.

This example works after one retry. (expand)

import urllib.request
import http.client

def fetch_url(url, retries=3):
    for attempt in range(retries):
        try:
            with urllib.request.urlopen(url) as response:
                return response.read()
        except http.client.IncompleteRead as e:
            if attempt < retries - 1:
                print(f"Incomplete read, retrying... ({attempt + 1}/{retries})")
                continue
            else:
                raise
    raise RuntimeError("Max retries exceeded")

try:
    content = fetch_url(url)
    print("Content fetched successfully")
except Exception as e:
    print(f"Failed to fetch content: {e}")

Actual Result

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/scotchneat/dev/temp/jwt_sandbox/.venv/lib/python3.12/site-packages/jwt/jwks_client.py", line 74, in get_jwk_set
    data = self.fetch_data()
           ^^^^^^^^^^^^^^^^^
  File "/home/scotchneat/dev/temp/jwt_sandbox/.venv/lib/python3.12/site-packages/jwt/jwks_client.py", line 57, in fetch_data
    jwk_set = json.load(response)
              ^^^^^^^^^^^^^^^^^^^
  File "/home/scotchneat/.pyenv/versions/3.12.2/lib/python3.12/json/__init__.py", line 293, in load
    return loads(fp.read(),
                 ^^^^^^^^^
  File "/home/scotchneat/.pyenv/versions/3.12.2/lib/python3.12/http/client.py", line 495, in read
    s = self._safe_read(self.length)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/scotchneat/.pyenv/versions/3.12.2/lib/python3.12/http/client.py", line 642, in _safe_read
    raise IncompleteRead(data, amt-len(data))
http.client.IncompleteRead: IncompleteRead(14594 bytes read, 5172 more expected)

Reproduction Steps

import jwt
import os

url = os.getenv('JWKS_URL')

cli = jwt.PyJWKClient(url)
key_set = get_jwk_set()

System Information

$ python -m jwt.help

{
  "cryptography": {
    "version": "43.0.3"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.12.2"
  },
  "platform": {
    "release": "6.8.0-47-generic",
    "system": "Linux"
  },
  "pyjwt": {
    "version": "2.9.0"
  }
}