Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

GDAL_HTTP_RETRY_ALL Configuration option #9941

Closed
quantumcode-martin opened this issue May 16, 2024 · 2 comments · Fixed by #9956
Closed

GDAL_HTTP_RETRY_ALL Configuration option #9941

quantumcode-martin opened this issue May 16, 2024 · 2 comments · Fixed by #9956
Assignees
@rouault
Labels
enhancement

Comments

@quantumcode-martin
Copy link
Contributor

Feature description

It would be nice to be able to set the HTTP retry behaviour of GDAL to happen on every HTTP error.

For example, a (probably wrongly configured) WFS server is sometimes returning an error 400 on a random valid request.
Then GDAL doesn't retry the request (which would make sense if the error code was correct) causing failure of the download.

This would allow more flexibility I think, and help to deal with wrongly configured WFS servers.

Additional context

No response
@jratike80
Copy link
Collaborator

Servers can be misconfigured in many ways but for me most of the 4xx and 5xx error codes are not worth retry https://httpwg.org/specs/rfc9110.html#status.400 https://httpwg.org/specs/rfc9110.html#status.500. Currently GDAL retries these:
HTTP errors 429, 502, 503, or 504..

Maybe some code could be added to the list if there is evidence that it could give more benefit than it generates spam.

@quantumcode-martin
Copy link
Contributor Author

I agree with what you say, but in my case it's really blocking as I can't bypass the issue on this WFS server.

@rouault rouault self-assigned this May 16, 2024
rouault added a commit to rouault/gdal that referenced this issue May 16, 2024
@rouault
CPLHTTPFetch(): add a RETRY_CODES option (GDAL_HTTP_RETRY_CODES confi…
f66020b 
…g option)

Fixes OSGeo#9941

```
-  .. config:: GDAL_HTTP_RETRY_CODES
      :choices: ALL or comma-separated list of codes
      :since: 3.10

      Specify which HTTP error codes should trigger a retry attempt.
      Valid values are ``ALL`` or a comma-separated list of HTTP codes.
      By default, 429, 500, 502, 503 or 504 HTTP errors are considered, as
      well as other situations with a particular HTTP or Curl error message.
```
@rouault rouault mentioned this issue May 16, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@rouault rouault

Labels
enhancement
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

CPLHTTPFetch(): add a RETRY_CODES option, and C++'ify HTTP retry logic rouault/gdal
3 participants
@rouault @jratike80 @quantumcode-martin