Add retry mechanism for HTTP requests and update dependencies #59
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request includes several changes to improve the robustness and reliability of HTTP requests in the
ckan2pycsw
script and updates to the project dependencies. Key changes include the addition of a retry mechanism for HTTP requests, improvements to error handling, and updates to thepyproject.toml
file.Improvements to HTTP requests and error handling:
ckan2pycsw/ckan2pycsw.py
: AddedRetry
andHTTPAdapter
from therequests
library to configure retries for HTTP requests, and updated theget_datasets
function to use this session with retries and a timeout. Improved error handling by adding specific exceptions forTimeout
andConnectionError
. [1] [2] [3]Dependency updates:
pyproject.toml
: Addedurllib3
to the dependencies to support the retry mechanism.pyproject.toml
: Removed an unnecessary section from thetool.pdm
configuration.