Skip to content
This repository has been archived by the owner on Nov 19, 2024. It is now read-only.

Don't require upstream patches for boto3 #3

Merged
merged 1 commit into from
Feb 7, 2024

Conversation

lpsinger
Copy link
Member

@lpsinger lpsinger commented Feb 7, 2024

The AWS Python SDK, boto3, has resource objects that provide high-level interfaces to AWS services. The DynamoDB resource greatly simplifies marshalling and unmarshalling data. We rely on the resource method for TransactWriteItems among others that are absent from boto3. We opened PR
boto/boto3#4010 to add that method.

The resource methods are synthesized at runtime from a data file. Fortunately, boto3 has a Loader mechanism that allows the user to add extra data files, and the loader search path is configurable.

In order to not depend upon our upstream PR for boto3, we distribute the extra data files and fix up the loader search path by putting it in a .pth file which Python executes automatically during startup. The data and .pth file are now part of an external package, boto3-missing.

@lpsinger lpsinger force-pushed the boto3-missing branch 2 times, most recently from 300c72c to 01baa52 Compare February 7, 2024 17:58
@lpsinger lpsinger marked this pull request as ready for review February 7, 2024 17:59
The AWS Python SDK, [boto3], has [resource] objects that provide
high-level interfaces to AWS services. The [DynamoDB resource]
greatly simplifies marshalling and unmarshalling data. We rely on
the resource method for [TransactWriteItems] among others that are
absent from boto3. We opened PR
boto/boto3#4010 to add that method.

The resource methods are synthesized at runtime from a data file.
Fortunately, boto3 has a [Loader] mechanism that allows the user to
add extra data files, and the [loader search path] is configurable.

In order to not depend upon our upstream PR for boto3, we distribute
the extra data files and fix up the loader search path by putting it
in a [.pth file] which Python executes automatically during startup.
The data and .pth file are now part of an external package,
[boto3-missing].

[boto3]: https://github.com/boto/boto3
[resource]: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/resources.html
[DynamoDB resource]: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/dynamodb.html#resources
[TransactWriteItems]: https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_TransactWriteItems.html
[Loader]: https://botocore.amazonaws.com/v1/documentation/api/latest/reference/loaders.html
[loader search path]: https://botocore.amazonaws.com/v1/documentation/api/latest/reference/loaders.html#the-search-path
[.pth file]: https://docs.python.org/3/library/site.html
[boto3-missing]: https://github.com/nasa-gcn/boto3-missing
@lpsinger lpsinger merged commit f78f8d9 into nasa-gcn:main Feb 7, 2024
1 check passed
@lpsinger lpsinger deleted the boto3-missing branch February 7, 2024 19:18
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant