SImpleDB stopped working after upgrade

[depesz]

Describe the bug

We had gem versions:

• aws-sdk-core (3.191.0)
• aws-sdk-simpledb (1.26.0)

And our script worked great.

Yesterday, for some reason core was upgraded to 3.199.0 and our script no longer works.

Expected Behavior

put attribute should work

Current Behavior

raises exception:

        12: from /usr/local/bin/find_available_hostname.rb:226:in `<main>'
        11: from /usr/local/bin/find_available_hostname.rb:71:in `check_lock_in_sdb'
        10: from /var/lib/gems/2.7.0/gems/aws-sdk-simpledb-1.49.0/lib/aws-sdk-simpledb/client.rb:860:in `put_attributes'
         9: from /var/lib/gems/2.7.0/gems/aws-sdk-core-3.199.0/lib/seahorse/client/request.rb:72:in `send_request'
         8: from /var/lib/gems/2.7.0/gems/aws-sdk-core-3.199.0/lib/seahorse/client/plugins/response_target.rb:24:in `call'
         7: from /var/lib/gems/2.7.0/gems/aws-sdk-core-3.199.0/lib/aws-sdk-core/plugins/response_paging.rb:12:in `call'
         6: from /var/lib/gems/2.7.0/gems/aws-sdk-core-3.199.0/lib/seahorse/client/plugins/request_callback.rb:89:in `call'
         5: from /var/lib/gems/2.7.0/gems/aws-sdk-core-3.199.0/lib/aws-sdk-core/plugins/param_converter.rb:26:in `call'
         4: from /var/lib/gems/2.7.0/gems/aws-sdk-core-3.199.0/lib/aws-sdk-core/plugins/idempotency_token.rb:19:in `call'
         3: from /var/lib/gems/2.7.0/gems/aws-sdk-core-3.199.0/lib/aws-sdk-core/plugins/invocation_id.rb:16:in `call'
         2: from /var/lib/gems/2.7.0/gems/aws-sdk-core-3.199.0/lib/aws-sdk-core/plugins/jsonvalue_converter.rb:16:in `call'
         1: from /var/lib/gems/2.7.0/gems/aws-sdk-core-3.199.0/lib/aws-sdk-core/plugins/checksum_algorithm.rb:111:in `call'
/var/lib/gems/2.7.0/gems/aws-sdk-core-3.199.0/lib/seahorse/client/plugins/raise_response_errors.rb:17:in `call': No attributes (Aws::SimpleDB::Errors::MissingParameter)

Reproduction Steps

Well, it would require some connection details, but generally the line that causes the error haven't changed in years, and looks like:

  sdb.put_attributes({domain_name: 'dbas_metadata', item_name: 'find_available_hostname_lock',
                      attributes: [{name: 'locked_by', value: lock, replace: true}]})

Possible Solution

No response

Additional Information/Context

No response

Gem name ('aws-sdk', 'aws-sdk-resources' or service gems like 'aws-sdk-s3') and its version

aws-sdk-core 3.199.0

Environment details (Version of Ruby, OS environment)

ruby 2.7.0p0 (2019-12-25 revision 647ee6f091) [aarch64-linux-gnu]; Ubuntu focal

[depesz]

Ah, we also tried to upgrade to simpledb 1.49.0, but it didn't help

[mullermp]

Thanks for opening an issue. This is likely broken in core 3.192.0 (can you confirm?) where some sweeping protocol changes were made. SimpleDB is effectively deprecated - no new accounts can even see it in the console. I can consider a fix for it if we can diagnose it (I can't, because I don't even have access to the service) but the likely situation is that you should pin your version to 3.191.0 and move to dynamodb or some other data store. New aws-sdk-simpledb versions have only been published because of code generation changes and not any API changes. Can you enable http_wire_trace: true so we can investigate the failure?

[depesz]

Pinned the module and it helped. As for http_wire_trace, not sure where to put it, but since it's "effectively deprecated" we can start working on migrating away.

[mullermp]

Apologies, http_wire_trace: true is an option for the Client. We can compare the output on the working version of 3.191.0 and a non-working version 3.192.0 to see what changed, maybe Content-Type header or missing body members. The error Aws::SimpleDB::Errors::MissingParameter is a service generated error, so likely we are no longer sending something that it expects. I am sorry for the trouble - the protocol changes we made were the guidance given to us by the service frameworks team, and likely SimpleDB was .. forgotten about .. I do agree that moving to DynamoDB would be best if possible. I do believe there are migration guides out there and it shouldn't be too bad.

[mullermp]

Unless we find the root cause and it's an easy fix, I'll unfortunately mark this as "wontfix" given SimpleDB's posture. I'm very sorry that our change broke you. Please re-open with new information if you have any, and any questions you may have about using DynamoDB.

[github-actions]

This issue is now closed. Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.

[mullermp]

I was able to get an account to reproduce and I think I have a fix for you. #3056 It should be released maybe Monday or Tuesday. @depesz

Can you verify the fix? You can pull the latest core and then apply this monkey patch

module Aws::SimpleDB
  module ClientApi
    BatchDeleteAttributesRequest.add_member(:items, Shapes::ShapeRef.new(shape: DeletableItemList, required: true, location_name: "Item"))
    BatchPutAttributesRequest.add_member(:items, Shapes::ShapeRef.new(shape: ReplaceableItemList, required: true, location_name: "Item"))
    DeletableItem.add_member(:attributes, Shapes::ShapeRef.new(shape: AttributeList, location_name: "Attribute"))
    DeleteAttributesRequest.add_member(:attributes, Shapes::ShapeRef.new(shape: AttributeList, location_name: "Attribute"))
    GetAttributesRequest.add_member(:attribute_names, Shapes::ShapeRef.new(shape: AttributeNameList, location_name: "AttributeName"))
    GetAttributesResult.add_member(:attributes, Shapes::ShapeRef.new(shape: AttributeList, location_name: "Attribute"))
    Item.add_member(:attributes, Shapes::ShapeRef.new(shape: AttributeList, required: true, location_name: "Attribute"))
    ListDomainsResult.add_member(:domain_names, Shapes::ShapeRef.new(shape: DomainNameList, location_name: "DomainName"))
    PutAttributesRequest.add_member(:attributes, Shapes::ShapeRef.new(shape: ReplaceableAttributeList, required: true, location_name: "Attribute"))
    ReplaceableItem.add_member(:attributes, Shapes::ShapeRef.new(shape: ReplaceableAttributeList, required: true, location_name: "Attribute"))
    SelectResult.add_member(:items, Shapes::ShapeRef.new(shape: ItemList, location_name: "Item"))
  end
end

[depesz]

SUCCESS!

First did:

=$ sudo gem install aws-sdk-core aws-sdk-simpledb
Fetching aws-sdk-core-3.200.0.gem
Successfully installed aws-sdk-core-3.200.0
Fetching aws-sdk-simpledb-1.49.0.gem
Successfully installed aws-sdk-simpledb-1.49.0
2 gems installed

Then modified /var/lib/gems/2.7.0/gems/aws-sdk-simpledb-1.49.0/lib/aws-sdk-simpledb/client_api.rb with your changes, and now my script works without pinning to old core/simpledb.

Thanks a lot :)

[jterapin]

Fix went out with the latest most recent version of aws-sdk-simpledb: https://rubygems.org/gems/aws-sdk-simpledb