You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Because of how explicit limits are applied, we've run into situations where a single find() call can spawn thousands of sequential queries internally. With a small enough limit, and with a filter that would ordinarily return no results, it effectively turns into a very slow scan of a partition. This is fairly unexpected behavior when specifying an explicit limit; furthermore, there's no way for TypeDorm to provide a limit directly to the DynamoDB query call without also triggering this potential runaway recursive query.
Suggestions:
There should be an option to specify whether the limit should be a DyanmoDB-only limit (IOW, the number results returned could be less than the limit provided, and guarantee only one query is made) or the TypeDorm notion of a limit (where it guarantees continually querying until either the limit is reached or the results are exhausted)
When using the TypeDorm notion of limit, recursive queries could potentially use exponentially increasing limits (or perhaps no limit at all) so that when a small limit is provided, it doesn't turn into a flood of sequential internal queries returning 0 results
Additionally, it may make sense to use a reasonable cap so that a single find(...) call will not spawn more than X queries internally, or look at more than X records (like a query/scanLimit). E.g. 2000+ queries from a single find(...) is probably never intended.
The text was updated successfully, but these errors were encountered:
An additional suggestion would be providing a RCU limit which describes the max read capacity units the find operation should use before stopping - regardless of reaching a provided limit. This may be better as it is more predictable in terms of cost.
Because of how explicit limits are applied, we've run into situations where a single
find()
call can spawn thousands of sequential queries internally. With a small enough limit, and with a filter that would ordinarily return no results, it effectively turns into a very slow scan of a partition. This is fairly unexpected behavior when specifying an explicitlimit
; furthermore, there's no way for TypeDorm to provide a limit directly to the DynamoDB query call without also triggering this potential runaway recursive query.Suggestions:
limit
, recursive queries could potentially use exponentially increasing limits (or perhaps no limit at all) so that when a small limit is provided, it doesn't turn into a flood of sequential internal queries returning 0 resultsfind(...)
call will not spawn more than X queries internally, or look at more than X records (like aquery/scanLimit
). E.g. 2000+ queries from a singlefind(...)
is probably never intended.The text was updated successfully, but these errors were encountered: