-
Notifications
You must be signed in to change notification settings - Fork 191
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
How to ingest data without duplication allowed? #1877
Comments
I find the cassandra's table definition :
Any way to custom this ? |
Not sure why you'd why exactly you'd want to customize that primary key, you can give it the data ID to be unique, and other things like sort and partition key come from the index (which again you could customize but probably don't want to). The issue is most likely that you are inserting rows into the index with the same adapter ID and data ID but different sort keys. This would happen, for example, if you were using a spatial index and the rows had different geometries (or similarly a temporal index with different date/times). In these rare cases you would want to delete the row prior to ingesting. The |
Thx for your reply, Where can i find the |
Do you have a "ROUND_ROBIN" partition strategy on your index (such as described in this add index help output, https://locationtech.github.io/geowave/latest/userguide.html#help-command)? This partition strategy would by design add random partition keys even to identical rows and explain this behavior you're seeing. |
With documentation, there is :
according to that, Adapter ID and Data ID define a unique identifier, so how to ingest data without duplication allowed?
now, my index looks like
Why this happened?
The values of
adapter_id
anddata_id
in these two records are the samei want to get a single record without a duplicated one, how can i do?
The text was updated successfully, but these errors were encountered: