Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement initial-default #737

Open
Fokko opened this issue Nov 28, 2024 · 0 comments
Open

Implement initial-default #737

Fokko opened this issue Nov 28, 2024 · 0 comments

Comments

@Fokko
Copy link
Contributor

Fokko commented Nov 28, 2024

Default values are part of the Iceberg specification V3: https://github.com/apache/iceberg/blob/main/format/spec.md#default-values

But they are handy for reading the Iceberg metadata as well. If you bump a table from V1 to V2, history is not rewritten. This is being done lazily because Iceberg tables can exceed petabytes. Therefore, if you read a V2 table, it might be V1 metadata. In this case, some fields (eg sequence numbers), need to be read with default-values:

image

Implementing initial-values is something that's part of V3, and also simplifies reading older Iceberg metadata as a newer version 🥳

@Fokko Fokko mentioned this issue Nov 28, 2024
28 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant