feat: add deltaOps set metadata operation #2474
Closed
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Allow for the explicit changing of the metadata of a delta table. This allows for simple schema migrations like changing the metadata of a column or adding new nullable columns. The code doesn't currently do any checks that the table would still be readable after changing the metadata. The setMetadata operation is similar to mergeSchema but doesn't require a write at the same time so it can be run and tested as part of a deployment instead of on the next write of data.
Note: you used to be able to do this by recalling DeltaOps::create with overwrite on an existing table but since that was recently fixed to delete old data this allows for recreating that original behavior.