Add expired_sstables.py tool #12
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
usage: expired_sstables.py [-h] --table TABLE --gc-grace-seconds GC_GRACE_SECONDS --default-ttl DEFAULT_TTL [--ignore-max-deletion-time]
[--safely-move-expired-sstables-to SAFELY_MOVE_EXPIRED_SSTABLES_TO]
optional arguments:
-h, --help show this help message and exit
--table TABLE Absolute path to table dir
--gc-grace-seconds GC_GRACE_SECONDS
GC grace period in seconds
--default-ttl DEFAULT_TTL
Default Time to Live in seconds
--ignore-max-deletion-time
Ignore max deletion time of never-expiring SSTable, and assume its TTL == default TTL
--safely-move-expired-sstables-to SAFELY_MOVE_EXPIRED_SSTABLES_TO
Specify path to backup dir, at which fully expired SSTables will be safely moved to. WARNING: please guarantee Scylla is not running for safety
reasons!
This tool allows the user to know:
1) The amount of disk space that can be eventually reclaimed from all expired SSTables.
2) The amount of disk space that can theoretically be reclaimed now from expired SSTables with no blockers.
An expired SSTable with no blocker is one that can be purged now without any chance of resurrecting data,
because there's no blocker, non-expired SSTables out there which data is shadowed by that expired SSTable.
The tool itself performs ALL the data shadowing checks before deciding that an expired SSTable has no blocker,
exactly like ScyllaDB itself does.
Additionally, the tool is able to calculate from scratch the max deletion time of SSTables. This feature
is useful for SSTables that were affected by the bug which reset default TTL in schemas. So it's able to
determine the SSTables that are essentially expired, even though the metadata doesn't tell us so.
Finally, the tool is able to reclaim the disk space used by all the expired SSTables with no blockers,
meaning that getting rid of them don't lead to data resurrection. This feature must be used ONLY when
the node is offline. The expired SSTables are moved to a backup directory specified by the user.
Tool output example:
https://gist.githubusercontent.com/raphaelsc/1616db26c033c1895df70cf468769aa2/raw/3bddce6113a8887759af6c55c374e9d842aa9033/expired%2520sstables%2520output
Signed-off-by: Raphael S. Carvalho <[email protected]>
denesb
reviewed
Jul 27, 2020
There was a problem hiding this comment.
I have not reviewed the correctness of the algorithm.
My only comment is that now that the script is in sstable-tools, you can reuse sstable_tools.sstablelib as well as the existing component parsers from the same modules. sstablelib already has parsers for a lot of existing disk types from sstables::, maybe worth reusing those.
| import psutil | ||
|
|
||
| # WARNING: don't tweak the value below whatsoever as it's a CONSTANT. | ||
| deletion_time_for_never_expiring_sstable = 2147483647 # timestamp in seconds |
There was a problem hiding this comment.
In python, constants usually have all caps names.
There was a problem hiding this comment.
I will change, thanks.
|
|
||
| return murmur3_token.get_token(first), murmur3_token.get_token(last) | ||
|
|
||
| def init_stats_metadata(self, filename, sstable_format): |
There was a problem hiding this comment.
You can use sstable_tools.statistics to parse the statistics file.
There was a problem hiding this comment.
@denesb I wish I could make this file self-contained, but it's better to reuse sstable_tools.statistics of course ;-)
avikivity
reviewed
Jul 27, 2020
| # WARNING: don't tweak the value below whatsoever as it's a CONSTANT. | ||
| deletion_time_for_never_expiring_sstable = 2147483647 # timestamp in seconds | ||
|
|
||
| class inclusive_range: |
There was a problem hiding this comment.
Better to use Interval from the intervaltree module.
There was a problem hiding this comment.
will change in next version. thanks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This tool allows the user to know:
Additionally, the tool is able to calculate from scratch the max deletion time of SSTables. This feature
is useful for SSTables that were affected by the bug which reset default TTL in schemas. So it's able to
determine the SSTables that are essentially expired, even though the metadata doesn't tell us so.
Finally, the tool is able to reclaim the disk space used by all the expired SSTables with no blockers,
meaning that getting rid of them don't lead to data resurrection. This feature must be used ONLY when
the node is offline. The expired SSTables are moved to a backup directory specified by the user.
Tool output example:
https://gist.githubusercontent.com/raphaelsc/1616db26c033c1895df70cf468769aa2/raw/3bddce6113a8887759af6c55c374e9d842aa9033/expired%2520sstables%2520output
Signed-off-by: Raphael S. Carvalho [email protected]