Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(mito): make use of options in RegionCreate/OpenRequest #2436

Merged
merged 14 commits into from
Sep 19, 2023

Conversation

evenyag
Copy link
Contributor

@evenyag evenyag commented Sep 18, 2023

I hereby agree to the terms of the GreptimeDB CLA

What's changed and what's your intention?

This PR stores RegionOptions from RegionCreateRequest and RegionOpenRequest in Version. Now we can get the options users specify while creating/opening the region.

Since RegionCreateRequest and RegionOpenRequest use HashMap<String, String> to transfer options, this PR serializes the map to a JSON string and deserializes it to the RegionOptions struct. Due to the limitation of serde, we still need to special handle nested structs.

This avoid writing lots of get() and parse() like this:

impl From<&HashMap<String, String>> for CompactionStrategy {
fn from(opts: &HashMap<String, String>) -> Self {
let Some(strategy_name) = opts.get(COMPACTION_STRATEGY_KEY) else {
return CompactionStrategy::default();
};
if strategy_name.eq_ignore_ascii_case(COMPACTION_STRATEGY_TWCS_VALUE) {
let mut twcs_opts = TwcsOptions::default();
if let Some(max_active_window_files) = opts
.get(TWCS_MAX_ACTIVE_WINDOW_FILES_KEY)
.and_then(|num| num.parse::<usize>().ok())
{
twcs_opts.max_active_window_files = max_active_window_files;
}
if let Some(max_inactive_window_files) = opts
.get(TWCS_MAX_INACTIVE_WINDOW_FILES_KEY)
.and_then(|num| num.parse::<usize>().ok())
{
twcs_opts.max_inactive_window_files = max_inactive_window_files;
}
if let Some(time_window) = opts
.get(TWCS_TIME_WINDOW_SECONDS_KEY)
.and_then(|num| num.parse::<i64>().ok()) && time_window > 0
{
twcs_opts.time_window_seconds = Some(time_window);
}
CompactionStrategy::Twcs(twcs_opts)
} else {
// unrecognized compaction strategy
CompactionStrategy::default()
}
}
}

Refactor

  • Renames CompactionStrategy to CompactionOptions
  • Uses Duration to store compaction time window
  • Moves RegionOptions to options.rs
  • Removes engine_options

Checklist

  • I have written the necessary rustdoc comments.
  • I have added the necessary unit tests and integration tests.

Refer to a related PR or issue link (optional)

@evenyag evenyag marked this pull request as ready for review September 18, 2023 13:50
@codecov
Copy link

codecov bot commented Sep 18, 2023

Codecov Report

Merging #2436 (a49157f) into develop (342cc0a) will increase coverage by 0.04%.
Report is 4 commits behind head on develop.
The diff coverage is 97.61%.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #2436      +/-   ##
===========================================
+ Coverage    77.63%   77.68%   +0.04%     
===========================================
  Files          729      731       +2     
  Lines       115114   116089     +975     
===========================================
+ Hits         89374    90189     +815     
- Misses       25740    25900     +160     

Copy link
Collaborator

@fengjiachun fengjiachun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Member

@waynexia waynexia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

serde master

src/mito2/src/compaction.rs Outdated Show resolved Hide resolved
@evenyag evenyag added this pull request to the merge queue Sep 19, 2023
Merged via the queue into GreptimeTeam:develop with commit 7b606ed Sep 19, 2023
11 checks passed
@evenyag evenyag deleted the feat/mito2-region-options branch September 19, 2023 09:15
paomian pushed a commit to paomian/greptimedb that referenced this pull request Oct 19, 2023
…Team#2436)

* refactor: move RegionOptions to options mod

* refactor: define compaction strategy in region/options.rs

* feat: use duration for time window

* refactor: rename CompactionStrategy to CompactionOptions

* feat: use serde to parse options

* feat: parse options

* feat: set options on creation/opening

* test: test create/open with options

* chore: remove todo

* feat: get compaction ttl and options from RegionOptions

* style: fix clippy

* chore: Remove unused engine_options

* style: fix clippy

* chore: remove todo
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants