Skip to content
This repository has been archived by the owner on Oct 17, 2018. It is now read-only.

Use goal state based placement instead of snapshot based placement #149

Open
cw9 opened this issue Jul 24, 2018 · 0 comments
Open

Use goal state based placement instead of snapshot based placement #149

cw9 opened this issue Jul 24, 2018 · 0 comments

Comments

@cw9
Copy link
Contributor

cw9 commented Jul 24, 2018

Currently M3Aggregator uses snapshot based placement to manage its placements, so when users make topology changes, it will schedule a new placement in the future and append to current placement and placement consumer(like collectors) will cut over to the new placement at the right time.

This was designed to address the clock skew issue when there is a large number of collectors, but this data structure added complexity to both placement management and operations, we should look into changing the placement from the current model to the goal state based model, which only contains one placement snapshot and all users(Both M3collectors and M3aggregators) should gracefully/eventually migrate to the goal state. By leveraging shard level cutover and cutoff times, we could still accommodate the clock skew issue among collectors and simplify the logic around placement managements.

This could also enable us to merge the topology tooling between M3aggregator and M3DB, which would greatly simplify the daily operation around M3aggregator placement management.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant