Distributed KV Store

TODOs

• Add Yml for configuration of ZK, https://github.com/go-yaml/yaml (though it may be redundant with zk-cluster.yml)
• change yml to json, for consistency
• Slave - service Registering.
• Slave - Put,Del,Get RPC APIs
• Slave - RPC registering
• Client - A REPL interface
• Master - Join/Leave/Query interface
• Client - Automatic primary/back selection in starting.
• Master - register logic
• Client - Put,Del,Get, CMDLINE-Interface, RPC call (after master done)
• Client - Del,Get Value not exist, Gracefully handle other failures, Update configuration when key-not-belong
• Master/Slave/Client - On configuration change, how to shift shards.
• Copy test from 6.824 / or write on my own
• Master/Slave/Client - handling network error, filter out duplicate RPCs
• Lock library (for slaves data access, not necessary)
• Slave backup & primary failed re-election logic
• Slave backup & primary normal logic
• Starting script for slave-master cluster

Features

• multiple backup nodes
• backup for master

Build & Run

zookeeper

$ docker-compose -f zk-cluster up

• deploy zkper using own script

Slave

$ go run go/slave_server.go [ip] [port] [hostname] [groupID] 
# ip and port expose as RPC service
# hostname is used to distinguish
# groupID  is for grouping.

Master

$ go run go/master_server.go [ip] [port] [hostname]

Client

$ go run go/client.go

Admin

$ go run go/admin.go [command] [groupIDs ...]
# command = "join-group" / "leave-group"
# groupIDs is an array of groupid to be configured

---

• Strange cases

  • when primary failed and become backup again. client's request should fail. (fixed)