BingoDB is light-weight and linearly scalable pre-defined (for now) in-memory database. BingoDB provides callback function when TTL expires.
DynamoDB is fast but it does not provide callback feature, and other SQL-based services are not fast enough (since accessing disk is slow relative to accessing memory). We specifically wanted to have two features: a database notifies after a event expires in certain period of time, and be able to search with certain index name with high performance. However, we could not find a solid service that satisfied our needs. So we decided to create our own database which provides two key features.
Install go 1.9 from official website. We only supports 1.9 and higher version of go. Get package manager dep by using command
go get github.com/golang/dep
and run the following from the project root directory to install dependencies:
dep ensure
To use BingoDB, you first need to create a {filename}.yml
and define of your tables (see example.yml below). Once you've done this, you can simply run a server with following command:
go run cmd/bingodb/bingodb.go -config /path/to/config.yml -addr address:port`
#example.yml
tables:
#your table name
onlines:
fields:
#define your fields
#we support string/integer only for now
id: 'string'
guestKey: 'string'
channelId: 'string'
expiresAt: 'integer'
lastSeen: 'integer'
expireKey: 'expiresAt'
hashKey: 'channelId'
sortKey: 'id'
subIndices:
#name of index you want to search for a particular case
guest:
#these fields' value suppose to be in fields
hashKey: 'channelId'
sortKey: 'lastSeen'
metrics:
#metrics table is optional. If you define metric with ttl and interval with milliseconds,
#bingoDB records the table state.
ttl: 600000
interval: 2000
- 존재하는 모든 테이블의 정보를 주는 API
- 해당 table 에서 hashKey가 hash, sortKey가 sort 인 item을 찾는 API
- 해당 table 에서 hashKey가 hash, sortKey가 sort 인 item을 지우는 API
- 새로운 document를 추가하는 API
- $setOnInsert는 해당 document가 디비에 없어 새로 추가되는 경우에만 값을 set하게 됨
- Response에는 디비에 이전 값이 있다면 이전 값과 새로 추가된 값, 교체된 여부를 알 수 있음
- Request example
{
"$set": {
"id": "soc1",
"personKey": "user1",
"updatedAt": 1505200000000,
"expiresAt": 1513008702000
},
"$setOnInsert": {
"createdAt": 1505200000000
}
}
- 해당 table 에 대한 정보를 주는 API
- 해쉬 값에 해당하는 아이템들을 list로 얻는 API
- since 값을 포함해 그 이후 데이터를 조회함(backward 값이 1일 경우 그 이전)
- 최대 limit 개수 만큼 조회
- index 이름을 가진 서브 인덱스에 대해 hashKey가 hash, sortKey가 sort 인 아이템을 찾는 API
GET
/tables/:table/indices/:index/scan?hash=[hash]&limit=[limit]&since=[since1]&since=[since2]&since=[since3]&backward=[backward]
- index 이름을 가진 서브 인덱스에 대해 해당하는 아이템들을 list로 얻는 API
- since 값을 포함해 그 이후 데이터를 조회함(backward 값이 1일 경우 그 이전)
- since1: subIndex sort key, since2: primary hash key, since3: primary sort key
- 최대 limit 개수 만큼 조회
- put: O(lg(n))
- lookup: O(lg(n))
- delete: O(lg(n))
- fetch with since, limit(m): O(lg(n)*m)
- Support distributed computing
- Support Fast concurrent lock-free binary search tree
- In-memory database and do not support persistency (completed)
- One bingoDB contains multiple tables (completed)
- A table is structure to store data which contains key/value pair (completed)
- Support JSON format for value type (completed)
- Support dynamic table configuration like NoSQL
- Support pre-define table configuration with config file (completed)
- Provide time to live feature on a value, and the value suppose to be deleted after expired
- Support duplicated TTL value for keys (completed)
- Expired data transits to Message Queue (AMQP, RabbitMQ, etc)
- One table can have multiple secondary indexes (completed)
- TBD