Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(storers): Add a new NutsMemcached storage backend built on NutsDB #452

Draft
wants to merge 6 commits into
base: master
Choose a base branch
from

Conversation

vejipe
Copy link
Collaborator

@vejipe vejipe commented Feb 9, 2024

Add a new storage backend: NutsMemcached. It is composed of both NutsDB and Memcached (i.e., two layers storage).

Layer 1: Keys+Headers alone are stored in NutsDB in RAM to take advantage of its fast prefix search.

Layer 2: Keys+Values are stored in Memcached to take advantage of its advanced RAM+disk eviction system and hard limits on RAM and disk usage.

Note: since NutsDB has no preemptive eviction system, keys would remain until the end of their TTL, while key+values stored in Memcached could be evicted. Question: could we bring back HintBPTSparseIdxMode which was very efficient for caching many keys.

HintBPTSparseIdxMode represents b+ tree sparse index mode.

The mode HintBPTSparseIdxMode is based b+ tree sparse index, this mode saves memory very much (1 billion data only uses about 80MB of memory).

Note: prefix search is not (easily?) supported on Memcached alone.

References

@vejipe vejipe marked this pull request as draft February 9, 2024 12:48
@vejipe vejipe force-pushed the feat/storers/add-nuts-memcached branch 7 times, most recently from 9a3071d to 95cace4 Compare February 13, 2024 13:25
@vejipe vejipe force-pushed the feat/storers/add-nuts-memcached branch from 182d3aa to 7578024 Compare March 5, 2024 14:15
@vejipe vejipe force-pushed the feat/storers/add-nuts-memcached branch from 7578024 to 086ede5 Compare April 24, 2024 12:20
Copy link

netlify bot commented Apr 24, 2024

Deploy Preview for teal-sprinkles-4c7f14 canceled.

Name Link
🔨 Latest commit f380cc3
🔍 Latest deploy log https://app.netlify.com/sites/teal-sprinkles-4c7f14/deploys/6630f1b971a83e0007f09484

Vincent Jordan added 3 commits April 24, 2024 14:25
In this commit NutsMemcached behave just like Nuts.
In this commit cache values are moved to memcached and only keys are kept in Nuts.
@vejipe vejipe force-pushed the feat/storers/add-nuts-memcached branch from 3a033bd to 92c02be Compare April 29, 2024 12:35
Copy link

Copy link

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant