greg/hanoidb
1
0
Fork 0
Code Issues 10 Pull requests 3 Projects Releases Packages Wiki Activity
HanoiDB implements an ordered k/v pairs storage engine in Erlang. The primary index is a log-structured merge tree (LSM-BTree) implemented using "doubling sizes" persistent ordered sets of kvp.
192 commits 4 branches 0 tags 2.4 MiB
Erlang 94.9%
Shell 3.5%
Nix 1.1%
Makefile 0.5%
Find a file
Kresten Krab Thorup 9a7e2131a1 Implement 2i 
Most code copied from eleveldb backend, except
we can do more precise range folds with hanoi
so no need to throw exceptions from fold functions.
2012-04-27 10:03:19 +02:00
doc sample 20 min run (MacBook Air/SSD) 2012-04-26 01:51:08 +02:00
include Introduce plain_rpc 2012-04-26 00:49:57 +02:00
src Implement 2i 2012-04-27 10:03:19 +02:00
test Fix unit tests 2012-04-27 09:59:09 +02:00
.gitignore Provide a Makefile for running dialyzer. 2012-01-06 21:22:23 +01:00
DESIGN.md Cleanup 2012-04-24 10:37:45 -04:00
enable-hanoi Rename "lsm-btree" to "hanoi". 2012-04-21 15:20:39 -04:00
LICENSE * Changed "lookup" to "get" just because 2012-04-15 10:35:39 -04:00
Makefile Rename "lsm-btree" to "hanoi". 2012-04-21 15:20:39 -04:00
README.md Update README 2012-04-26 17:12:37 +02:00
rebar Initial work-in-progress 2012-01-04 15:05:31 +01:00
rebar.config Implement 2i 2012-04-27 10:03:19 +02:00
TODO Minor change. 2012-04-25 14:07:06 -04:00
visualize-hanoi.sh New visualizer 2012-04-24 01:43:53 +02:00

README.md

Hanoi Ordered Key/Value Storage

Hanoi implements an ordered key/value storage engine, implemented using "doubling sizes" persistent ordered sets of key/value pairs, much like LevelDB.

Here's the bullet list:

  • Insert, Delete and Read all have worst case log2(N) latency.
  • Incremental space reclaimation: The cost of evicting stale key/values is amortized into insertion
    • you don't need a separate eviction thread to keep memory use low
    • you don't need to schedule merges to happen at off-peak hours
  • Operations-friendly "append-only" storage
    • allows you to backup live system
    • crash-recovery is very fast and the logic is straight forward
  • Supports efficient range queries
  • Uses bloom filters to avoid unnecessary lookups on disk
  • Efficient resource utilization
    • Doesn't store all keys in memory
    • Uses a modest number of file descriptors proportional to the number of levels
    • IO is generally balanced between random and sequential
    • Low CPU overhead
  • ~2000 lines of pure Erlang code in src/*.erl

Hanoi is developed by Trifork, a Riak expert solutions provider. You're most welcome to contact us if you want help optimizing your Riak setup.

How to deploy Hanoi as a Riak/KV backend

This storage engine can function as an alternative backend for Basho's Riak/KV.

You can deploy hanoi into a Riak devrel cluster using the enable-hanoi script. Clone the riak repo, change your working directory to it, and then execute the enable-hanoi script. It adds hanoi as a dependency, runs make all devrel, and then modifies the configuration settings of the resulting dev nodes to use the hanoi storage backend.

  1. git clone git://github.com/basho/riak.git
  2. cd riak/deps
  3. git clone git://github.com/basho/hanoi.git
  4. cd ..
  5. ./deps/hanoi/enable-hanoi