greg/hanoidb
1
0
Fork 0
Code Issues 10 Pull requests 3 Projects Releases Packages Wiki Activity
HanoiDB implements an ordered k/v pairs storage engine in Erlang. The primary index is a log-structured merge tree (LSM-BTree) implemented using "doubling sizes" persistent ordered sets of kvp.
158 commits 4 branches 0 tags 2.4 MiB
Erlang 94.9%
Shell 3.5%
Nix 1.1%
Makefile 0.5%
Find a file
Kresten Krab Thorup 3b451d5863 Fix folding 
The "blocking range fold" only works for modest
data sets, otherwise it gets prohibitively slow,
so for now we always do "snapshot range fold".
2012-04-23 02:10:18 +02:00
doc Rename "lsm-btree" to "hanoi". 2012-04-21 15:20:39 -04:00
include Rename "lsm-btree" to "hanoi". 2012-04-21 15:20:39 -04:00
src Fix folding 2012-04-23 02:10:18 +02:00
test Rename "lsm-btree" to "hanoi". 2012-04-21 15:20:39 -04:00
.gitignore Provide a Makefile for running dialyzer. 2012-01-06 21:22:23 +01:00
DESIGN.md Implementation notes belong in "DESIGN.md", not "README.md". 2012-04-22 09:27:17 -04:00
enable-hanoi Rename "lsm-btree" to "hanoi". 2012-04-21 15:20:39 -04:00
LICENSE * Changed "lookup" to "get" just because 2012-04-15 10:35:39 -04:00
Makefile Rename "lsm-btree" to "hanoi". 2012-04-21 15:20:39 -04:00
README.md Implementation notes belong in "DESIGN.md", not "README.md". 2012-04-22 09:27:17 -04:00
rebar Initial work-in-progress 2012-01-04 15:05:31 +01:00
rebar.config add riak kv backend 2012-04-14 20:49:56 -04:00
TODO Update todo items. 2012-04-21 17:24:37 -04:00
visualize-hanoi.sh Switch order ABCX -> CBAX in visualizer 2012-04-22 23:53:31 +02:00

README.md

Hanoi Key/Value Storage Engine

This Erlang-based storage engine implements a structure somewhat like LSM-trees (Log-Structured Merge Trees, see docs/10.1.1.44.2782.pdf). The notes below describe how this storage engine work; I have not done extensive studies as how it differs from other storage mechanisms, but a brief brows through available online resources on LSM-trees indicates that this storage engine is quite different in several respects.

The storage engine can function as an alternative backend for Basho's Riak/KV.

Here's the bullet list:

  • Very fast writes and deletes,
  • Reasonably fast reads (N records are stored in log2(N) B-trees),
  • Operations-friendly "append-only" storage (allows you to backup live system, and crash-recovery is very simple)
  • The cost of evicting stale key/values is amortized into insertion, so you don't need to schedule merge to happen at off-peak hours.
  • Supports range queries (and thus eventually Riak 2i.)
  • Doesn't need a boat load of RAM
  • All in 1000 lines of pure Erlang code

Once we're a bit more stable, we'll provide a Riak backend.

Deploying the hanoi for testing with Riak/KV

You can deploy hanoi into a Riak devrel cluster using the enable-hanoi script. Clone the riak repo, change your working directory to it, and then execute the enable-hanoi script. It adds hanoi as a dependency, runs make all devrel, and then modifies the configuration settings of the resulting dev nodes to use the hanoi storage backend.

  1. git clone git://github.com/basho/riak.git
  2. cd riak/deps
  3. git clone git://github.com/basho/hanoi.git
  4. cd ..
  5. ./deps/hanoi/enable-hanoi # which does make all devrel