greg/hanoidb
1
0
Fork 0
Code Issues 10 Pull requests 3 Projects Releases Packages Wiki Activity
HanoiDB implements an ordered k/v pairs storage engine in Erlang. The primary index is a log-structured merge tree (LSM-BTree) implemented using "doubling sizes" persistent ordered sets of kvp.
166 commits 4 branches 0 tags 2.4 MiB
Erlang 94.9%
Shell 3.5%
Nix 1.1%
Makefile 0.5%
Find a file
Kresten Krab Thorup 755788ecfb Fix for hanoi store recovery 
If we're opening a hanoi store configured with
smaller nursery size than the default, then
we need to make sure that we also open the
small levels.

Future feature is to actually squash the
smaller levels.
2012-04-23 14:14:12 +02:00
doc Rename "lsm-btree" to "hanoi". 2012-04-21 15:20:39 -04:00
include Rename "lsm-btree" to "hanoi". 2012-04-21 15:20:39 -04:00
src Fix for hanoi store recovery 2012-04-23 14:14:12 +02:00
test Rename "lsm-btree" to "hanoi". 2012-04-21 15:20:39 -04:00
.gitignore Provide a Makefile for running dialyzer. 2012-01-06 21:22:23 +01:00
DESIGN.md Implementation notes belong in "DESIGN.md", not "README.md". 2012-04-22 09:27:17 -04:00
enable-hanoi Rename "lsm-btree" to "hanoi". 2012-04-21 15:20:39 -04:00
LICENSE * Changed "lookup" to "get" just because 2012-04-15 10:35:39 -04:00
Makefile Rename "lsm-btree" to "hanoi". 2012-04-21 15:20:39 -04:00
README.md Implementation notes belong in "DESIGN.md", not "README.md". 2012-04-22 09:27:17 -04:00
rebar Initial work-in-progress 2012-01-04 15:05:31 +01:00
rebar.config Implement compression + block size 2012-04-23 03:49:08 +02:00
TODO Implement compression + block size 2012-04-23 03:49:08 +02:00
visualize-hanoi.sh Switch order ABCX -> CBAX in visualizer 2012-04-22 23:53:31 +02:00

README.md

Hanoi Key/Value Storage Engine

This Erlang-based storage engine implements a structure somewhat like LSM-trees (Log-Structured Merge Trees, see docs/10.1.1.44.2782.pdf). The notes below describe how this storage engine work; I have not done extensive studies as how it differs from other storage mechanisms, but a brief brows through available online resources on LSM-trees indicates that this storage engine is quite different in several respects.

The storage engine can function as an alternative backend for Basho's Riak/KV.

Here's the bullet list:

  • Very fast writes and deletes,
  • Reasonably fast reads (N records are stored in log2(N) B-trees),
  • Operations-friendly "append-only" storage (allows you to backup live system, and crash-recovery is very simple)
  • The cost of evicting stale key/values is amortized into insertion, so you don't need to schedule merge to happen at off-peak hours.
  • Supports range queries (and thus eventually Riak 2i.)
  • Doesn't need a boat load of RAM
  • All in 1000 lines of pure Erlang code

Once we're a bit more stable, we'll provide a Riak backend.

Deploying the hanoi for testing with Riak/KV

You can deploy hanoi into a Riak devrel cluster using the enable-hanoi script. Clone the riak repo, change your working directory to it, and then execute the enable-hanoi script. It adds hanoi as a dependency, runs make all devrel, and then modifies the configuration settings of the resulting dev nodes to use the hanoi storage backend.

  1. git clone git://github.com/basho/riak.git
  2. cd riak/deps
  3. git clone git://github.com/basho/hanoi.git
  4. cd ..
  5. ./deps/hanoi/enable-hanoi # which does make all devrel