greg/hanoidb
1
0
Fork 0
Code Issues 10 Pull requests 3 Projects Releases Packages Wiki Activity
149 commits 4 branches 0 tags 2.4 MiB
Commit graph

149 commits

This branch
This branch
All branches
Author SHA1 Message Date
Gregory Burd
43f095b3f0 Rename "lsm-btree" to "hanoi". 2012-04-21 15:20:39 -04:00
Gregory Burd
fec14a1c51 Put the copyright/license header into a few overlooked files. 2012-04-19 18:09:01 -04:00
Gregory Burd
bd3e638a96 Merge branch 'master' into gsb-merge-krab-20120419 2012-04-19 18:03:19 -04:00
Gregory Burd
197914939c Removed redundant function. 2012-04-19 18:00:06 -04:00
Gregory Burd
23f6d76a72 Merge branch 'krab-incremental-merge' into gsb-merge-krab-20120419 
Conflicts:
	src/lsm_btree.erl
	src/lsm_btree.hrl
 2012-04-19 16:54:23 -04:00
Kresten Krab Thorup
6289602045 Improve incremental merge 
This change makes incremental merge be concurrent 
with filling up the nursery. So in stead of waiting
for an incremental merge to complete before returning
from insert, it

- blocks waiting for a possible previous incremental merge to complete
- issues a new incremental merge.

This improves put latencies, but not throughput.
 2012-04-19 22:33:27 +02:00
Kresten Krab Thorup
ee90944c62 Implement incremental insert 
This slows down insert to be log2(N), where N is
the total number of objects in the store.  The upside
is that it also removes the terrible worst case
scenarios for insert.
 2012-04-19 19:57:39 +02:00
Kresten Krab Thorup
3d80b164d5 Introduce 3rd file in each level to reduce worst-case 
Now, each level is comprised of 3 files,

   A=Oldest, B=Older, C=Old

As in [Overmars and Leeuwen, 1983]. As soon as we have A & B,
we initiate a merge, (to the M=New) file, i.e. we merge more
eagerly than previously.

Next step in this refactoring is to add a scheduler that enforces
some merge activity as part of a PUT.
 2012-04-19 16:07:11 +02:00
Gregory Burd
3e85a266a8 More things to do... 2012-04-18 17:40:21 -04:00
Gregory Burd
b14887f9da Merge branch 'master' of github.com:basho/lsm_btree 2012-04-18 17:12:50 -04:00
Gregory Burd
1da9a858d1 Keep track of todo items here 2012-04-18 17:12:40 -04:00
Gregory Burd
0a72ca1df9 The Log-Structured Merge-Tree (LSM-Tree) by Patrick O'Neil, Edward Cheng Dieter Gawlick, Elizabeth O'Neil 2012-04-18 17:06:24 -04:00
Gregory Burd
3f02eadc27 Too large a nursery opens up the potential for long (in seconds) merges. 2012-04-18 16:49:57 -04:00
Kresten Krab Thorup
4e53b0a083 Allow fold worker to send {fold_results, PID, KVs} 
Not just individual KVs, but lists of KVs
 2012-04-18 09:28:59 +02:00
Kresten Krab Thorup
b70d2af1da Add doc to lsm_btree.hrl 2012-04-18 09:26:54 +02:00
Kresten Krab Thorup
4b91c71781 Invoke local rebar 
Some of us doesn't have rebar on the general path
 2012-04-18 09:26:21 +02:00
Kresten Krab Thorup
c3f916c350 Make fold_worker link to the consumer 
This should ensure proper cleanup if a process
calling fold exits while folding.
 2012-04-18 09:25:47 +02:00
Kresten Krab Thorup
5facc3df18 Undo fancy-pancy sext key encoding 
Sadly, this didn't work
Have to investigate more…
 2012-04-16 21:51:01 -04:00
Kresten Krab Thorup
1cf4805da5 Utilize sext to optimize bucket range queries 
Also re-fix the range arguments which were broken
in a previous commit.
 2012-04-16 21:51:01 -04:00
Kresten Krab Thorup
e7a621e449 Handle exceptions inside sync_fold_range 
Someone tried (tsk, tsk) to terminate a fold operation
by throwing an exception.  Now we also should handle
such situations gracefully.
 2012-04-16 21:49:15 -04:00
Steve Vinoski
4dab3a65e5 remove throw statements from fold functions 2012-04-16 21:42:04 -04:00
Kresten Krab Thorup
454a111ad7 Handle exceptions inside sync_fold_range 
Someone tried (tsk, tsk) to terminate a fold operation
by throwing an exception.  Now we also should handle
such situations gracefully.
 2012-04-17 00:29:28 +02:00
Steve Vinoski
a911734134 lsm_btree:sync_fold_range/5 no longer exists, use /4 instead 2012-04-16 17:46:43 -04:00
Steve Vinoski
79872680da rename temp_riak_kv_backend to lsm_btree_temp_riak_kv_backend 
The module temp_riak_kv_backend is already used in another experimental
storage backend. Rename it to avoid collisions when trying to use both
backends in riak together.
 2012-04-16 17:25:51 -04:00
Steve Vinoski
eefada16ac restore lookup/2 function, for compatibility 2012-04-16 16:50:55 -04:00
Kresten Krab Thorup
5f9f5c18d6 Add #btree_range to all fold ops 
lets you specify this for fold operations

  #btree_range {
     from_key :: binary(),
     from_inclusive = true :: boolean(),
     to_key :: binary() | undefined,
     to_inclusive = false :: boolean(),
     limit = undefined :: pos_integer() | undefined
  }
 2012-04-16 16:46:57 -04:00
Kresten Krab Thorup
9a7959ff4c Simplify KEY_IN_RANGE macro 2012-04-16 16:45:25 -04:00
Gregory Burd
8409df447d Another notfound -> not_found fix 2012-04-16 07:31:14 -04:00
Gregory Burd
5c5934549f The nursery was far too small, increase it from 32 to 8192 objects. 2012-04-15 17:23:42 -04:00
Gregory Burd
e81a3480ab Fix silly oversight with variable names, add better data dir prep. 2012-04-15 16:56:19 -04:00
Gregory Burd
4eaa02ac3f Formatting 2012-04-15 16:55:12 -04:00
Gregory Burd
b4823d3e8f Formatting 2012-04-15 16:54:27 -04:00
Gregory Burd
61d360550e Use not_found uniformly (rather than notfound). 2012-04-15 15:34:42 -04:00
Gregory Burd
5338a07c54 Don't prefix vnode directories with the backend name 2012-04-15 14:36:05 -04:00
Gregory Burd
a04475b871 Add lsm_btree into the list of applications in riak_kv.app.src 2012-04-15 14:35:31 -04:00
Gregory Burd
b325f3e792 * Changed "lookup" to "get" just because 
* Added copyright notices to files
 * Added Apache 2.0 License file with permission from Kresten/Trifork
 * Changed the handle from "Db" to "Tree" because... it made me feel better
 * Other minor changes here and there
 2012-04-15 10:35:39 -04:00
Gregory Burd
e6e3b55d23 Minor renaming 2012-04-15 07:24:26 -04:00
Gregory Burd
95d18b6cd5 Minor name change 2012-04-15 07:23:44 -04:00
Gregory Burd
38457ebc91 Minor rename 2012-04-15 07:20:19 -04:00
Gregory Burd
e581370242 Minor rename 2012-04-15 07:19:50 -04:00
Gregory Burd
771522d0a9 Use proper naming 2012-04-15 07:13:10 -04:00
Steve Vinoski
4bc1eb6e19 add riak kv backend 2012-04-14 20:49:56 -04:00
Kresten Krab Thorup
f0def8231b Introduce btree_range record for range queries 
This allows specifying ranges with from/to
being inclusive or not, and providing a result
limit (latter not implemented yet).

This change just makes all current tests pass.
 2012-01-23 00:51:31 +01:00
Kresten Krab Thorup
afec223f03 Give range query test some more time... 2012-01-23 00:49:43 +01:00
Kresten Krab Thorup
42b353ecfd Implement sequential/random reader API 2012-01-23 00:49:07 +01:00
Kresten Krab Thorup
739fd03715 Merge pull request #9 from jlouis/qc-sync-fold-range 
Qc sync fold range
 2012-01-22 04:57:51 -08:00
Jesper Louis Andersen
363e94c3cf Implement sync_fold_range as part of the test. 2012-01-22 13:10:34 +01:00
Jesper Louis Andersen
e61f80f4e0 Do not clean up trees twice. Once is enough. 2012-01-22 13:10:33 +01:00
Kresten Krab Thorup
90ba15fca8 Merge pull request #8 from jlouis/qc-sync-range 
Qc sync range
 2012-01-21 13:42:17 -08:00
Jesper Louis Andersen
baa779ddaa Fix a bug in lsm_tree:close/1. 
There is a race condition based on the monitor set in a call. We might
get a normal exit from the monitor message deep inside gen_server.
This has to be handled. I've seen this race in my QC tests.
 2012-01-21 21:17:33 +01:00