Commit graph

206 commits

Author SHA1 Message Date
Kresten Krab Thorup
afec223f03 Give range query test some more time... 2012-01-23 00:49:43 +01:00
Kresten Krab Thorup
42b353ecfd Implement sequential/random reader API 2012-01-23 00:49:07 +01:00
Kresten Krab Thorup
739fd03715 Merge pull request #9 from jlouis/qc-sync-fold-range
Qc sync fold range
2012-01-22 04:57:51 -08:00
Jesper Louis Andersen
363e94c3cf Implement sync_fold_range as part of the test. 2012-01-22 13:10:34 +01:00
Jesper Louis Andersen
e61f80f4e0 Do not clean up trees twice. Once is enough. 2012-01-22 13:10:33 +01:00
Kresten Krab Thorup
90ba15fca8 Merge pull request #8 from jlouis/qc-sync-range
Qc sync range
2012-01-21 13:42:17 -08:00
Jesper Louis Andersen
baa779ddaa Fix a bug in lsm_tree:close/1.
There is a race condition based on the monitor set in a call. We might
get a normal exit from the monitor message deep inside gen_server.
This has to be handled. I've seen this race in my QC tests.
2012-01-21 21:17:33 +01:00
Jesper Louis Andersen
bb17564d9c Make the sync_range call test work.
Currently, we are also synchronizing all the key collection in the
call. But in the future we might want to interleave that with other
operations.
2012-01-21 21:16:24 +01:00
Jesper Louis Andersen
e00378639b Build a record for the trees.
This is a needed step to support range-queries since we may collect
range queries at a later time, interleaved with the normal operations.
2012-01-21 15:15:19 +01:00
Jesper Louis Andersen
6293ada2f4 Provide a target for an eunit_console. 2012-01-21 15:13:22 +01:00
Kresten Krab Thorup
fc024e95b6 Make proper range fold in nursery 2012-01-20 14:08:07 +01:00
Kresten Krab Thorup
ec2fe4ce8c Make close/1 resilient to noproc
Getting noproc exceptions here is ok, we simply
ignore such errors.
2012-01-20 10:14:47 +01:00
Kresten Krab Thorup
30a0bd4b01 Use ?BTREE_SIZE macro everywhere 2012-01-20 10:09:54 +01:00
Kresten Krab Thorup
c26e0695c5 Finish renaming lsm_btree_merger2 2012-01-20 10:06:43 +01:00
Kresten Krab Thorup
07b6b17534 Rename merger2 -> merger 2012-01-20 10:05:08 +01:00
Kresten Krab Thorup
1ad7bb2158 Remove unused merger 2012-01-20 10:04:10 +01:00
Kresten Krab Thorup
30ad1f0794 Use ?TOMBSTONE macro everywhere 2012-01-19 15:19:22 +01:00
Kresten Krab Thorup
f56f530d7a Add both sync and async range fold
Sync blocks insert/lookup while doing a range
query, but still buffers the results in a
per-range query process.  

Async fold runs on a hard link copy of the
underlying tree data.

This commit and also fixes a number of bugs 
related to folding; it was not taking nursery 
data into account.
2012-01-19 14:25:47 +01:00
Kresten Krab Thorup
ead8d3a41d Make lsm_btree:close/1 stop more processes
Closing a tree did not stop ongoing merge
processes beyond the current top level.
Now close synchronously calls down through all
levels and closes each one.
2012-01-19 14:19:16 +01:00
Kresten Krab Thorup
6e9ef1ee47 Update readme 2012-01-19 14:15:57 +01:00
Kresten Krab Thorup
3ddfd97aea Close trees before cleanup
Problem is that we sometimes get errors resulting
from files being written as a result of pending
merges.  Closing a tree should stop all such
file-level activity.
2012-01-19 14:15:46 +01:00
Kresten Krab Thorup
29d1493415 Rename lsm_btree:range/3 to lsm_btree:async_range/3 2012-01-16 15:13:47 +01:00
Erik Søe Sørensen
49c8d5b06f Clarify level handling in writer. 2012-01-16 07:50:31 +08:00
Erik Søe Sørensen
fa39f85a00 README: Explain 'LSM' and provide link to paper. 2012-01-16 07:49:58 +08:00
Kresten Krab Thorup
771d18f9f7 Implement lsm_btree:fold_range/5
First implementation of range fold

Range folding doesn't prohibit insert/lookup or
merge operations, but each level can only have
one range fold operation active.

Thus, worst case active range folds can double
space requirements, because it holds hard-linked
copies of used btree files.
2012-01-16 00:37:52 +01:00
Kresten Krab Thorup
2923ef8eff Bugfix merged in basho/ebloom
so we can go back to depending on basho/ebloom
2012-01-11 00:14:45 -04:00
Kresten Krab Thorup
f2629c3fd2 Improve nursery handling
- Code moved to separate module
- Recovery now operational; re-opening a
  tree will actually read the nursery log.

We use a sequential log file for the nursery
and also keep inserted {K,V} in memory
only 32 K/V pairs at a time.

NURSERY_SIZE = (1 bsl TOP_LEVEL)
configured in src/lsm_btree.htl
2012-01-10 18:04:13 -04:00
Kresten Krab Thorup
3945abe45a Merge branch 'statem-failing-lookups' of https://github.com/jlouis/lsm_btree 2012-01-08 10:39:26 +01:00
Jesper Louis Andersen
793f8e3a3a Optimize open dict check.
A dict is open and with keys if its size is greater than 0. Go through
the dicts one by one with lists:any/2 and use that to optimize the
check. Ought to run faster than traversing the full set.
2012-01-08 00:01:18 +01:00
Jesper Louis Andersen
60af669400 Implement failing key lookup.
We generate a set of names that are highly unlikely to be in the tree
as well because this makes it way easier for the ?SUCHTHAT macro to
succeed.
2012-01-07 23:35:27 +01:00
Jesper Louis Andersen
c8b9a6183f Work on failing keys. Optimize generator.
To make failing keys work we must be able to produce a key in fewer
failures then currently. So encode the picker for non-empty trees
directly so it is much faster and doesn't try to do it by generating a
case and then asking "is this one good enough?". A simple
dict:filter/2 does it.
2012-01-07 22:58:38 +01:00
Jesper Louis Andersen
6b8683bb08 Implement failing lookups in the driver. 2012-01-07 22:57:01 +01:00
Kresten Krab Thorup
fab3bf8427 Minor typo 2012-01-07 18:49:20 +01:00
Kresten Krab Thorup
eae41cec83 Update REAME 2012-01-07 17:54:35 +01:00
Kresten Krab Thorup
15736dd82a Rename modules fractal_btree -> lsm_btree 2012-01-07 17:17:48 +01:00
Kresten Krab Thorup
db2399ee4a Rename fractal_btree -> lsm_btree, phase I 2012-01-07 17:14:52 +01:00
Kresten Krab Thorup
1feef35d33 Add option to test with 'TRIQ' or 'PROPER' 2012-01-07 15:44:31 +01:00
Kresten Krab Thorup
03350aa096 Add close to shame tests 2012-01-07 15:16:57 +01:00
Kresten Krab Thorup
99a6985eed Allow put(Key, Binary|'deleted')
Last missing piece to make delete work it seems
2012-01-07 14:49:25 +01:00
Kresten Krab Thorup
5ca4443f04 Remove warning 2012-01-07 14:48:10 +01:00
Kresten Krab Thorup
11d135c3ed Merge branch 'statem-test' of https://github.com/jlouis/fractal_btree 2012-01-07 14:46:22 +01:00
Jesper Louis Andersen
84d4b56779 test_tree_simple_2: correctly close down the fractal Btree. 2012-01-07 14:21:38 +01:00
Kresten Krab Thorup
07c4df29ec Fix unit test
We currently have no check for concurrently
opening the same fractal_btree; we probably need
to implement a simple locking mechanism.
2012-01-07 13:56:14 +01:00
Kresten Krab Thorup
0064b0405e Refix readme 2012-01-07 13:55:03 +01:00
Jesper Louis Andersen
b99a206494 Restructure file, add deletion.
A refactoring run made the code more digestible. Also add the needed
clauses for handling random deletion.
2012-01-07 12:44:07 +01:00
Jesper Louis Andersen
c80d7d1a6a Add deletion support to the driver. 2012-01-07 12:44:07 +01:00
Jesper Louis Andersen
e10208462b Make it much less likely to open up a new btree. 2012-01-07 12:44:07 +01:00
Kresten Krab Thorup
aa0fa64928 Merge branch 'statem-test' of https://github.com/jlouis/fractal_btree 2012-01-07 01:19:29 +01:00
Jesper Louis Andersen
fcff982050 Utilize the new lookup of existing keys in PropEr.
This patch implements another possible command: Lookup of keys that
already exist in the tree somewhere. We use a dict as the test tool:
the postcondition verifies that the value obtained from the fractal
btree matches the one we put in the dict earlier.
2012-01-07 01:02:09 +01:00
Jesper Louis Andersen
fd87d7963a Improve driver cleanup, implement lookup of existing keys.
If we decide to stop the driver, we should not reply back before we
have a clean state with all BTrees closed.

Also, implement lookup of existing keys in the tree.
2012-01-07 01:00:31 +01:00