greg/hanoidb
1
0
Fork 0
Code Issues 10 Pull requests 3 Projects Releases Packages Wiki Activity
295 commits 4 branches 0 tags 2.4 MiB
Commit graph

295 commits

This branch
This branch
All branches
Author SHA1 Message Date
Gregory Burd
95082bd4d3 Formatting 2012-07-12 13:17:04 -04:00
Gregory Burd
0f3c649f80 Some naming changes and more specific types in read/write path. 2012-06-25 14:27:42 +05:30
Gregory Burd
3140f9f281 Cleanup 2012-06-25 11:55:07 +05:30
Gregory Burd
db243b9794 Don't bother writing out the bloom filter when a level is empty, instead just write 0 for the bloom size and before that record the level's filter size so as to re-create a properly sized filter later when reading this file. 2012-06-25 11:54:38 +05:30
Gregory Burd
59282eaf2a Calculate the header size 2012-06-25 11:52:02 +05:30
Gregory Burd
7ab09f5983 Cleanup 2012-06-25 07:27:00 +05:30
Gregory Burd
ba068a3175 Consider the nursery full when it is just below the threshold so we don't overflow the bloom filter. 2012-06-25 07:26:29 +05:30
Gregory Burd
5efb09a5f3 Formatting, whitespace. 2012-06-25 07:24:46 +05:30
Gregory Burd
c46b99956d WIP: more cleanup 2012-06-24 21:40:31 +05:30
Gregory Burd
848ea1e976 Use a more accurate bloom (larger) 2012-06-24 21:40:18 +05:30
Gregory Burd
244aeb0f0b Arguments were reversed. 2012-06-24 21:39:41 +05:30
Gregory Burd
b67d1f8449 WIP: sitll fixing things. 2012-06-24 17:16:42 +05:30
Gregory Burd
246e891d01 WIP: another bug 2012-06-23 16:06:49 +01:00
Gregory Burd
4bcd4b5de1 WIP: more bugs fixed, closer to working again... 2012-06-23 15:56:53 +01:00
Gregory Burd
102c518269 WIP: still chasing the bug... 2012-06-23 09:38:03 +01:00
Gregory Burd
a79dde264f WIP 2012-06-22 16:39:10 +01:00
Gregory Burd
d0245c85c0 Oops, I left a merge in. 2012-06-22 10:53:04 +01:00
Gregory Burd
9eabb78d06 WIP: still sorting out some bugs I've introduced (just for fun). 2012-06-22 10:48:22 +01:00
Gregory Burd
c9d8ffa8ea WIP 2012-06-21 23:12:32 +01:00
Gregory Burd
39d06fb29d Whitespace 2012-06-19 23:00:18 +01:00
Steve Vinoski
538518045c fix badmatch 2012-06-19 14:30:11 -04:00
Gregory Burd
fe73e4a48b WIP: working toward fixing the blocking fold test 2012-06-19 15:03:42 +01:00
Gregory Burd
32de0e4f7a Update to 1.1.0 2012-06-19 15:02:10 +01:00
Gregory Burd
0cb8d6a86b Make eunit a bit more noisy, add more structure and debug config. 2012-06-19 15:01:50 +01:00
Gregory Burd
8f8cda0776 Remove pices related to Riak, clean up other things. 2012-06-19 15:00:48 +01:00
Gregory Burd
30fd2ded4c Cleanup 2012-06-19 15:00:06 +01:00
Gregory Burd
bfdb63094f Remove Riak-specific integration code. Deliver it later as a separate repo. 2012-06-19 11:15:29 +01:00
Gregory Burd
fb01a7dd29 Improve tests. 2012-06-14 18:15:47 +01:00
Gregory Burd
95a133c7cc Centralize file format version tag. 2012-06-14 18:15:33 +01:00
Gregory Burd
fee95b9d62 Don't depend on lager 2012-06-14 18:14:51 +01:00
Gregory Burd
49604c77d9 Whitespace, comments, etc. 2012-06-14 18:14:33 +01:00
Gregory Burd
75df2b86bf Improve the compression code, add potential for LZ4 algorithm when ready. 2012-06-14 18:13:27 +01:00
Gregory Burd
db41532258 Allow keys to expire at different points in time as well as due to a global expiry time. 2012-06-14 18:11:41 +01:00
Gregory Burd
b6987c1e47 Whitespace 2012-06-14 18:04:39 +01:00
Gregory Burd
d3ce4ce73d Minor fixes and reorg. 2012-06-14 18:03:55 +01:00
Gregory Burd
9d3c99404a Use locally installed rebar. 2012-06-14 18:03:34 +01:00
Gregory Burd
af4d2ba4d8 Merge branch 'master' of github.com:basho/hanoidb 2012-06-11 23:41:49 +01:00
Gregory Burd
ee89f93009 Use a uniform method to calculate bytes that can manage lists as well. 2012-06-11 23:40:28 +01:00
Gregory Burd
f4eea4a594 Whitespace and comment cleanup. 2012-06-11 23:40:07 +01:00
Gregory Burd
acbcf9d601 Return properly shaped tuple. 2012-06-11 23:39:04 +01:00
Gregory Burd
0e98543a84 Don't depend on basho_bench. 2012-06-06 18:17:11 +02:00
Kresten Krab Thorup
669f589d0c Implement smaller incremental merge steps 
Right now, this is controlled by the macro
INC_MERGE_STEP in hanoidb_nursery; eventually
we should turn this into a configuration option.

Making this small, (minimum is 1), hurts average perf
but reduces the 99.9 percentile latency.
 2012-05-11 22:31:23 +02:00
Kresten Krab Thorup
1b42172cbe Fix bug when merge result is an empty file 
This happens when all entries are expired, or
if all entries would have been tombstones.
 2012-05-11 14:58:53 +02:00
Kresten Krab Thorup
61720065d9 Add some doc/spec 2012-05-11 12:33:33 +02:00
Kresten Krab Thorup
fbda7af576 Honor expiry_secs == 0 
When this is the case, we use the old on-disk
encoding to shave 4 bytes off every entry.
 2012-05-11 12:30:07 +02:00
Kresten Krab Thorup
181b1debb8 Update basho_bench driver to recognize config flags 
Also, add default values to template configuration
 2012-05-11 12:29:16 +02:00
Kresten Krab Thorup
b6955c9a75 Refactor for expiry_secs option 
Tree nodes now hold entries at the form

   {Key, ?TOMBSTONE 
       | BinValue
       | {?TOMBSTONE, TStamp}
       | {BinValue, TStamp}}

We use the form without TStamp when expiry_secs
is unset or set to 0 (i.e., values don't expire).

merger/writer: Move KV count into writer, because
now the writer:add determines if a value is expired
and thus wither a value is actually written.  Thus,
writer now has a new API function which returns the
KV count written so far.

reader: lookup/fold API hides the TStamp tuples,
so only the next_node API used by the merger
is exposed to these {Key, {_, TStamp}} entries.

nursery: like reader, the TStamp'ed tuples are
not exposed in the client API; expired values
are simply not returned from fold/lookup.

hanoidb: add config option {expiry_secs, N}.

other modules: Make sure that config is passed
all the way down through (sub) processes to be
able to utilize the config option everywhere.

test: update to work with new option.
 2012-05-11 12:00:32 +02:00
Kresten Krab Thorup
245d815e4c Add magic "HAN1" to btree file format 
BREAKING CHANGE!  This change provides for future
file format changes, but also breaks backwards
compatibility.

Also describe the file format in design_document
 2012-05-08 17:27:05 +02:00
Kresten Krab Thorup
f7681da1db Add some more current design diagrams 2012-05-08 00:02:08 +02:00
Kresten Krab Thorup
7622b4e4b8 Implement concurrent GETs 
With this change, GETs will flow concurrently
down through the level controllers, replying
directly to the caller via gen_server:reply.
Very actor-like :-)
 2012-05-08 00:00:10 +02:00