greg/hanoidb
1
0
Fork 0
Code Issues 10 Pull requests 3 Projects Releases Packages Wiki Activity
348 commits 4 branches 0 tags 2.4 MiB
Commit graph

348 commits

This branch
This branch
All branches
Author SHA1 Message Date
Gregory Burd
9d145ecdb1 Minor changes. 2012-07-17 16:57:45 -04:00
Gregory Burd
4e9ddbe65c Minor change 2012-07-17 16:54:51 -04:00
Gregory Burd
ce8982c79c Finished the cleanup and integration of an erlang bloom implementation. 2012-07-12 16:54:15 -04:00
Gregory Burd
95082bd4d3 Formatting 2012-07-12 13:17:04 -04:00
Gregory Burd
0f3c649f80 Some naming changes and more specific types in read/write path. 2012-06-25 14:27:42 +05:30
Gregory Burd
3140f9f281 Cleanup 2012-06-25 11:55:07 +05:30
Gregory Burd
db243b9794 Don't bother writing out the bloom filter when a level is empty, instead just write 0 for the bloom size and before that record the level's filter size so as to re-create a properly sized filter later when reading this file. 2012-06-25 11:54:38 +05:30
Gregory Burd
59282eaf2a Calculate the header size 2012-06-25 11:52:02 +05:30
Gregory Burd
7ab09f5983 Cleanup 2012-06-25 07:27:00 +05:30
Gregory Burd
ba068a3175 Consider the nursery full when it is just below the threshold so we don't overflow the bloom filter. 2012-06-25 07:26:29 +05:30
Gregory Burd
5efb09a5f3 Formatting, whitespace. 2012-06-25 07:24:46 +05:30
Gregory Burd
c46b99956d WIP: more cleanup 2012-06-24 21:40:31 +05:30
Gregory Burd
848ea1e976 Use a more accurate bloom (larger) 2012-06-24 21:40:18 +05:30
Gregory Burd
244aeb0f0b Arguments were reversed. 2012-06-24 21:39:41 +05:30
Gregory Burd
b67d1f8449 WIP: sitll fixing things. 2012-06-24 17:16:42 +05:30
Gregory Burd
246e891d01 WIP: another bug 2012-06-23 16:06:49 +01:00
Gregory Burd
4bcd4b5de1 WIP: more bugs fixed, closer to working again... 2012-06-23 15:56:53 +01:00
Gregory Burd
102c518269 WIP: still chasing the bug... 2012-06-23 09:38:03 +01:00
Gregory Burd
a79dde264f WIP 2012-06-22 16:39:10 +01:00
Gregory Burd
d0245c85c0 Oops, I left a merge in. 2012-06-22 10:53:04 +01:00
Gregory Burd
9eabb78d06 WIP: still sorting out some bugs I've introduced (just for fun). 2012-06-22 10:48:22 +01:00
Gregory Burd
c9d8ffa8ea WIP 2012-06-21 23:12:32 +01:00
Gregory Burd
39d06fb29d Whitespace 2012-06-19 23:00:18 +01:00
Steve Vinoski
538518045c fix badmatch 2012-06-19 14:30:11 -04:00
Gregory Burd
fe73e4a48b WIP: working toward fixing the blocking fold test 2012-06-19 15:03:42 +01:00
Gregory Burd
32de0e4f7a Update to 1.1.0 2012-06-19 15:02:10 +01:00
Gregory Burd
0cb8d6a86b Make eunit a bit more noisy, add more structure and debug config. 2012-06-19 15:01:50 +01:00
Gregory Burd
8f8cda0776 Remove pices related to Riak, clean up other things. 2012-06-19 15:00:48 +01:00
Gregory Burd
30fd2ded4c Cleanup 2012-06-19 15:00:06 +01:00
Gregory Burd
bfdb63094f Remove Riak-specific integration code. Deliver it later as a separate repo. 2012-06-19 11:15:29 +01:00
Gregory Burd
fb01a7dd29 Improve tests. 2012-06-14 18:15:47 +01:00
Gregory Burd
95a133c7cc Centralize file format version tag. 2012-06-14 18:15:33 +01:00
Gregory Burd
fee95b9d62 Don't depend on lager 2012-06-14 18:14:51 +01:00
Gregory Burd
49604c77d9 Whitespace, comments, etc. 2012-06-14 18:14:33 +01:00
Gregory Burd
75df2b86bf Improve the compression code, add potential for LZ4 algorithm when ready. 2012-06-14 18:13:27 +01:00
Gregory Burd
db41532258 Allow keys to expire at different points in time as well as due to a global expiry time. 2012-06-14 18:11:41 +01:00
Gregory Burd
b6987c1e47 Whitespace 2012-06-14 18:04:39 +01:00
Gregory Burd
d3ce4ce73d Minor fixes and reorg. 2012-06-14 18:03:55 +01:00
Gregory Burd
9d3c99404a Use locally installed rebar. 2012-06-14 18:03:34 +01:00
Gregory Burd
af4d2ba4d8 Merge branch 'master' of github.com:basho/hanoidb 2012-06-11 23:41:49 +01:00
Gregory Burd
ee89f93009 Use a uniform method to calculate bytes that can manage lists as well. 2012-06-11 23:40:28 +01:00
Gregory Burd
f4eea4a594 Whitespace and comment cleanup. 2012-06-11 23:40:07 +01:00
Gregory Burd
acbcf9d601 Return properly shaped tuple. 2012-06-11 23:39:04 +01:00
Gregory Burd
0e98543a84 Don't depend on basho_bench. 2012-06-06 18:17:11 +02:00
Kresten Krab Thorup
669f589d0c Implement smaller incremental merge steps 
Right now, this is controlled by the macro
INC_MERGE_STEP in hanoidb_nursery; eventually
we should turn this into a configuration option.

Making this small, (minimum is 1), hurts average perf
but reduces the 99.9 percentile latency.
 2012-05-11 22:31:23 +02:00
Kresten Krab Thorup
1b42172cbe Fix bug when merge result is an empty file 
This happens when all entries are expired, or
if all entries would have been tombstones.
 2012-05-11 14:58:53 +02:00
Kresten Krab Thorup
61720065d9 Add some doc/spec 2012-05-11 12:33:33 +02:00
Kresten Krab Thorup
fbda7af576 Honor expiry_secs == 0 
When this is the case, we use the old on-disk
encoding to shave 4 bytes off every entry.
 2012-05-11 12:30:07 +02:00
Kresten Krab Thorup
181b1debb8 Update basho_bench driver to recognize config flags 
Also, add default values to template configuration
 2012-05-11 12:29:16 +02:00
Kresten Krab Thorup
b6955c9a75 Refactor for expiry_secs option 
Tree nodes now hold entries at the form

   {Key, ?TOMBSTONE 
       | BinValue
       | {?TOMBSTONE, TStamp}
       | {BinValue, TStamp}}

We use the form without TStamp when expiry_secs
is unset or set to 0 (i.e., values don't expire).

merger/writer: Move KV count into writer, because
now the writer:add determines if a value is expired
and thus wither a value is actually written.  Thus,
writer now has a new API function which returns the
KV count written so far.

reader: lookup/fold API hides the TStamp tuples,
so only the next_node API used by the merger
is exposed to these {Key, {_, TStamp}} entries.

nursery: like reader, the TStamp'ed tuples are
not exposed in the client API; expired values
are simply not returned from fold/lookup.

hanoidb: add config option {expiry_secs, N}.

other modules: Make sure that config is passed
all the way down through (sub) processes to be
able to utilize the config option everywhere.

test: update to work with new option.
 2012-05-11 12:00:32 +02:00