Right now, this is controlled by the macro
INC_MERGE_STEP in hanoidb_nursery; eventually
we should turn this into a configuration option.
Making this small, (minimum is 1), hurts average perf
but reduces the 99.9 percentile latency.
Tree nodes now hold entries at the form
{Key, ?TOMBSTONE
| BinValue
| {?TOMBSTONE, TStamp}
| {BinValue, TStamp}}
We use the form without TStamp when expiry_secs
is unset or set to 0 (i.e., values don't expire).
merger/writer: Move KV count into writer, because
now the writer:add determines if a value is expired
and thus wither a value is actually written. Thus,
writer now has a new API function which returns the
KV count written so far.
reader: lookup/fold API hides the TStamp tuples,
so only the next_node API used by the merger
is exposed to these {Key, {_, TStamp}} entries.
nursery: like reader, the TStamp'ed tuples are
not exposed in the client API; expired values
are simply not returned from fold/lookup.
hanoidb: add config option {expiry_secs, N}.
other modules: Make sure that config is passed
all the way down through (sub) processes to be
able to utilize the config option everywhere.
test: update to work with new option.
BREAKING CHANGE! This change provides for future
file format changes, but also breaks backwards
compatibility.
Also describe the file format in design_document
With this change, GETs will flow concurrently
down through the level controllers, replying
directly to the caller via gen_server:reply.
Very actor-like :-)
Current code base silently ignores CRC errors,
meaning that KVs that have errors will just
disappear, or may show up as a previously stored
value for the same key.
With this change, the fold worker does not
link to the receiver; now it simply monitors
the receiving process. If the receiver dies,
the fold worker dies normally.
The individual fold processes running on level
files are linked to the fold worker; so between
fold merge worker and those, normal link/kill
applies.
