This is a Riak storage backend that uses WiredTiger, an Erlang interface to the WiredTiger k/v pair storage engine, and an abstract async thread-pool layer for Erlang's NIF API called
c_src/async_nif.h
.
19268b7c77
* No longer expose WT_SESSION into Erlang at all as WT's model is to maintain one WT_SESSION per-thread and we don't know anything about threads in Erlang. * async_nif worker threads don't pull both mutexes on every loop when processing requests, only one * async_nif provides a worker_id (int, 0 - MAX_WORKERS) within the work block scope which we use to find our per-worker WT_SESSIONs * async_nif maintained a number of globals which I'm moving into the NIF's priv_data so that on upgrade/reload we have a fighting chance to "Do the Right Thing(TM)". * NIF Upgrades/Reloads started to plumb this in. * Use a khash to manage the cache of URI->WT_CURSORs per WT_SESSION. * Added start/stop positions into truncate call to allow for truncating sub-ranges data. * Lots of other details I'm sure I've forgotten and more left undone. Search for "TODO:" or try to compile to see what's left, and then there is a need for a lot more tests given all this new complexity. |
||
---|---|---|
c_src | ||
src | ||
.gitignore | ||
enable-wterl | ||
Makefile | ||
README.md | ||
rebar | ||
rebar.config | ||
update-version.sh |
wterl
is an Erlang interface to the WiredTiger database, and is written
to support a Riak storage backend that uses WiredTiger.
This backend currently supports only key-value storage and retrieval.
Remaining work includes:
- The
wterl:session_create
function currently returns an error under certain circumstances, so we currently ignore its return value. - The
riak_kv_wterl_backend
module is currently designed to rely on the fact that it runs in just a single Erlang scheduler thread, which is necessary because WiredTiger doesn't allow a session to be used concurrently by different threads. If the KV node design ever changes to involve concurrency across scheduler threads, this current design will no longer work correctly. - Currently the
riak_kv_wterl_backend
module is stored in this repository, but it really belongs in theriak_kv
repository. - There are currently some stability issues with WiredTiger that can sometimes cause errors when restarting KV nodes with non-empty WiredTiger storage.
Future support for secondary indexes requires WiredTiger features that are under development but are not yet available.
Deploying
You can deploy wterl
into a Riak devrel cluster using the enable-wterl
script. Clone the riak
repo, change your working directory to it, and
then execute the enable-wterl
script. It adds wterl
as a dependency,
runs make all devrel
, and then modifies the configuration settings of the
resulting dev nodes to use the WiredTiger storage backend.