This is a Riak storage backend that uses WiredTiger, an Erlang interface to the WiredTiger k/v pair storage engine, and an abstract async thread-pool layer for Erlang's NIF API called
c_src/async_nif.h
.
77e0dacdb5
to avoid a) opening a large number of Sessions or Cursors and b) avoid reusing a Session/Cursor pair in more than one thread. Each operation, except folds, use the Session/Cursor pair corresponding to the scheduler id (roughly "core") they are running on. Each vnode instance will have (count(schedulers) + count(active folds)) Session/Cursor pairs in use so be sure to set session_max large enough when opening the connection to WiredTiger. This method will cause problems with scheduler threads going to "sleep" so be sure to run schedmon to wake them up. |
||
---|---|---|
c_src | ||
src | ||
.gitignore | ||
enable-wterl | ||
Makefile | ||
README.md | ||
rebar | ||
rebar.config |
wterl
is an Erlang interface to the WiredTiger database, and is written
to support a Riak storage backend that uses WiredTiger.
This backend currently supports only key-value storage and retrieval.
Remaining work includes:
- The
wterl:session_create
function currently returns an error under certain circumstances, so we currently ignore its return value. - The
riak_kv_wterl_backend
module is currently designed to rely on the fact that it runs in just a single Erlang scheduler thread, which is necessary because WiredTiger doesn't allow a session to be used concurrently by different threads. If the KV node design ever changes to involve concurrency across scheduler threads, this current design will no longer work correctly. - Currently the
riak_kv_wterl_backend
module is stored in this repository, but it really belongs in theriak_kv
repository. - There are currently some stability issues with WiredTiger that can sometimes cause errors when restarting KV nodes with non-empty WiredTiger storage.
Future support for secondary indexes requires WiredTiger features that are under development but are not yet available.
Deploying
You can deploy wterl
into a Riak devrel cluster using the enable-wterl
script. Clone the riak
repo, change your working directory to it, and
then execute the enable-wterl
script. It adds wterl
as a dependency,
runs make all devrel
, and then modifies the configuration settings of the
resulting dev nodes to use the WiredTiger storage backend.