This is a Riak storage backend that uses WiredTiger, an Erlang interface to the WiredTiger k/v pair storage engine, and an abstract async thread-pool layer for Erlang's NIF API called
c_src/async_nif.h.
1bf66ae960
the lower C-code. We use that to scale the reduction count penalty so that we can (hopefully) signal to the Erlang scheduler enough information for it to properly throttle work. 'eagain' should only happen when queues are full, we have no choice but to keep this calling proc busy in a recursive loop trying the request over and over if we're going to preserve request ordering. |
||
|---|---|---|
| c_src | ||
| src | ||
| tools | ||
| .gdbinit | ||
| .gitignore | ||
| AUTHORS | ||
| enable-wterl | ||
| Makefile | ||
| README.md | ||
| rebar | ||
| rebar.config | ||
| update-version.sh | ||
wterl is an Erlang interface to the WiredTiger database, and is written to
support a Riak storage backend that uses WiredTiger.
Remaining work includes:
TODO:
- Find/fix any code marked "TODO:"
- Why do we see {error, {eperm, _}} result on wterl:cursor_close/1 during fold_objects/4?
- Why do we see {error, {eperm, _}} result on wterl:cursor_close/1?
- Why do we see {error, {eperm, _}} result on wterl:cursor_next/1 during is_empty/1?
- Why do we see {error, {eperm, _}} result on wterl:cursor_next_value/1 during status/1?
- Why do we see {error, {ebusy, _}} result on wterl:drop/2?
- Determine a better way to estimate the number of sessions we should configure WT for at startup in riak_kv_wterl_backend:max_sessions/1.
- Make sure Erlang is optimizing for selective receive in async_nif_enqueue/3 because in the eLevelDB driver there is a comment: "This cannot be a separate function. Code must be inline to trigger Erlang compiler's use of optimized selective receive."
- Provide a way to configure the cursor options, right now they are always "raw,overwrite".
- Add support for Riak/KV 2i indexes using the same design pattern as eLevelDB (in a future version consider alternate schema)
- If an operation using a shared cursor results in a non-normal error then it should be closed/discarded from the recycled pool
- Cache cursors based on hash(table/config) rather than just table.
- Finish NIF unload/reload functions and test.
- Test an upgrade, include a format/schema/WT change.
- When WT_PANIC is returned first try to unload/reload then driver and reset all state, if that fails then exit gracefully.
- Currently the
riak_kv_wterl_backendmodule is stored in this repository, but it really belongs in theriak_kvrepository. - wterl:truncate/5 can segv, and its tests are commented out
- Add async_nif and wterl NIF stats to the results provided by the stats API
- Longer term ideas/changes to consider:
- More testing, especially pulse/qc
- Riak/KV integration
- Store 2i indexes in separate tables
- Store buckets, in separate tables and keep a <<bucket/key>> index to ensure that folds across a vnode are easy
- Provide a drop bucket API call
- Support key expirey
- An ets API (like the LevelDB's lets project)
- Use mime-type to inform WT's schema for key value encoding
- Other use cases within Riak
- An AAE driver using WT
- An ability to store the ring file via WT
Deploying
You can deploy wterl into a Riak devrel cluster using the enable-wterl
script. Clone the riak repo, change your working directory to it, and
then execute the enable-wterl script. It adds wterl as a dependency,
runs make all devrel, and then modifies the configuration settings of the
resulting dev nodes to use the WiredTiger storage backend.