2.8 KiB
2.8 KiB
wterl is an Erlang interface to the WiredTiger database, and is written to
support a Riak storage backend that uses WiredTiger.
Remaining work includes:
TODO:
- Find/fix any code marked "TODO:"
- Why do we see {error, {eperm, _}} result on wterl:cursor_close/1 during fold_objects/4?
- Why do we see {error, {eperm, _}} result on wterl:cursor_close/1?
- Why do we see {error, {eperm, _}} result on wterl:cursor_next/1 during is_empty/1?
- Why do we see {error, {eperm, _}} result on wterl:cursor_next_value/1 during status/1?
- Why do we see {error, {ebusy, _}} result on wterl:drop/2?
- Determine a better way to estimate the number of sessions we should configure WT for at startup in riak_kv_wterl_backend:max_sessions/1.
- Make sure Erlang is optimizing for selective receive in async_nif_enqueue/3 because in the eLevelDB driver there is a comment: "This cannot be a separate function. Code must be inline to trigger Erlang compiler's use of optimized selective receive."
- Provide a way to configure the cursor options, right now they are always "raw,overwrite".
- Add support for Riak/KV 2i indexes using the same design pattern as eLevelDB (in a future version consider alternate schema)
- If an operation using a shared cursor results in a non-normal error then it should be closed/discarded from the recycled pool
- Cache cursors based on hash(table/config) rather than just table.
- Finish NIF unload/reload functions and test.
- Test an upgrade, include a format/schema/WT change.
- When WT_PANIC is returned first try to unload/reload then driver and reset all state, if that fails then exit gracefully.
- Currently the
riak_kv_wterl_backendmodule is stored in this repository, but it really belongs in theriak_kvrepository. - wterl:truncate/5 can segv, and its tests are commented out
- Add async_nif and wterl NIF stats to the results provided by the stats API
- Longer term ideas/changes to consider:
- More testing, especially pulse/qc
- Riak/KV integration
- Store 2i indexes in separate tables
- Store buckets, in separate tables and keep a <<bucket/key>> index to ensure that folds across a vnode are easy
- Provide a drop bucket API call
- Support key expirey
- An ets API (like the LevelDB's lets project)
- Use mime-type to inform WT's schema for key value encoding
- Other use cases within Riak
- An AAE driver using WT
- An ability to store the ring file via WT
Deploying
You can deploy wterl into a Riak devrel cluster using the enable-wterl
script. Clone the riak repo, change your working directory to it, and
then execute the enable-wterl script. It adds wterl as a dependency,
runs make all devrel, and then modifies the configuration settings of the
resulting dev nodes to use the WiredTiger storage backend.