`wterl` is an Erlang interface to the WiredTiger database, and is written to support a Riak storage backend that uses WiredTiger. Remaining work includes: TODO: * Find/fix any code marked "TODO:" * Why do we see {error, {eperm, _}} result on wterl:cursor_close/1 during fold_objects/4? * Why do we see {error, {eperm, _}} result on wterl:cursor_close/1? * Why do we see {error, {eperm, _}} result on wterl:cursor_next/1 during is_empty/1? * Why do we see {error, {eperm, _}} result on wterl:cursor_next_value/1 during status/1? * Why do we see {error, {ebusy, _}} result on wterl:drop/2? * Determine a better way to estimate the number of sessions we should configure WT for at startup in riak_kv_wterl_backend:max_sessions/1. * Make sure Erlang is optimizing for selective receive in async_nif_enqueue/3 because in the eLevelDB driver there is a comment: "This cannot be a separate function. Code must be inline to trigger Erlang compiler's use of optimized selective receive." * Provide a way to configure the cursor options, right now they are always "raw,overwrite". * Add support for Riak/KV 2i indexes using the same design pattern as eLevelDB (in a future version consider alternate schema) * If an operation using a shared cursor results in a non-normal error then it should be closed/discarded from the recycled pool * Cache cursors based on hash(table/config) rather than just table. * Finish NIF unload/reload functions and test. * Test an upgrade, include a format/schema/WT change. * When WT_PANIC is returned first try to unload/reload then driver and reset all state, if that fails then exit gracefully. * Currently the `riak_kv_wterl_backend` module is stored in this repository, but it really belongs in the `riak_kv` repository. * wterl:truncate/5 can segv, and its tests are commented out * Add async_nif and wterl NIF stats to the results provided by the stats API * Longer term ideas/changes to consider: * More testing, especially pulse/qc * Riak/KV integration * Store 2i indexes in separate tables * Store buckets, in separate tables and keep a <> index to ensure that folds across a vnode are easy * Provide a drop bucket API call * Support key expirey * An ets API (like the LevelDB's lets project) * Use mime-type to inform WT's schema for key value encoding * Other use cases within Riak * An AAE driver using WT * An ability to store the ring file via WT Deploying --------- You can deploy `wterl` into a Riak devrel cluster using the `enable-wterl` script. Clone the `riak` repo, change your working directory to it, and then execute the `enable-wterl` script. It adds `wterl` as a dependency, runs `make all devrel`, and then modifies the configuration settings of the resulting dev nodes to use the WiredTiger storage backend.