greg/wterl
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity
This is a Riak storage backend that uses WiredTiger, an Erlang interface to the WiredTiger k/v pair storage engine, and an abstract async thread-pool layer for Erlang's NIF API called c_src/async_nif.h.
195 commits 19 branches 0 tags 33 MiB
C 51.3%
Erlang 44.4%
Shell 2.9%
Makefile 1.4%
Find a file
Gregory Burd 422dcfda89 Return 'eagain' when request queue is full and then try the request again. 
In the worst case is the request queue remains full and we loop between
the NIF and Erlang forever trying over and over to enqueue the request. If
that happens we shouldn't take schedulers offline as the NIF calls are fast
and we shouldn't run out of memory as that is bounded.  CPU will show a lot
of activity, but progress will continue in Erlang.
2013-04-25 15:18:23 -04:00
c_src Return 'eagain' when request queue is full and then try the request again. 2013-04-25 15:18:23 -04:00
src Return 'eagain' when request queue is full and then try the request again. 2013-04-25 15:18:23 -04:00
tools Driver and config for basho_bench testing. 2013-04-24 14:27:56 -04:00
.gdbinit Move the FIFO Queue implementation into its own file (fifo_q.h). Work 2013-04-18 10:37:36 -04:00
.gitignore Inline the fifo_q functions to speed them up and silence compiler warnings 2013-04-17 11:17:13 -04:00
AUTHORS Opening a connection now involves passing in both a configuration 2013-04-06 17:19:59 -04:00
enable-wterl Remove check for other module. 2013-03-28 10:23:56 -04:00
Makefile Improve, extend some targets. 2013-04-21 11:10:06 -04:00
README.md add README 2012-03-27 22:27:50 -04:00
rebar update rebar and rebar.config 2012-10-26 09:25:35 -04:00
rebar.config Remove debug compiler flags. 2013-04-23 15:00:44 -04:00
update-version.sh Inline the fifo_q functions to speed them up and silence compiler warnings 2013-04-17 11:17:13 -04:00

README.md

wterl is an Erlang interface to the WiredTiger database, and is written to support a Riak storage backend that uses WiredTiger.

This backend currently supports only key-value storage and retrieval.

Remaining work includes:

  • The wterl:session_create function currently returns an error under certain circumstances, so we currently ignore its return value.
  • The riak_kv_wterl_backend module is currently designed to rely on the fact that it runs in just a single Erlang scheduler thread, which is necessary because WiredTiger doesn't allow a session to be used concurrently by different threads. If the KV node design ever changes to involve concurrency across scheduler threads, this current design will no longer work correctly.
  • Currently the riak_kv_wterl_backend module is stored in this repository, but it really belongs in the riak_kv repository.
  • There are currently some stability issues with WiredTiger that can sometimes cause errors when restarting KV nodes with non-empty WiredTiger storage.

Future support for secondary indexes requires WiredTiger features that are under development but are not yet available.

Deploying

You can deploy wterl into a Riak devrel cluster using the enable-wterl script. Clone the riak repo, change your working directory to it, and then execute the enable-wterl script. It adds wterl as a dependency, runs make all devrel, and then modifies the configuration settings of the resulting dev nodes to use the WiredTiger storage backend.