reasonable setting for the WiredTiger cache size at runtime. This cache
is shared across all vnodes regarless of how many are active at any
given time. The algorithm is: max(1GB, 1/3 (RAM - Beam RSS size)). We don't
enable direct_io on purpose and data will be double buffered in WiredTiger's
cache and the filesystem buffer cache. This turns out to be faster than
direct I/O despite wasting a bit of RAM.
answer is 'no'). So far there has been no effort to validate that these
settings are in fact the best for Riak/KV or CS data access patterns. These
particular settings are, at best, an educated guess based on past experience,
the docs and reading about the benchmark the WiredTiger team published here:
https://github.com/wiredtiger/wiredtiger/wiki/YCSB-Mapkeeper-benchmark
Versions of WiredTiger prior to 1.3.3 had trouble handling riak restarts,
returning errors from wterl:session_create that would prevent riak from
restarting if it was not cleanly shut down on the previous run. We
previously avoided checking the return value of wterl:session_create to
avoid this problem. WT 1.3.3 fixes this, so reinstate the check of the
wterl:session_create return value.
Since 1.3.0 folded the session sync function into the session checkpoint
function, the wterl:sync/(2,3) functions are now renamed
wterl:checkpoint/(1,2). See
http://source.wiredtiger.com/1.3.0/upgrading.html for details.
Make the unit tests call riak_kv_backend:standard_test like other
backends do for their unit tests. This is a little awkward at the
moment because riak_kv_backend.erl doesn't live in this repository, so
it requires temporarily copying it into src to run "rebar eunit".
The backend fold_objects was failing because it was trying to use
wterl:fold_keys/3, which doesn't fold over values. Add fold/3 to
wterl.erl to fix this.
Also clean up a few dialyzer warnings.
WiredTiger session count defaults to 50, but we need at least one per
vnode, so for now hard-code it to 100. Also ignore table creation
results in the kv backend because WiredTiger is currently producing a
strange error if the table already exists.
In order to reduce the number of wiredtiger session open/close calls,
open a session at startup and keep it in the kv backend state. This is
safe as far as using these sessions on scheduler threads goes because
the riak_kv_vnode fsm serializes all calls through the backend, so
we'll never have a case of multiple scheduler threads concurrently
trying to use the same session.
Add a temporary copy of riak_kv_backend, renamed to
temp_riak_kv_backend, to allow wterl to build on its own. This file
must be deleted when riak_kv_wterl_backend is moved to its proper
place in riak_kv.
Change cursor_{next,prev} to return {ok, Key, Value}. Add
cursor_{next,prev}_{key,value} functions to allow cursor movement
followed by fetch of either key or value. Add fold_keys function. Add
unit tests for these changes. Also specify "inorder" on the unit test
generator functions, since the tests they specified are intended to be
run that way.
Break the single wterl unit test into a series of tests separated by
function. Use eunit assert macros and add test setup and teardown
functions. Add descriptive messages for the tests for display under
"rebar -v eunit".
Add app, supervisor, and worker to open and cache the database
connection. Revert to using binaries for WT config in order to allow
wterl callers to use proplists for configuration. Change {error,
not_found} to not_found return values from wterl.
Add support for cursor.{next,prev,search,search_near,reset}.
Change table_create to no longer pass back the table name, WiredTiger's
model is that you pass in the object name each time (and the current
model means we can't drop the table before we run the tests).
Upgrade to the WiredTiger 1.0 release, there's a bug where static
libraries aren't created correctly, for now create shared libraries
for wterl.
not a pure integer).
Remove "exclusive" from the list of options, it's not in the 1.0 release.
Set the cache_size for the test, just to prove it works.
A number of changes:
* Added exported functions to close connections and sessions.
* Added exported functions to create and drop tables. Currently not
happy with the shape of these functions, though, as they treat
tables in Erlang just as strings, and they force users of sessions
to have to pass both sessions and tables into get/put/delete
operations. It might be better to take a table identifier as part of
the session creation function and store that identifier with the
session resource.
* Added implementations for key-value get, put, and delete.
* Added very minimal unit tests for the changes above.
