Commit graph

165 commits

Author SHA1 Message Date
Gregory Burd
9ed2137730 A bit of cleanup 2013-04-14 19:33:14 -04:00
Super-User
15a9a70c41 bzero isn't portable, use memset instead. Include bzip2 and snappy source for use by WT block compression
e
2013-04-14 21:23:57 +00:00
Gregory Burd
22643fc312 Change from waking up all threads to waking up one thread. 2013-04-14 13:54:45 -04:00
Gregory Burd
8fe9dc9bad This is a different approach from async2, instead of a single queue
and many workers this has a queue per scheduler and a few workers per
queue.
2013-04-14 08:44:54 -04:00
Gregory Burd
456129e7f3 Reusing closed cursors is bad for your health (SEGV), don't do that.
Also a bit of paranoia, bzero after all enif_alloc calls.
2013-04-12 16:59:10 -04:00
Gregory Burd
1962640382 Move the cleanup code into conn_close and remove the need for a dtor
function for resource GC/cleanup.  It's easier.
2013-04-12 16:09:00 -04:00
Gregory Burd
b0ca1e4098 WIP -- 41 tests pass, 3 fail = progress! Right now I have the connection
dtor commented out, otherwise it would SEGV on GC.  Some of the truncate
tests fail (race?) but don't SEGV, so that's not so bad.  Fixed numerous
issues and also removed a mutex and queue of idle worker threads because
it isn't used so why bother with it?
2013-04-12 15:25:56 -04:00
Gregory Burd
5a73264307 WIP -- Most tests passing, truncate and drop failing due to EBUSY,
also not sure why there is a segv when exiting but not in gdb.
Unlock mutex before returning error codes.  Alloc/free copies of some
strings allocated on the stack.
2013-04-11 11:57:41 -04:00
Gregory Burd
5c0295624d WIP -- most tests running, a few commented out, sometimes a segv on exit 2013-04-08 17:21:48 -04:00
Gregory Burd
07061ed6e8 WIP -- more tests passing, plenty left to fix 2013-04-07 22:16:44 -04:00
Gregory Burd
3b41805a71 WIP -- continue to fix tests in wterl 2013-04-07 10:07:19 -04:00
Gregory Burd
5ac006630e WIP -- tests failing -- WIP
Fixed numerous bugs.  Pushed config_to_bin down to just before
calling the _nif() function, everywhere else config is a proplist.
2013-04-07 09:21:47 -04:00
Gregory Burd
2675aa501a Opening a connection now involves passing in both a configuration
string for the connection and for any sessions.  Don't convert
configuration proplists into strings (Erlang binaries) until calling
into the NIF layer.
2013-04-06 17:19:59 -04:00
Gregory Burd
f7219dcaec WIP: Fix minor C89 issue 2013-04-06 11:14:23 -04:00
Gregory Burd
b4f82a388d WIP-- Compiling, not yet tested/functional -- WIP
Changes required to iron out compiler errors, warnings, etc.  Code now
compiles with clang or gcc.
2013-04-06 11:05:41 -04:00
Gregory Burd
19268b7c77 WIP-- Not Yet Compiling or Functional -- WIP
* No longer expose WT_SESSION into Erlang at all as WT's model is to
maintain one WT_SESSION per-thread and we don't know anything about
threads in Erlang.

* async_nif worker threads don't pull both mutexes on every loop
when processing requests, only one

* async_nif provides a worker_id (int, 0 - MAX_WORKERS) within the
work block scope which we use to find our per-worker WT_SESSIONs

* async_nif maintained a number of globals which I'm moving into
the NIF's priv_data so that on upgrade/reload we have a fighting
chance to "Do the Right Thing(TM)".

* NIF Upgrades/Reloads started to plumb this in.

* Use a khash to manage the cache of URI->WT_CURSORs per WT_SESSION.

* Added start/stop positions into truncate call to allow for truncating
sub-ranges data.

* Lots of other details I'm sure I've forgotten and more left undone.
Search for "TODO:" or try to compile to see what's left, and then
there is a need for a lot more tests given all this new complexity.
2013-04-05 18:09:54 -04:00
Gregory Burd
163a5073cb An updated Emacs cmode leads to all sorts of whitespace fixes intermixed with
some merge changes.
2013-04-02 09:42:07 -04:00
Gregory Burd
6aa51437cc Merge remote-tracking branch 'origin/master' into gsb-async-nifs2
Conflicts:
	c_src/wterl.c
	src/riak_kv_wterl_backend.erl
	src/wterl.erl
2013-04-02 09:33:41 -04:00
Gregory Burd
a75d60bf27 Add an RPATH to the extionsion shared libs so they can find objects in the priv/ directory at runtime 2013-04-01 16:17:18 -04:00
Gregory Burd
95a6f52a1d Whitespace 2013-04-01 14:00:06 -04:00
Gregory Burd
e3219ff2c5 Add linker flags so that shared libraries are found relative to either the wterl layout or the Riak layout. 2013-03-28 22:21:36 -04:00
Gregory Burd
3ef48e0ac0 Change from static linking to dynamic. Enable snappy and bzip2 compression. 2013-03-28 10:24:26 -04:00
Gregory Burd
9f4e08ca6e Queue and execute work from scheduler threads on other threads to prevent schedulers from sleeping. 2013-03-24 21:00:48 -04:00
Gregory Burd
0203b06a61 Cursor delete operation only requires cursor handle and key (no need for a value) 2013-03-21 18:44:13 -04:00
Gregory Burd
ab45690a4b Pull via http to avoid certificate errors with https on certain platforms
(Illumos, SmartOS, Solaris... I'm looking at you.) Speed up builds by
running make in parallel (use all the cores!).
2013-03-16 16:41:26 -04:00
Gregory Burd
021dcb10f5 Build from source pulled from a specific branch on GitHub for "basho"
related work and integration of WiredTiger rather than the latest
.tar.bz2 file they distribute.
2013-03-12 17:16:15 -04:00
Gregory Burd
9a02718a2e Pass configuration along when opening sessions. 2013-03-12 17:16:15 -04:00
Steve Vinoski
446362d11a update to WiredTiger 1.4.2 2013-01-14 19:13:45 -05:00
Steve Vinoski
f67a2dd7af update to WiredTiger 1.4.1 2012-12-12 23:57:21 -05:00
Steve Vinoski
6768b86484 update to WiredTiger 1.4.0 2012-12-04 11:20:33 -05:00
Steve Vinoski
c9df3959c8 update to WiredTiger 1.3.8 2012-11-22 23:14:01 -05:00
Steve Vinoski
bb4de898ab update to WiredTiger 1.3.7 2012-11-09 12:51:40 -05:00
Steve Vinoski
317ae4164d update to WiredTiger 1.3.6 2012-11-06 12:41:36 -05:00
Steve Vinoski
f8454b11e1 update to WiredTiger 1.3.5 2012-10-26 09:19:47 -04:00
Steve Vinoski
62252ee6ba update to WiredTiger 1.3.4 2012-10-19 08:17:19 -04:00
Steve Vinoski
d7127b03e2 update to WiredTiger 1.3.3 2012-10-11 13:33:37 -04:00
Steve Vinoski
b2807e2c14 upgrade to WiredTiger version 1.3.0
Since 1.3.0 folded the session sync function into the session checkpoint
function, the wterl:sync/(2,3) functions are now renamed
wterl:checkpoint/(1,2). See
http://source.wiredtiger.com/1.3.0/upgrading.html for details.
2012-09-17 10:58:48 -04:00
Steve Vinoski
6c808da7af update to WiredTiger 1.2.2 2012-06-20 15:24:16 -04:00
Steve Vinoski
d766ca1790 update to WiredTiger 1.2.1 2012-06-15 08:54:13 -04:00
Gregory Burd
57a70dc331 Merge branch 'master' of github.com:basho/wterl
Conflicts:
	c_src/build_deps.sh
2012-06-04 09:18:15 +01:00
Gregory Burd
d14e171f4b WIP: Update to WiredTiger 1.2.0 release 2012-06-04 09:16:35 +01:00
Steve Vinoski
3e676f82ad update to WiredTiger 1.1.5 2012-04-26 10:16:52 -04:00
Steve Vinoski
7ca3bb4385 update to WiredTiger 1.1.4 2012-04-16 19:55:12 -04:00
Steve Vinoski
509cc92cfc fix copyright dates 2012-04-09 16:49:26 -04:00
Steve Vinoski
cd8a563aa4 apply evict-fixes patch to wiredtiger 1.1.3 2012-04-07 09:49:00 -04:00
Steve Vinoski
2e37fc1bd1 avoid using cursor-fetched values after closing cursor
The wterl:session_get function was fetching a value using a cursor, then
closing the cursor, and then trying to use the fetched value to create the
return binary. According to the WiredTiger documentation, this is a no-no
-- a value is good only until the next operation on the cursor used to
fetch it. We've been seeing occasional segmentation violations in the
memcpy for the fetched value, and this might explain it.
2012-04-04 20:13:58 -04:00
Steve Vinoski
fbc6f9b440 update to WiredTiger 1.1.3
Update to WiredTiger 1.1.3 and also reinstate checking that
wterl:session_create returns ok.
2012-04-04 15:20:08 -04:00
Steve Vinoski
ca36db5709 NIF code cleanup
Use camel case for type names, make brace placement consistent.
2012-04-02 18:01:02 -04:00
Steve Vinoski
ea3e73c0d7 update to WiredTiger 1.1.2 2012-03-28 10:20:42 -04:00
Steve Vinoski
221df5337d update to WiredTiger 1.1.1 2012-03-12 10:22:22 -04:00
Keith Bostic
cddaf8e1d9 Replace enif_alloc_binary/enif_make_binary call pairs with a single
enif_make_new_binary call.
2012-02-20 13:30:09 -05:00
Steve Vinoski
6c0f14da6e add fold_keys, and funs for getting keys and values via cursors
Change cursor_{next,prev} to return {ok, Key, Value}. Add
cursor_{next,prev}_{key,value} functions to allow cursor movement
followed by fetch of either key or value. Add fold_keys function. Add
unit tests for these changes. Also specify "inorder" on the unit test
generator functions, since the tests they specified are intended to be
run that way.
2012-02-19 20:20:38 -05:00
Keith Bostic
3323cf5844 I messed up some minor style/indentation, ignore all calls to cursor->close
(that's not correct, but it will do for now).
2012-02-17 09:07:25 -05:00
Steve Vinoski
7e6d947765 make wterl an app, cache the WT connection
Add app, supervisor, and worker to open and cache the database
connection. Revert to using binaries for WT config in order to allow
wterl callers to use proplists for configuration. Change {error,
not_found} to not_found return values from wterl.
2012-02-17 00:42:22 -05:00
Keith Bostic
c89d6deaea minor stylistic updates. 2012-02-16 19:24:18 -05:00
Keith Bostic
212a735339 Implement the rest of the interesting cursor operations (insert, update, remove). 2012-02-16 19:06:56 -05:00
Keith Bostic
33b8ea60cf Implement the rest of the session methods.
Format pass, naming pass.
2012-02-16 18:44:28 -05:00
Keith Bostic
7a19859ede Rename table_{create,drop} session_{create,drop}, they operate on things
other than tables, and we no longer hardcode the object type.
2012-02-16 15:07:44 -05:00
Keith Bostic
6f0357b4c5 Fix cursor.{open,close}, it now works.
Add support for cursor.{next,prev,search,search_near,reset}.

Change table_create to no longer pass back the table name, WiredTiger's
model is that you pass in the object name each time (and the current
model means we can't drop the table before we run the tests).

Upgrade to the WiredTiger 1.0 release, there's a bug where static
libraries aren't created correctly, for now create shared libraries
for wterl.
2012-02-16 14:57:28 -05:00
Keith Bostic
5a7936bf1c Change the cursor open/close code to match other resources (RESOURCE structure that holds the
WiredTiger resource).

Change names to be consistent, open/XXX/close.

Add cursor next call.
2012-02-16 13:22:03 -05:00
Gregory Burd
816e916658 Add cursor create/close functions. 2012-02-15 14:31:40 -05:00
Steve Vinoski
d77abe34d7 add missing include of string.h in wterl.c 2012-02-15 09:59:28 -05:00
Steve Vinoski
b93946a5e0 more WiredTiger prototyping
A number of changes:

* Added exported functions to close connections and sessions.

* Added exported functions to create and drop tables. Currently not
  happy with the shape of these functions, though, as they treat
  tables in Erlang just as strings, and they force users of sessions
  to have to pass both sessions and tables into get/put/delete
  operations. It might be better to take a table identifier as part of
  the session creation function and store that identifier with the
  session resource.

* Added implementations for key-value get, put, and delete.

* Added very minimal unit tests for the changes above.
2012-02-08 17:31:16 -05:00
Dave Smith
dcb4270f9d Basics of conn/session instantiation working 2011-12-21 21:46:35 -07:00
Dave Smith
2cdf2584f2 Initial commit 2011-12-18 13:54:46 -07:00