greg/wterl
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity
284 commits 19 branches 0 tags 33 MiB
Commit graph

284 commits

This branch
This branch
All branches
Author SHA1 Message Date
Gregory Burd
08acdbefc8 Retry three times, then bail out and return not found. 2013-09-04 13:11:02 -04:00
Gregory Burd
033661875e Trace line numbers within macros. 2013-09-04 13:10:16 -04:00
Gregory Burd
9496f99019 WIP: race condition on close causes double-free 2013-09-04 13:09:17 -04:00
Gregory Burd
f4a1126fde Fix a few mistakes from the merge. 2013-08-26 13:07:46 -04:00
Gregory Burd
da418b4abf Merge remote-tracking branch 'origin/master' into gsb-rcu 
Conflicts:
	c_src/async_nif.h
	c_src/build_deps.sh
	src/async_nif.hrl
 2013-08-21 15:56:31 -04:00
Gregory Burd
48419ce4d0 Start the penalty after queues are 25% full because a) that makes sense, and b) 
that avoids some odd badarith errors when PctBusy is very small.
 2013-08-21 14:19:52 -04:00
Gregory Burd
2ddf0da53e Use malloc/free rather than enif_alloc/enif_free so as to avoid BEAM allocator 
overhead (bytes and time).  Create static references to commonly used Erlang
atoms to avoid overhead re-creating them on each request cycle.
 2013-08-21 12:20:19 -04:00
Gregory Burd
83c3faf74f Use malloc/free rather than enif_alloc/enif_free so as to avoid BEAM allocator 
overhead (bytes and time).
 2013-08-21 12:18:24 -04:00
Gregory Burd
2043e8ccc6 Because the build decends into the ext/compressors/snappy directory the 
relative paths won't find system/include, so use the absolute paths instead.
 2013-08-21 12:17:18 -04:00
Gregory Burd
33c8e53ccf Update to latest release of WiredTiger. Also, make sure Snappy builds before WiredTiger. 2013-08-21 12:16:24 -04:00
Gregory Burd
1bf66ae960 Every enqueued request now includes a hint as to how much work is pending in 
the lower C-code.  We use that to scale the reduction count penalty so that we
can (hopefully) signal to the Erlang scheduler enough information for it to
properly throttle work.  'eagain' should only happen when queues are full, we
have no choice but to keep this calling proc busy in a recursive loop trying
the request over and over if we're going to preserve request ordering.
 2013-08-21 12:15:34 -04:00
Gregory Burd
e67da86a9b Change backpressure method from EAGAIN to bump_reductions so as not to block Riak/KV vnode processes when queues backup. 2013-08-19 13:32:58 -04:00
Gregory Burd
2047104cda Remove the sleep from async_nif's EAGAIN path because it doesn't seem to have a positive effect. 2013-08-19 12:20:36 -04:00
Gregory Burd
a984a6dae0 Adding a small patch to URCU build process. 2013-08-13 08:54:22 -04:00
Gregory Burd
96d43d5d17 Re-use the unchanging value of 'Args' rather than including it in every recursive call. 2013-08-02 14:22:30 -04:00
Gregory Burd
05c8c615ef I think the make_ref() needs to be within the fun()'s context to trigger selective receive optimization in the beam's runtime. 2013-08-01 10:02:21 -04:00
Gregory Burd
f153509409 With some input from Jon I've managed to reduce this back into a macro rather than a fun and a macro calling a fun. He also suggested that on eagain I sleep a small amount of time so as to allow other work to catch up a bit. 2013-07-31 15:39:55 -04:00
Gregory Burd
ee904b4769 Lower the queue size to shrink potential for latency in queue. Remove earlier idea that more queues would lead to more even worker progress, just have 1 queue per Erlang-scheduler thread (generally, 1 per CPU core available). Also change the way worker threads decide when to cond_wait or migrate to other queues looking for work. 2013-07-31 15:06:28 -04:00
Gregory Burd
c9a4ab8325 Revert changes to async_nif and re-enable stats. Fixed selective recv. 2013-07-31 09:41:36 -04:00
Gregory Burd
2393257bef Really disable stats. 2013-07-30 14:34:04 -04:00
Gregory Burd
211ffd884c Ignore requests for stats for right now. 2013-07-30 14:30:04 -04:00
Gregory Burd
4418a74183 Increase the number of queues for work to reside. Worker threads, once started, don't exit until shutdown. 2013-07-30 14:21:26 -04:00
Gregory Burd
1623d5293c Increase the max queue size. 2013-07-30 13:30:43 -04:00
Gregory Burd
56c2ac27c2 Revert to a macro-only, non-recursive on eagain method for managing requests. 2013-07-30 13:27:13 -04:00
Gregory Burd
27dba903ef The ref needs to be in-scope of the recieve for it to be optimized. 2013-07-30 13:20:49 -04:00
Gregory Burd
45037cbcca Selective receive requires that the ref is created in the scope of the receive. 2013-07-30 12:53:33 -04:00
Gregory Burd
e9f9d13e47 WIP: replaced all locked queues in async_nif with lock-free RCU queues, compiles but SEGVs. 2013-07-28 20:16:52 -04:00
Gregory Burd
866b2a2ed1 Add the User-Space RCU library. 2013-07-27 08:21:40 -04:00
Gregory Burd
8f415df69c Merge pull request #10 from basho-labs/gsb-workers-migrate 
Worker threads should check for work in other queues before exiting.
 2013-07-26 17:12:15 -07:00
Gregory Burd
cce163db9f Fix potential to use uninitialized value when branching. 2013-07-26 20:08:49 -04:00
Gregory Burd
9a5defd8c9 Merge remote-tracking branch 'origin/master' into gsb-workers-migrate 
Conflicts:
	c_src/async_nif.h
 2013-07-26 10:31:23 -04:00
Gregory Burd
452d7694a6 Added some sanity checking of key/value sizes. Check for EAGAIN/INVAL/NOMEM when starting worker threads. Switch back to the 1.6.3 release branch of WT. 2013-07-26 10:27:21 -04:00
Gregory Burd
3627ff8690 Ensure that on EAGAIN we continue to try to spawn a worker. When workers finish with a queue have them migrate to the other queues looking for work. 2013-07-25 13:29:16 -04:00
Gregory Burd
122963133a Seems logging isn't a valid config value anymore, so remove it. 2013-07-18 13:26:53 -04:00
Gregory Burd
2a847b82d0 Forgot to remove this when I dumped the MRU. 2013-07-18 13:21:22 -04:00
Gregory Burd
2694cc1dba Remove the MRU, it wasn't really effective in most cases anyway and complicated logic (and was buggy). For now the cache mutex will be hot, but eventually I hope to move the cache to a lock-free dequeue. khash.h and cas.h aren't used anymore, so they have been removed. 2013-07-18 13:14:54 -04:00
Gregory Burd
bbadc81d53 Queue depth and num workers can race, so make sure that we start at least one worker when there are none active for that queue. 2013-07-15 16:51:08 -04:00
Gregory Burd
c3d3d39c36 Remove default setting from configuration. 2013-07-15 12:36:28 -04:00
Gregory Burd
bd0323af7a Update to WiredTiger 1.6.3. Fix a condition where a mutex was unlocked twice on eagain when queues were all full. 2013-07-15 12:21:10 -04:00
Gregory Burd
420b658e27 Really fix the lower bound for session handles (session_max) to 1024 (upper bound is 8192). 2013-07-08 19:59:00 -04:00
Gregory Burd
fea52c4ec3 Change lower bound for session handles (session_max) to 1024 (upper bound is 8192). 2013-07-08 19:45:45 -04:00
Gregory Burd
ac835f7617 Reduce precision of bloom filters to something more reasonable to avoid having very large bloom filters in cache. Reduce leaf pages to a more reasonable default. Given all data in Riak is <<Bucket, Key>> enable prefix_compression to hopefully reduce key overhead. 2013-07-08 13:49:44 -04:00
Gregory Burd
565f95f9b5 Cleanup makefile a bit 2013-07-08 11:12:50 -04:00
Gregory Burd
b6e8b175e3 Merge pull request #9 from basho-labs/gsb-ctx-cache 
Cache session/[{cursor, config}] for reuse and spawn threads when needed.
 2013-07-03 05:31:14 -07:00
Gregory Burd
bc0f5dbfc7 Evict older half of items in the cache by removing items from the end of the list, don't waste cycles computing timestamps. 2013-07-02 22:23:32 -04:00
Gregory Burd
b727538162 Fix shutdown 2013-07-02 22:07:34 -04:00
Gregory Burd
2672bab3ea Stats overhead due to hitting the clock and pulling a mutex caused a massive slowdown so now work is assigned to a new queue only when the candidate queue is deeper than the average of the other queues and threads are created only when the depth of the queue is larger than the number of threads working on that queue. 2013-07-02 19:58:00 -04:00
Gregory Burd
00e5889ac9 Changed conditions for worker thread creation. 2013-07-02 16:46:04 -04:00
Gregory Burd
4300b3036f Working on triggers that start/stop worker threads. 2013-07-01 21:09:21 -04:00
Gregory Burd
c7b45a7c2b Still ironing out stats. 2013-06-27 10:57:41 -04:00