greg/wterl
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

Commit graph

  • cc807a97d6 Add new mmap config option. master Gregory Burd 2014-01-15 10:45:04 -0500
  • ea99493ea3 Compenstate for LSM config API changes Gregory Burd 2013-12-09 12:54:58 -0500
  • 08b2d18463 Fix checkpoint config. Gregory Burd 2013-11-20 13:10:28 -0500
  • 3302ab26ed Use the develop branch for now. Gregory Burd 2013-11-19 14:16:26 -0500
  • db2daf99b2 Default logging off. Gregory Burd 2013-11-19 14:16:01 -0500
  • 68d9ed942b Update to WiredTiger 1.6.6 Gregory Burd 2013-11-18 20:54:59 -0500
  • 36faa4e713 Forgot to remove second use of checkpoint setting. Gregory Burd 2013-11-18 20:50:26 -0500
  • e560185420 When logging enable checkpoints, even when using LSM. Gregory Burd 2013-11-18 20:46:22 -0500
  • 448c0b555c Update config to match latest available options. Gregory Burd 2013-10-30 15:00:41 -0400
  • 634bcd188a Integrate new configuration options available in WiredTiger. Gregory Burd 2013-10-30 14:50:14 -0400
  • 1664fdcf8c API for handlers in WiredTiger changed to include session state, update our use of the API to match that change. Gregory Burd 2013-10-30 13:11:14 -0400
  • 95515f111c Merge pull request #11 from basho-labs/gsb-2.0-fixes Gregory Burd 2013-10-30 08:53:49 -0700
  • ac2c5caeff Change a few default configs and comment out the stats gathering for now. gsb-2.0-fixes Gregory Burd 2013-10-30 11:50:20 -0400
  • 75305dae94 Minor updates. Gregory Burd 2013-10-12 21:48:05 -0400
  • 7d0ad2dce1 Update the version strings and a few config values which changed names. Gregory Burd 2013-10-02 14:42:26 -0400
  • 84a85bbe38 Open a *statistics* cursor when gathering statistics. Gregory Burd 2013-10-02 14:41:36 -0400
  • 9d2896016b A few build automation changes/fixes. Gregory Burd 2013-10-02 14:41:06 -0400
  • 17585a99b1 priv now has the schema file in it, so be more specific with what we ignore in that dir Gregory Burd 2013-10-02 14:38:41 -0400
  • 942e51b753 OS/X uses ".dylib" rather than ".so" for shared libraries (because it's special) so I've worked around that. Also tightened up some tests so that we're not rebuilding the libraries when not necessary. Gregory Burd 2013-09-06 09:54:55 -0400
  • c60fa22422 Retry three times, then bail out and return not found. Gregory Burd 2013-09-04 13:11:02 -0400
  • 08acdbefc8 Retry three times, then bail out and return not found. gsb-rcu Gregory Burd 2013-09-04 13:11:02 -0400
  • 033661875e Trace line numbers within macros. Gregory Burd 2013-09-04 13:10:16 -0400
  • 9496f99019 WIP: race condition on close causes double-free Gregory Burd 2013-09-04 13:09:17 -0400
  • f4a1126fde Fix a few mistakes from the merge. Gregory Burd 2013-08-26 13:07:46 -0400
  • da418b4abf Merge remote-tracking branch 'origin/master' into gsb-rcu Gregory Burd 2013-08-21 15:56:31 -0400
  • 48419ce4d0 Start the penalty after queues are 25% full because a) that makes sense, and b) that avoids some odd badarith errors when PctBusy is very small. Gregory Burd 2013-08-21 14:19:52 -0400
  • 2ddf0da53e Use malloc/free rather than enif_alloc/enif_free so as to avoid BEAM allocator overhead (bytes and time). Create static references to commonly used Erlang atoms to avoid overhead re-creating them on each request cycle. Gregory Burd 2013-08-21 12:20:19 -0400
  • 83c3faf74f Use malloc/free rather than enif_alloc/enif_free so as to avoid BEAM allocator overhead (bytes and time). Gregory Burd 2013-08-21 12:18:24 -0400
  • 2043e8ccc6 Because the build decends into the ext/compressors/snappy directory the relative paths won't find system/include, so use the absolute paths instead. Gregory Burd 2013-08-21 12:17:18 -0400
  • 33c8e53ccf Update to latest release of WiredTiger. Also, make sure Snappy builds before WiredTiger. Gregory Burd 2013-08-21 12:16:24 -0400
  • 1bf66ae960 Every enqueued request now includes a hint as to how much work is pending in the lower C-code. We use that to scale the reduction count penalty so that we can (hopefully) signal to the Erlang scheduler enough information for it to properly throttle work. 'eagain' should only happen when queues are full, we have no choice but to keep this calling proc busy in a recursive loop trying the request over and over if we're going to preserve request ordering. Gregory Burd 2013-08-21 12:15:34 -0400
  • e67da86a9b Change backpressure method from EAGAIN to bump_reductions so as not to block Riak/KV vnode processes when queues backup. Gregory Burd 2013-08-19 13:32:58 -0400
  • 2047104cda Remove the sleep from async_nif's EAGAIN path because it doesn't seem to have a positive effect. Gregory Burd 2013-08-19 12:20:36 -0400
  • a984a6dae0 Adding a small patch to URCU build process. Gregory Burd 2013-08-13 08:54:22 -0400
  • 96d43d5d17 Re-use the unchanging value of 'Args' rather than including it in every recursive call. Gregory Burd 2013-08-02 14:18:19 -0400
  • ef3bc102f2 Replace all enif implementations of mutexes and conditions with their POSIX pthread equivalent on the theory that we don't want to be bumping heads with the Erlang runtime. gsb-less-enif-more-posix Gregory Burd 2013-08-02 14:20:04 -0400
  • e9b1a9ea0b Replace all enif implementations of mutexes and conditions with their POSIX pthread equivalent on the theory that we don't want to be bumping heads with the Erlang runtime. Gregory Burd 2013-08-02 14:19:44 -0400
  • 6919515de5 Re-use the unchanging value of 'Args' rather than including it in every recursive call. Gregory Burd 2013-08-02 14:18:19 -0400
  • 05c8c615ef I think the make_ref() needs to be within the fun()'s context to trigger selective receive optimization in the beam's runtime. Gregory Burd 2013-08-01 10:02:21 -0400
  • f153509409 With some input from Jon I've managed to reduce this back into a macro rather than a fun and a macro calling a fun. He also suggested that on eagain I sleep a small amount of time so as to allow other work to catch up a bit. Gregory Burd 2013-07-31 15:39:55 -0400
  • ee904b4769 Lower the queue size to shrink potential for latency in queue. Remove earlier idea that more queues would lead to more even worker progress, just have 1 queue per Erlang-scheduler thread (generally, 1 per CPU core available). Also change the way worker threads decide when to cond_wait or migrate to other queues looking for work. Gregory Burd 2013-07-31 15:06:28 -0400
  • c9a4ab8325 Revert changes to async_nif and re-enable stats. Fixed selective recv. Gregory Burd 2013-07-31 09:41:36 -0400
  • 2393257bef Really disable stats. Gregory Burd 2013-07-30 14:34:04 -0400
  • 211ffd884c Ignore requests for stats for right now. Gregory Burd 2013-07-30 14:30:04 -0400
  • 4418a74183 Increase the number of queues for work to reside. Worker threads, once started, don't exit until shutdown. Gregory Burd 2013-07-30 14:21:26 -0400
  • 1623d5293c Increase the max queue size. Gregory Burd 2013-07-30 13:30:43 -0400
  • 56c2ac27c2 Revert to a macro-only, non-recursive on eagain method for managing requests. Gregory Burd 2013-07-30 13:27:13 -0400
  • 27dba903ef The ref needs to be in-scope of the recieve for it to be optimized. Gregory Burd 2013-07-30 13:20:49 -0400
  • 45037cbcca Selective receive requires that the ref is created in the scope of the receive. Gregory Burd 2013-07-30 12:53:33 -0400
  • e9f9d13e47 WIP: replaced all locked queues in async_nif with lock-free RCU queues, compiles but SEGVs. Gregory Burd 2013-07-28 20:16:52 -0400
  • 866b2a2ed1 Add the User-Space RCU library. Gregory Burd 2013-07-27 08:21:40 -0400
  • 8f415df69c Merge pull request #10 from basho-labs/gsb-workers-migrate Gregory Burd 2013-07-26 17:12:15 -0700
  • 37424474c1 Merge cce163db9f into 452d7694a6 Gregory Burd 2013-07-26 17:11:58 -0700
  • cce163db9f Fix potential to use uninitialized value when branching. gsb-workers-migrate Gregory Burd 2013-07-26 20:08:49 -0400
  • 9a5defd8c9 Merge remote-tracking branch 'origin/master' into gsb-workers-migrate Gregory Burd 2013-07-26 10:31:23 -0400
  • 452d7694a6 Added some sanity checking of key/value sizes. Check for EAGAIN/INVAL/NOMEM when starting worker threads. Switch back to the 1.6.3 release branch of WT. Gregory Burd 2013-07-26 10:27:21 -0400
  • 3627ff8690 Ensure that on EAGAIN we continue to try to spawn a worker. When workers finish with a queue have them migrate to the other queues looking for work. Gregory Burd 2013-07-25 13:29:16 -0400
  • 122963133a Seems logging isn't a valid config value anymore, so remove it. Gregory Burd 2013-07-18 13:26:53 -0400
  • 2a847b82d0 Forgot to remove this when I dumped the MRU. Gregory Burd 2013-07-18 13:21:22 -0400
  • 2694cc1dba Remove the MRU, it wasn't really effective in most cases anyway and complicated logic (and was buggy). For now the cache mutex will be hot, but eventually I hope to move the cache to a lock-free dequeue. khash.h and cas.h aren't used anymore, so they have been removed. Gregory Burd 2013-07-18 13:14:54 -0400
  • ef281a0c44 WIP: cleanup a few things. gsb-atomic-ops Gregory Burd 2013-07-18 13:06:15 -0400
  • 72448d2b54 Merge remote-tracking branch 'origin/master' into gsb-atomic-ops Gregory Burd 2013-07-17 16:34:10 -0400
  • bbadc81d53 Queue depth and num workers can race, so make sure that we start at least one worker when there are none active for that queue. Gregory Burd 2013-07-15 16:51:08 -0400
  • c3d3d39c36 Remove default setting from configuration. Gregory Burd 2013-07-15 12:36:28 -0400
  • bd0323af7a Update to WiredTiger 1.6.3. Fix a condition where a mutex was unlocked twice on eagain when queues were all full. Gregory Burd 2013-07-15 12:21:10 -0400
  • 420b658e27 Really fix the lower bound for session handles (session_max) to 1024 (upper bound is 8192). Gregory Burd 2013-07-08 19:59:00 -0400
  • fea52c4ec3 Change lower bound for session handles (session_max) to 1024 (upper bound is 8192). Gregory Burd 2013-07-08 19:45:45 -0400
  • ac835f7617 Reduce precision of bloom filters to something more reasonable to avoid having very large bloom filters in cache. Reduce leaf pages to a more reasonable default. Given all data in Riak is <<Bucket, Key>> enable prefix_compression to hopefully reduce key overhead. Gregory Burd 2013-07-08 13:49:44 -0400
  • aedb91aca5 WIP: integrated AO but not quite right yet. Gregory Burd 2013-07-08 13:34:42 -0400
  • 565f95f9b5 Cleanup makefile a bit Gregory Burd 2013-07-08 11:12:50 -0400
  • b6e8b175e3 Merge pull request #9 from basho-labs/gsb-ctx-cache Gregory Burd 2013-07-03 05:31:14 -0700
  • 05ad453e13 Merge bc0f5dbfc7 into 013251e6d9 Gregory Burd 2013-07-03 05:29:48 -0700
  • bc0f5dbfc7 Evict older half of items in the cache by removing items from the end of the list, don't waste cycles computing timestamps. gsb-ctx-cache Gregory Burd 2013-07-02 22:23:32 -0400
  • b727538162 Fix shutdown Gregory Burd 2013-07-02 22:07:34 -0400
  • 2672bab3ea Stats overhead due to hitting the clock and pulling a mutex caused a massive slowdown so now work is assigned to a new queue only when the candidate queue is deeper than the average of the other queues and threads are created only when the depth of the queue is larger than the number of threads working on that queue. Gregory Burd 2013-07-02 19:58:00 -0400
  • 00e5889ac9 Changed conditions for worker thread creation. Gregory Burd 2013-07-02 16:46:04 -0400
  • 4300b3036f Working on triggers that start/stop worker threads. Gregory Burd 2013-07-01 21:09:21 -0400
  • c7b45a7c2b Still ironing out stats. Gregory Burd 2013-06-27 10:57:41 -0400
  • c41e411a92 Worker threads come and go as needed with a lower bound of 2 and an upper bound of ASYNC_NIF_MAX_WORKERS. Stats were improved to use thread local storage for measures. With stats working again wterl uses them to determine who to evict. Wterl's signature calculation for an operation wasn't correct and so the cache wasn't efficient at all, this has been fixed. Gregory Burd 2013-06-25 13:31:43 -0400
  • a3c54b1610 Cleanup a bit. Gregory Burd 2013-06-19 14:54:27 -0400
  • 060abffcff Minor oversight, fixed. Gregory Burd 2013-06-19 14:42:56 -0400
  • 0f180a6531 Fixed a few mistakes. Gregory Burd 2013-06-19 14:37:30 -0400
  • 450299dc2d Comment out a test that's not yet working. Gregory Burd 2013-06-18 16:40:43 -0400
  • 4ae8ffb4cd Update debugging messages a bit. Fix a bug in the signature function. Gregory Burd 2013-06-18 13:49:25 -0400
  • 34e88c9234 Add some debugging output. Gregory Burd 2013-06-18 13:12:10 -0400
  • 50e24d0f48 Add a longer, multi-table test. Use a release version of WiredTiger from now on. Gregory Burd 2013-06-18 09:21:58 -0400
  • 53307e8c01 A great deal of cleanup. EUnit and EQC tests pass. Gregory Burd 2013-06-14 16:57:53 -0400
  • ff7d1d6e20 WIP: further simplifying context cache Gregory Burd 2013-06-14 10:52:45 -0400
  • 7952358781 WIP: cache wasn't returning items found Gregory Burd 2013-06-12 09:08:09 -0400
  • 4460434db1 WIP: remove potential for infinite loops with CAS and fix a few issues in async Gregory Burd 2013-06-12 08:09:51 -0400
  • 110b482962 Some paranoia and a few fixes Gregory Burd 2013-06-11 12:13:06 -0400
  • 8ea866bf20 Logic inversion on CAS() operation. Gregory Burd 2013-06-10 16:18:55 -0400
  • 2a4b8ee7d2 WIP: simplify the cache from hash-of-lists to list; use a CAS() operation to protect the most-recently-used (mru) list. Gregory Burd 2013-06-10 14:31:59 -0400
  • b2c0b65114 Fixes for OS/X and whitespace cleanup. Gregory Burd 2013-06-06 15:16:50 -0400
  • 0fef28de92 WIP: basho_bench tests are running fine now, need more work to ensure cache is functioning properly. Gregory Burd 2013-06-05 11:41:41 -0400
  • 778ba20352 WIP: a bit of cleanup fixes a few mistakes Gregory Burd 2013-06-04 17:21:50 -0400
  • f1b7d8322d WIP: replcae the kbtree with khash, we don't need the tree features (yet, if ever) and hash is faster; add a most-recently-used stash for contexts as it's highly likely that worker threads will do many operations with the same shape/signature of session/cursors/tables/config and that path can be lock-free as well making it much faster (one would hope); somewhere something is stepping on read-only ErlNifBinary data and so a crc check is failing and causing the runtime to abort, that's the latest item to find/fix. Gregory Burd 2013-06-04 14:45:23 -0400
  • b002294c4e WIP: all tests (but drop) passing again, need to fix that and valgrind next. Gregory Burd 2013-05-31 20:32:02 -0400
  • 9468870e1f WIP: use a log2 histogram to track how long items live in the cache, evict items which have been in cache greater than the mean time. Gregory Burd 2013-05-30 17:10:51 -0400
  • 15fbc71ea7 WIP: pieces in place, need to work out the kinks now. Gregory Burd 2013-05-30 14:21:34 -0400