greg/wterl
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity
251 commits 19 branches 0 tags 33 MiB
Commit graph

148 commits

Author SHA1 Message Date
Gregory Burd
2a847b82d0 Forgot to remove this when I dumped the MRU. 2013-07-18 13:21:22 -04:00
Gregory Burd
2694cc1dba Remove the MRU, it wasn't really effective in most cases anyway and complicated logic (and was buggy). For now the cache mutex will be hot, but eventually I hope to move the cache to a lock-free dequeue. khash.h and cas.h aren't used anymore, so they have been removed. 2013-07-18 13:14:54 -04:00
Gregory Burd
bbadc81d53 Queue depth and num workers can race, so make sure that we start at least one worker when there are none active for that queue. 2013-07-15 16:51:08 -04:00
Gregory Burd
bd0323af7a Update to WiredTiger 1.6.3. Fix a condition where a mutex was unlocked twice on eagain when queues were all full. 2013-07-15 12:21:10 -04:00
Gregory Burd
bc0f5dbfc7 Evict older half of items in the cache by removing items from the end of the list, don't waste cycles computing timestamps. 2013-07-02 22:23:32 -04:00
Gregory Burd
b727538162 Fix shutdown 2013-07-02 22:07:34 -04:00
Gregory Burd
2672bab3ea Stats overhead due to hitting the clock and pulling a mutex caused a massive slowdown so now work is assigned to a new queue only when the candidate queue is deeper than the average of the other queues and threads are created only when the depth of the queue is larger than the number of threads working on that queue. 2013-07-02 19:58:00 -04:00
Gregory Burd
00e5889ac9 Changed conditions for worker thread creation. 2013-07-02 16:46:04 -04:00
Gregory Burd
4300b3036f Working on triggers that start/stop worker threads. 2013-07-01 21:09:21 -04:00
Gregory Burd
c7b45a7c2b Still ironing out stats. 2013-06-27 10:57:41 -04:00
Gregory Burd
c41e411a92 Worker threads come and go as needed with a lower bound of 2 and an 
upper bound of ASYNC_NIF_MAX_WORKERS.  Stats were improved to use
thread local storage for measures.  With stats working again wterl
uses them to determine who to evict.  Wterl's signature calculation
for an operation wasn't correct and so the cache wasn't efficient at
all, this has been fixed.
 2013-06-25 13:31:43 -04:00
Gregory Burd
a3c54b1610 Cleanup a bit. 2013-06-19 14:54:27 -04:00
Gregory Burd
0f180a6531 Fixed a few mistakes. 2013-06-19 14:37:30 -04:00
Gregory Burd
450299dc2d Comment out a test that's not yet working. 2013-06-18 16:40:43 -04:00
Gregory Burd
4ae8ffb4cd Update debugging messages a bit. Fix a bug in the signature function. 2013-06-18 13:49:25 -04:00
Gregory Burd
34e88c9234 Add some debugging output. 2013-06-18 13:12:10 -04:00
Gregory Burd
50e24d0f48 Add a longer, multi-table test. Use a release version of WiredTiger from now on. 2013-06-18 09:21:58 -04:00
Gregory Burd
53307e8c01 A great deal of cleanup. EUnit and EQC tests pass. 2013-06-14 16:57:53 -04:00
Gregory Burd
ff7d1d6e20 WIP: further simplifying context cache 2013-06-14 10:52:45 -04:00
Gregory Burd
7952358781 WIP: cache wasn't returning items found 2013-06-12 09:08:09 -04:00
Gregory Burd
4460434db1 WIP: remove potential for infinite loops with CAS and fix a few issues in async 2013-06-12 08:09:51 -04:00
Gregory Burd
110b482962 Some paranoia and a few fixes 2013-06-11 12:13:06 -04:00
Gregory Burd
8ea866bf20 Logic inversion on CAS() operation. 2013-06-10 16:18:55 -04:00
Gregory Burd
2a4b8ee7d2 WIP: simplify the cache from hash-of-lists to list; use a CAS() operation to protect the most-recently-used (mru) list. 2013-06-10 14:31:59 -04:00
Gregory Burd
b2c0b65114 Fixes for OS/X and whitespace cleanup. 2013-06-06 15:16:50 -04:00
Gregory Burd
0fef28de92 WIP: basho_bench tests are running fine now, need more work to ensure cache is functioning properly. 2013-06-05 11:41:41 -04:00
Gregory Burd
778ba20352 WIP: a bit of cleanup fixes a few mistakes 2013-06-04 17:21:50 -04:00
Gregory Burd
f1b7d8322d WIP: replcae the kbtree with khash, we don't need the tree features (yet, if ever) and hash is faster; add a most-recently-used stash for contexts as it's highly likely that worker threads will do many operations with the same shape/signature of session/cursors/tables/config and that path can be lock-free as well making it much faster (one would hope); somewhere something is stepping on read-only ErlNifBinary data and so a crc check is failing and causing the runtime to abort, that's the latest item to find/fix. 2013-06-04 14:45:23 -04:00
Gregory Burd
b002294c4e WIP: all tests (but drop) passing again, need to fix that and valgrind next. 2013-05-31 20:32:02 -04:00
Gregory Burd
9468870e1f WIP: use a log2 histogram to track how long items live in the cache, evict items which have been in cache greater than the mean time. 2013-05-30 17:10:51 -04:00
Gregory Burd
15fbc71ea7 WIP: pieces in place, need to work out the kinks now. 2013-05-30 14:21:34 -04:00
Gregory Burd
f0d5baeb0e WIP: more caching work, still not done. 2013-05-29 14:57:35 -04:00
Gregory Burd
a2cd1d562c WIP: devising a better way to cache/reuse session/cursor pairs. 2013-05-28 16:14:19 -04:00
Gregory Burd
013251e6d9 Update WiredTiger statistics less frequently and fix a build script 
mistake preventing checking out by branch name.
 2013-05-14 13:51:44 -04:00
Gregory Burd
786142ce73 Add a bit of statistics tracking for two reasons, a) to help inform 
where a request should be enqueded and b) to track request latency.
 2013-05-01 22:02:37 -04:00
Gregory Burd
ae64a5e26f Move async nif struct definition back to where it belongs. 2013-05-01 22:02:21 -04:00
Gregory Burd
d1d648af1e Now matches against proper library name and prevents rebuilding when unnecessary. 2013-05-01 22:00:43 -04:00
Gregory Burd
7c9c3b78db Remove this vestige of bzip2 support. 2013-05-01 21:59:54 -04:00
Gregory Burd
12fd9134fa Add a few placeholders for missing functions. 2013-05-01 21:59:12 -04:00
Gregory Burd
d93f0bc162 A more friendly approach to make clean in c_src. 2013-04-26 18:57:17 -04:00
Gregory Burd
bfc180eeb3 Merge branch 'master' of github.com:basho-labs/wterl 
Conflicts:
	c_src/wiredtiger-build.patch
 2013-04-26 10:37:06 -04:00
Gregory Burd
f1cb6f8c61 Remove support for bzip2. Change build_deps.sh to only remove the 
build files from wiredtiger when cleaning rather than all the files.
 2013-04-26 10:34:57 -04:00
Gregory Burd
eafee02865 Only start 2 * num_queues worker threads initially. num_queues is generally 
equal to the number of cores reported by Erlang (info.scheduler_threads) which
is either determined automatically by the Erlang BEAM runtime or via the +S
flag.  The minimum num_queues is 2, so the minimum number of workers is 4.  The
maximum number of workers is ASYNC_NIF_MAX_WORKER_QUEUE_SIZE (currently set to
128), but that would only happen if there were 64 cores (or you set +S 64:64 at
startup).
 2013-04-26 10:15:15 -04:00
Gregory Burd
c410312c40 Switch to the develop branch. 2013-04-26 10:07:10 -04:00
Gregory Burd
3a51afcc20 Copy paste error in patch file prevented proper build arguments for bzip2. 2013-04-25 16:36:17 -04:00
Gregory Burd
422dcfda89 Return 'eagain' when request queue is full and then try the request again. 
In the worst case is the request queue remains full and we loop between
the NIF and Erlang forever trying over and over to enqueue the request. If
that happens we shouldn't take schedulers offline as the NIF calls are fast
and we shouldn't run out of memory as that is bounded.  CPU will show a lot
of activity, but progress will continue in Erlang.
 2013-04-25 15:18:23 -04:00
Gregory Burd
6b393ac47c Keep allocated req and ErlNifEnv around for reuse rather than re-alloc'ing them on each request should save us some overhead on the hot path. 2013-04-25 11:30:11 -04:00
Gregory Burd
46243ff54a Remove debug compiler flags. 2013-04-23 15:00:44 -04:00
Gregory Burd
fc59ae3f90 WIP: Finished transitioning to a gen_server and a few other tasks. This works 
except that when trying to enif_send a bunch of messages eventually one causes
a segv just after reporting: size_object: bad tag for 0x80
 2013-04-22 17:45:48 -04:00
Gregory Burd
652771003e WIP: a good start, I need to switch over wterl_event_handler to be a 
gen_server and I need to add a way to set the pid of the message handler
process to the NIF API.
 2013-04-22 09:52:21 -04:00