greg/stasis-bLSM
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
174 commits 1 branch 0 tags 670 KiB
Commit graph

35 commits

Author SHA1 Message Date
sears
bded48a306 add new server implementation that uses FILE to buffer I/O, and handles a smaller, fixed number of connections -- leads to an almost 4x speedup on no-ops on nehalem 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@986 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-08-11 22:46:55 +00:00
sears
569b46363b moved most of the request dispatching logic from logserver.cpp to requestDispatch.cpp 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@985 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-08-11 17:43:39 +00:00
sears
2a45c2cda8 add no-op network call, and a simple benchmark that exercises it 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@980 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-08-09 20:21:41 +00:00
sears
7bed4b67ba fix a bunch of 64-bit integer signedness bugs 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@940 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-07-16 21:43:21 +00:00
sears
6eecf1557a restore support for concurrent disk reads; logserver was accidentally using a findTuple function that assumed diffs. This commit uses rwlc, and requires Stasis -r1393 or better 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@814 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-05-27 01:49:27 +00:00
sears
2a1157602a Rewrite merge thread synchronization code, update status messages, implement preliminary (and ineffective) admission control, and force write merged data every megabyte (so that prograess can be tracked by admission control. Also, refactor quite a few things. 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@805 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-05-19 23:42:06 +00:00
sears
2f15673c80 pass sequential I/O hints to stasis 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@773 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-04-12 20:56:54 +00:00
sears
e1c937a602 Refactoring of logtable. Cleaned a bunch of includes, and logtable is now a template. This is in preparation for the new merge policy 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@698 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-03-17 21:51:26 +00:00
sears
c8c48a188d more refactoring; all iterators now live in their respective tree components, fixed some hardcoded parameters; encapsulate allocators inside of diskTreeComponent 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@689 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-03-13 00:05:06 +00:00
sears
5f0d5c4f97 use RegionAllocator for InternalNodes and DataPages 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@685 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-03-09 23:17:03 +00:00
sears
8d7199676c made diskTreeComponent more object oriented; many of its functions are now private 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@684 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-03-09 22:18:55 +00:00
sears
01c65d7a3c diskTreeComponent -> diskTreeComponent::internalNodes 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@680 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-03-09 19:47:12 +00:00
sears
156c015b84 fix boundary condition in histogram code 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@674 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-03-09 17:03:18 +00:00
sears
452ee97442 diskTreeComponentIterator -> diskTreeComponent::iterator 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@671 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-03-09 00:09:38 +00:00
sears
06fa7999cb c++-ify diskTreeComponentIterator 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@667 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-03-05 22:21:44 +00:00
sears
632afe6d9c cleanup statistics gathering and reporting. 
constify inputs to merge functions (so they can't call freetuple())


git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@662 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-03-05 19:07:47 +00:00
sears
1243d999a5 Hacky histogram generation; API's for client-side iterators. 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@656 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-03-03 23:20:05 +00:00
sears
747f4df1f3 Reopening existing trees now seems to work (no thorough tests yet). Added some network opcodes for statistics and debugging (not all are implemented yet). Reduce verbosity of server and client library. 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@654 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-03-01 21:26:07 +00:00
sears
7c2397340c Fixed a bunch of iterator bugs and racy merges. Added drop_database utility. 
Bugfixes:
 - Atomically deallocate regions and update the logstore object
 - Proactively invalidate iterators after each merge (before, it would simply set a not-valid bit.  This doesn't work because iterators hold page pins, which breaks force-writes)
 - Clarify semantics of opening iterators mid-stream:  All calls now return iterators that return first key >= the requested one.  revalidate() needs the key > the requested one, so it calls peek(), then (if necessary) getnext().
 - Add asserts to check that the header is latched at update, and that tuples returned by iterators are strictly monotonically increasing'
 - Improve error handling in network.h  We still get (and terminate on) SIGPIPE.

Refactoring:
 - Add dispatch function to network.h.



git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@620 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-02-25 01:29:32 +00:00
sears
abce19c67e Fixed server hang. It used to sleep forever inside select if all the clients hung. Now, it uses the self pipe trick to wake itself up. 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@614 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-02-23 23:54:44 +00:00
sears
6a2d6a1cd4 Generalized comparator, fixed some bugs in record length handling. 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@613 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-02-23 17:05:47 +00:00
sears
63a14d8509 network iterator API is now working (though it could use a better test suite) 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@601 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-02-20 01:18:39 +00:00
sears
57c9afc8d8 rename logtree to diskTreeComponent 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@591 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-02-19 00:59:14 +00:00
sears
7738b67bc0 Implemented network iterator protocol. It hasn't been tested or hooked up to anything because logstore iterators don't exist yet. 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@589 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-02-19 00:34:41 +00:00
sears
ed9d4fc377 Encapsulate allocator state inside logtree. Remove redundant information from merger_args. Rewrite merge threads. This seems to fix the double free issues, and gets rid of disk space leaks. It's probably leaking ram, and does not keep the logstore headers up to date. 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@587 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-02-18 23:31:57 +00:00
sears
5032e77b6c more allocator cleanups (still leaks space). Added dump_blockmap utility 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@578 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-02-17 22:11:22 +00:00
sears
e2ea48d233 remove a bunch of whitespace / comments. clean up network thread. add constant for dumping the block map 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@570 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-02-16 23:15:29 +00:00
sears
d769bc0de6 fix compiler warnings. This commit makes use of an API change, and requires Stasis >= r1352 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@561 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-02-11 21:38:05 +00:00
sears
940a6da6fe Reworked memory allocation and network protocol. This gargantuan commit also reduces the default size of C0, and removes quite a bit of redundant logic and error handling. 
The tests now pass, except that check_merge never terminates (it takes too long) and check_mergelarge still is not passing.

For better luck running this version of the code, turn off stasis' concurrent buffer manager.  We're doing something bad that leads to deadlocks with the concurrent buffer 
manager.  Another (the same?) bug less-frequently leads to page corruption with the old stasis buffer manager.




git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@556 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-02-10 21:49:50 +00:00
sears
d5b4106b7e silence verbose messages 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@549 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-02-04 22:19:13 +00:00
sears
1725b6569d backport client code from ydht load generator 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@543 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-02-02 23:50:16 +00:00
sears
3cbcb1e274 check_tcpclient now works (sort of...) 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@541 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-02-02 22:50:09 +00:00
sears
23359c2570 fix more compiler warnings 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@531 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-01-27 23:34:33 +00:00
sears
2f562a175a remove more duplicated code 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@528 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-01-26 20:19:36 +00:00
sears
d016498f8d initial import; removed cruft from mert's tarball, tweaked make's clean targets 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@520 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-01-23 02:13:59 +00:00