greg/stasis-bLSM
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
249 commits 1 branch 0 tags 670 KiB
Commit graph

249 commits

This branch
This branch
All branches
Author SHA1 Message Date
sears
ae7c827ab5 add hacky garbage collection mechanism 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@2484 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2011-04-28 21:56:25 +00:00
sears
c910a7af8f fix latching bug in scan 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@2483 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2011-04-28 20:18:54 +00:00
sears
e81b8522d7 fix memory leaks 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@2457 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2011-04-25 15:28:45 +00:00
sears
36790f6c2c add comment 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@2455 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2011-04-24 21:09:54 +00:00
sears
5010688123 fix compiler warning on 32 bit architectures; 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@2453 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2011-04-23 18:53:38 +00:00
sears
5be84f59e8 add support for early tombstone dropping, extend datatuple format to support timestamps 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@2451 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2011-04-22 22:54:49 +00:00
sears
9672bc4f48 add commented out debugging code 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@2450 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2011-04-22 17:41:13 +00:00
sears
5343f3021e fix broken merge logic. it was sometimes dropping live tuples, and never dropping tombstones... 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@2449 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2011-04-22 17:40:37 +00:00
sears
4f74dfd45f documentation cleanup 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@2448 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2011-04-22 17:39:52 +00:00
sears
38f1793eaa fix argument parsing typo 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@2447 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2011-04-22 17:39:28 +00:00
sears
ab9cfd1443 add support for timestamps to datatuple comparator 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@2446 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2011-04-22 17:38:18 +00:00
sears
65efc768a4 forgot to commit change_log_mode utility 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@2438 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2011-04-21 00:53:23 +00:00
sears
c42a275ab4 fix shutdown bug; add debugging utility that allows the log to be enabled / batched / disabled at runtime 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@2437 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2011-04-21 00:28:05 +00:00
sears
8135bbcc2e misc log cleanups; add command line parameter to newserver to control log mode 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@2435 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2011-04-20 21:51:04 +00:00
sears
0508f6cb40 initial wal implementation 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@2434 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2011-04-20 20:17:26 +00:00
sears
e5058da393 fix merge deadlock; port to fixed stasis buffer manager api; enable new buffer manager by default 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@2330 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2011-03-25 20:05:49 +00:00
sears
0a0646e75e ignore sigpipe 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@1713 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2011-01-25 02:07:16 +00:00
sears
95c7290ac6 import eclipse project goo 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@1712 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2011-01-25 02:06:42 +00:00
sears
f4936551bc optimize datatuple in-memory layout 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@1590 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2011-01-11 22:08:10 +00:00
sears
975410845f add empirically determined estimators of c0 size, and pessimistic ones that are the default 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@1589 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2011-01-11 22:07:39 +00:00
sears
abe7e36640 add comment explaining how to enable stlslab froms stasis 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@1588 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2011-01-11 22:06:18 +00:00
sears
e674f4459a whitespace 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@1587 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2011-01-11 22:05:52 +00:00
sears
dad55b564c Misc statistics cleanup; improved merge cost estimations during backpressure. 
Misc statistics cleanups include:
 - fixes for corner case where c0 is empty, and it would divide by zero
 - always compute number of tuples (in / out, but not base)
 - rework amount of memory allocated when --test and --newserver are passed in.



git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@1582 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2011-01-11 19:24:16 +00:00
sears
42b52a4d17 fix datapage padding behavior; the old padding overhead was ~ 100%; now it is much lower 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@1522 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-12-17 21:16:43 +00:00
sears
83fe4b7703 fix array-out-of-bounds bug in unit test 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@1521 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-12-17 21:15:26 +00:00
sears
a6a0ce3f54 extended stats cleanup, verbose mode, documentation 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@1518 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-12-17 18:48:26 +00:00
sears
052508fb76 remove mergeStats->current_size field, which is derivable from other fields in mergeStats 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@1508 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-12-15 00:15:59 +00:00
sears
cd27f50baf remove ltable->tsize and ltable->tree_bytes fields 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@1507 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-12-14 23:06:21 +00:00
sears
7e2d12f6ba fix typo in table of response code constants 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@1505 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-12-14 21:43:50 +00:00
sears
73e72b47b2 more stats cleanups 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@1493 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-12-14 01:49:23 +00:00
sears
43425128ba clean up initialization / marshalling code. Lets some of the public mergeStat fields to be protected 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@1490 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-12-14 00:06:32 +00:00
sears
eb4446f409 readability fix: fields in mergeStats that are only used for computing extended stats for pretty print are now prefixed with "stats_", and can be optionally disabled at compile time 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@1489 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-12-13 22:27:13 +00:00
sears
7e44641799 move c1-c2 backpresure to linear backpressure model; removes a few hundred lines of fussy code, and should enable further simplifiactions 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@1488 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-12-13 21:17:35 +00:00
sears
347a4126f3 remove the NO_SNOWSHOVEL c-preprocessor stuff. There is no way it was working anyway... 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@1480 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-12-11 01:04:33 +00:00
sears
a00531ae6d simplified merge_scheduler 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@1479 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-12-11 00:51:19 +00:00
sears
8dddc7e168 remove line from unit test that had no effect, but broke compilation 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@1478 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-12-10 20:42:57 +00:00
sears
a8e905c0d6 fix deadlock on shutdown; be more stingy with bloom filter RAM with small indexes, add bloom filter to c2, default back to old buffer manager 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@1468 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-12-08 19:49:13 +00:00
sears
79936da79b loosen some scheduling parameters 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@1467 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-12-08 19:46:55 +00:00
sears
e9b68056b8 whitespace 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@1466 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-12-08 19:18:05 +00:00
sears
fbdc900623 fix #include mistake 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@1239 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-10-05 18:27:56 +00:00
sears
ccf3895595 conservatively make the bloom filter really big (should behave well down to 100 byte keys, waste ram otherwise...) 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@1238 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-10-05 16:55:29 +00:00
sears
31cafbef6c get rid of -Wno-variadic-macros since it suddenly broke compilation for some reason 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@1237 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-10-05 16:54:27 +00:00
sears
8936df750a fix includes to avoid linker errors 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@1236 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-10-05 16:52:32 +00:00
sears
0c5700257c re-re-re-...re-disable new buffer manager optimizations 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@1235 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-10-05 16:50:39 +00:00
sears
b9dd11a7e4 reduce default datapage_size to 1 (4096) to avoid useless seeks for small values 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@1234 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-10-05 16:23:31 +00:00
sears
f9a1f411e1 initial checkin of bloom filter logic for lsm trees 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@1233 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-10-03 23:05:45 +00:00
sears
3619e86271 initial checkin of bloom filter 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@1232 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-10-03 18:06:19 +00:00
sears
6eaf5da8de port to newer gcc/g++ 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@1231 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-10-02 12:52:41 +00:00
sears
4c0e8a7186 change cerr to log4cpp 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@1204 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-09-29 17:58:34 +00:00
sears
e5518eac26 enable debugging 
git-svn-id: svn+ssh://svn.corp.yahoo.com/yahoo/yrl/labs/pnuts/code/logstore@1178 8dad8b1f-cf64-0410-95b6-bcf113ffbcfe
 2010-09-22 19:28:26 +00:00