greg/stasis-aries-wal
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
1516 commits 3 branches 1 tag 14 MiB
Commit graph

281 commits

Author SHA1 Message Date
Sears Russell
c30aaa3a1a fix compiler warning; -1 is not comparable to entry->type 2010-01-07 21:50:17 +00:00
Sears Russell
1af589dd18 remove malloc and memcpy from the in_memory_log read path 2009-12-31 23:25:56 +00:00
Sears Russell
1409a9eb27 more refactoring; log handles now manage their log entries memory 2009-12-31 23:01:37 +00:00
Sears Russell
a5788688c8 refactor getUpdateArgs; remove a bunch of casts, fix a bunch of warnings 2009-12-31 20:20:47 +00:00
Sears Russell
9aa1d9d652 pull memcpy out of allocUpdateEntry() 2009-12-31 19:30:40 +00:00
Sears Russell
32e40c949d added log modes to sequentialThroughput benchmark, added methods for concurrent appends to log api (and stubbed them out) 2009-12-31 19:02:34 +00:00
Sears Russell
7ff80f37ef add deferred, end of transaction callbacks. addresses issue 15 2009-12-15 06:30:37 +00:00
Sears Russell
efc6a0eabd add missing function declaration 2009-12-10 01:53:15 +00:00
Sears Russell
cd8c63889d clarify documentation 2009-12-09 19:55:27 +00:00
Sears Russell
83a0ec68fe fix rose bit rot, have unintialized page clear the impl pointer (for tidyness), remove some resolved @todos. 2009-12-04 19:33:09 +00:00
Sears Russell
7d1b702239 revert part of opensolaris patch--pack recordids again 2009-11-30 01:01:30 +00:00
Sears Russell
2ff421bff0 initial opensolaris port 2009-11-29 03:22:36 +00:00
Sears Russell
2535f69fbc switch the concurrent hash to use lru->getStaleAndRemove; fix crashes due to partial leak of thread local pages on thread shutdown 2009-11-24 01:43:36 +00:00
Sears Russell
38b999dfed add missing stlredblack.h header 2009-11-24 01:41:55 +00:00
Sears Russell
675bdce24a wrote some wrappers that emulate redblack; default to stl tree for now 2009-11-24 00:24:18 +00:00
Sears Russell
1b0c10b37b clean up handling of uninitialized pages (removes some special case code) 2009-11-09 23:02:27 +00:00
Sears Russell
795e044ab9 documentation fix 2009-11-09 22:58:37 +00:00
Sears Russell
b4a0f2436b concurrent lru implementation for new buffer manager 2009-11-09 20:53:05 +00:00
Sears Russell
8bd0616f97 new concurrent buffer manager 2009-11-09 19:50:00 +00:00
Sears Russell
f5e3ab2d91 concurrent hashtable seems to be working; still need to work out atomicity primitives (eg; options to leave the bucket locked on success) 2009-10-19 21:55:25 +00:00
Sears Russell
1cf16f62f2 add optional support for gcc atomics to transaction table 2009-10-15 22:46:25 +00:00
Sears Russell
e6a718a7f9 improved concurrency for transaction table 2009-10-15 20:11:43 +00:00
Sears Russell
331f680c09 partial rewrite of transaction table to reduce thread contention 2009-10-15 02:33:08 +00:00
Sears Russell
4565aff9de more refactoring; no more globals in transactionTable.c 2009-10-14 21:22:50 +00:00
Sears Russell
dbe3ecf0d0 refactoring transaction table; moved implementation into its own file. still need to remove globals 2009-10-14 18:57:50 +00:00
Sears Russell
20ef74b104 use pthread_rwlock instead of rwlock userspace emulation of it 2009-10-13 00:29:35 +00:00
Sears Russell
20a3fe9df5 add support for soft commits, and making prior soft commits durable 2009-10-12 19:16:41 +00:00
Sears Russell
942ba7fffc bugfixes + support for hard limits on log size 2009-10-08 22:48:58 +00:00
Sears Russell
6a02f2f159 buffer manager implementation is now determined by an overridable factory method 2009-10-05 22:39:09 +00:00
Sears Russell
24650fa190 buffer manager refactoring; started work on new dirty page table writeback interface (to avoid circular dependencies) 2009-10-05 21:19:01 +00:00
Sears Russell
4689582cb5 improve log performance by disabling O_SYNC by default (and use fdatasync instead) 2009-09-12 16:20:13 +00:00
Sears Russell
c4e80b41b5 more include cleaning on alloc, begin cleaning of bufferHash 2009-08-22 21:29:27 +00:00
Sears Russell
1ce1496226 dependency cleanups; removed compenastions from alloc.c 2009-08-22 00:01:02 +00:00
Sears Russell
94c542f8b1 Increase dirtyPageTable concurrency 
Add special-casing of lhtable; special case for buffer manager.
 2009-08-14 06:31:13 +00:00
Sears Russell
dd0bcae074 add thread synchronization logic to ensure that only one thread is performing writeback at a time. This fixes weird I/O interleavings on some machines 2009-08-13 23:18:49 +00:00
Sears Russell
0154f7d29d IMPORTANT API CHANGE: stasis_dirty_page_table_flush_range() no longer force writes to disk. You now need to call forcePageRange() after flush page range returns. 
This commit should significantly improve sequential write thoughput.

I found a segfault bug this morning, but cannot reproduce.  Could be a heisenbug, or could have been a hardware fault (gojira repeatedly kernel paniced, then both problems went away...)
 2009-08-13 05:53:48 +00:00
Sears Russell
0cc65aefaf Remove LRU-ordered writeback. Instead, writeback pages sequentially once some fraction of the page file is dirty. Also. further simplifies writeback code. 2009-08-11 17:23:43 +00:00
Sears Russell
907a24b68e dirty page table flush was pinning pages, then asserting they were not pinned; updated page writeback api to allow correct usage 2009-08-08 01:43:03 +00:00
Sears Russell
c92ee87c32 improved behavior for sequential I/O performance, fixed races during writeback 2009-08-07 18:27:52 +00:00
Sears Russell
a6aa8ed942 added method to access dirtyPageTable 2009-08-05 20:02:33 +00:00
Sears Russell
5b10bcc63b Reimplemented allocationPolicy.c; updated API + added more cases to the unit tests. 2009-08-05 06:35:11 +00:00
Sears Russell
c961b34903 forgot to commit b-tree source files 2009-07-29 21:04:08 +00:00
Sears Russell
ebd6b14dbf single-page btrees are working 2009-07-29 21:03:16 +00:00
Sears Russell
0a308f035c more header cleaning; updated INSTALL 2009-07-26 19:47:05 +00:00
Sears Russell
081c61b414 cleaned up includes, autoconf is now optional, since cmake now creates a config.h 2009-07-26 18:51:45 +00:00
Sears Russell
260310e151 Add page ops for tree implementations. 2009-07-26 06:18:54 +00:00
Sears Russell
7f2edd880d New (more thorough, less frequent) tests for slotted pages. Renamed slotted methods, added microbenchmark for slotted pages. 2009-07-20 17:42:43 +00:00
Sears Russell
9ef3adf40c - Fix compile warning + broken fcn pointer from last commit. 
- Address crashes in Tupdate() and recovery caused by mixing segments and page oriented recovery  (add loadPageForOperation() call)
 2009-07-16 03:05:32 +00:00
Sears Russell
b44f8b17b3 Add getCachedPage() call. This allows dirtyPageTable to write back pages with out accidentally reading them back in from disk with loadPage(). This should improve performance and allow loadPageOfType() to be used safely. 2009-07-14 07:46:47 +00:00
Sears Russell
4493dbb88b Added support for segment based recovery. For now, Stasis doesn't support hybrid recovery, so mixing page deallocation and segments will make recovery corrupt data / segfault. 2009-07-13 17:18:01 +00:00
Sears Russell
d6c91241d6 fixed bug that caused loadPageOfType to ignore type parameter. Recovery is still broken, since it calls loadPage, even when it can't 2009-07-10 01:19:44 +00:00
Sears Russell
eb0661ceeb added segment.c 2009-07-09 21:09:36 +00:00
Sears Russell
ade1dc896e Add loadPageOfType() call. (Untested for now) 2009-07-07 21:33:49 +00:00
Sears Russell
cf49c7dbf1 forgot to svn rm indirect's files 2009-07-07 20:05:15 +00:00
Sears Russell
2593de7a2c Move page type to Page_s struct. This breaks indirect.c (which is poorly thought out, and will be removed shortly), and was the next step on the road to segment-based recovery 2009-07-07 19:20:22 +00:00
Sears Russell
b66328192c in-memory group by implementation 2009-07-01 01:33:24 +00:00
Sears Russell
5efebbbb10 fix doxygen warning 2009-06-28 19:01:13 +00:00
Sears Russell
09af29abad clean up iterator interface; remove Titerator_releaseLock() 2009-06-28 18:54:57 +00:00
Sears Russell
120665ae53 cleaned up arrayList. stasis_record_write no longer takes an LSN (it was ignoring it anyway). Added some missing calls to stasis_page_lsn_write (each time a page is written to, stasis_page_lsn_write should be called; this happens automatically for operations called via tupdate 2009-06-02 18:25:35 +00:00
Sears Russell
25c6be64b4 add assertunlocked() call 2009-05-30 02:22:04 +00:00
Sears Russell
7db06197ca refactored truncation + dirty page table, fixed doxygen warnings. 2009-05-20 21:23:51 +00:00
Sears Russell
73c4b4fb71 Fix for Issue 1. Reworked log format so that nested top actions do not need to look up old entries during forward operation. This saves log bandwidth and avoids the bug that caused the performance regression 2009-05-14 21:28:09 +00:00
Sears Russell
3ff75ff568 made pass over log documentation 2009-05-13 22:06:58 +00:00
Sears Russell
87218ad929 yet another refactoring; broke groupForce into its own module, removed more static variables and namespace violations 2009-05-13 18:04:53 +00:00
Sears Russell
bc554efc85 renamed some functions to match naming convention 2009-05-08 06:53:30 +00:00
Sears Russell
651a1a22e5 more refactoring; removed static variables from pageHandle.c 2009-05-08 04:56:34 +00:00
Sears Russell
221b2e4acb remove static variables, fix filehandle leak 2009-05-07 09:56:36 +00:00
Sears Russell
7b4cf40221 refactored/created stasis_buffer_manager_open and stasis_handle_open 2009-05-07 08:52:06 +00:00
Sears Russell
b5b414490a fix config.h error for installed copies of stasis 2009-04-29 21:31:39 +00:00
Sears Russell
55c43358b6 fix recovery bug due to interactions between truncation and the old CLR format. check_faultInjection should now pass. 2009-04-22 22:03:38 +00:00
Sears Russell
731da7456a more eclipse refactoring; cleaned up some names, whitespace and documentation 2009-04-13 22:50:12 +00:00
Sears Russell
7db35ecd39 removed static variables from bufferPool.c; renamed methods, moved Page typedef to common.h 2009-04-13 21:55:08 +00:00
Sears Russell
4b07b538a6 Renamed some methods; fixed bug that caused recovery to create potentially unbounded numbers of concurrent, active transactions. 
(Note: this commit contains quite a few instances of auto-cleaned whitespace....)
 2009-04-11 17:17:42 +00:00
Sears Russell
05077b62fd nested top action cleanup and bugfix; subsequent recovery calls could undo logical undos in some circumstances 2009-04-03 22:01:37 +00:00
Sears Russell
fba041469f Added macro that provides id numbers for third-party Operations. 2009-04-03 22:00:27 +00:00
Sears Russell
83a982d0bf fail earlier if record_type and record_size apis are abused 2009-04-01 21:12:50 +00:00
Sears Russell
564dfe426b Renamed / refactored operation implementation + registration functions. Added support for third party operations 2009-03-31 05:02:54 +00:00
Sears Russell
83ad15b6d0 more benchmarks; transactions can now be multithreaded 2009-03-20 11:36:27 +00:00
Sears Russell
d413bb27f3 support for lsn-free writeback; partial benchmark code for submission 2009-03-19 03:36:13 +00:00
Sears Russell
b1f44ab005 added simplistic support for log reordering 2009-03-16 08:11:49 +00:00
Sears Russell
45a2410a25 add support for lsn-free set (but not alloc, etc...) 2009-03-15 07:21:37 +00:00
Sears Russell
bd2015443f expose nextAvailableLSN for lsn-free pages 2009-03-15 02:43:09 +00:00
Sears Russell
9e54ad60d5 renamed hash() to avoid linker collisions; inlined some hash functions 2009-02-20 23:42:54 +00:00
Sears Russell
95e9158847 checkin partially implemented log based on file pool so that svn will compile 2009-01-05 21:57:33 +00:00
Sears Russell
93885a3ab4 removed static variables from logWriter.c 2008-12-29 23:05:13 +00:00
Sears Russell
5ba41bb3cb renamed logWriter.[ch] to safeWrites.[ch] 2008-12-29 20:01:37 +00:00
Sears Russell
6f809b00e2 renamed open/deleteLogWriter 2008-12-29 19:46:51 +00:00
Sears Russell
1f336255f5 fix shutdown of empty rose trees 2008-12-05 20:52:39 +00:00
Sears Russell
32950b2b98 fixed memory leaks and pointer aliasing bugs in lsmTree_iterator. 2008-12-05 00:08:46 +00:00
Sears Russell
79d1512a5e fixed problem in lsm tree recovery; it was not updating the lsm tree header when passing C1 into C2's merger thread. Then C2 freed C1, and C1 pointed to uninitialized data. 2008-12-04 09:22:17 +00:00
Sears Russell
e4341e10b9 partially fixed rose recovery; clean shutdowns now work 2008-12-04 04:12:47 +00:00
Sears Russell
ff641be5dc add an explicit ID parameter to StaticTuple. Instantiators of StaticTuple need to make sure this is different for each static tuple type that may coexist in the same page file. This allows multiple copies of rose to run at once. 2008-12-02 21:45:58 +00:00
Sears Russell
c6564e67c7 documentation update; eliminated some dead code 2008-12-02 19:39:20 +00:00
Sears Russell
2dcc6598d2 added support for softcommit, and for compile / runtime configuration of stasis log + storefile locations 2008-12-01 22:45:32 +00:00
Sears Russell
17d4bb3ed4 log refactoring 2008-12-01 19:48:59 +00:00
Sears Russell
d9b00d457d Remove old comment. 2008-12-01 04:29:42 +00:00
Sears Russell
41460d041a did away with memoization for rle_ (cost of mutex was too high). pthread_[get|set]specific does not provide enough keys to make sense as an alternative 2008-11-27 00:08:59 +00:00
Sears Russell
fe6c865737 support multiple include 2008-11-26 22:37:49 +00:00
Sears Russell
549f97d297 rewrote mergeThread, added ability to start iterators mid lsmTable 2008-11-26 07:14:23 +00:00
Sears Russell
1985ec6424 defensively check that caller locked pages as appropriate 2008-11-26 07:12:51 +00:00
Sears Russell
b8624d09b1 obtain readlocks on lsm tree pages while iterating over them 2008-11-26 07:05:22 +00:00
Sears Russell
6a5e7f5589 Fix two problems in lsmTree: 
- rewrite lookup() to be simpler, and fixed a few bugs
 - was keeping node records around after pages were released
 2008-11-26 07:01:43 +00:00
Sears Russell
9012bef5fc Fixes nasty concurrency bug; rle's readRecord() was not reentrant(!) 2008-11-26 06:58:22 +00:00
Sears Russell
11082997d8 revert accidentally committed files 2008-11-24 23:43:26 +00:00
Sears Russell
a321ba6e4e handle null lsmIiterator_ gracefully 2008-11-24 23:41:11 +00:00
Sears Russell
2b63991014 preliminary (and largely untested) support for opening LSM-tree iterators starting at a given tuple 2008-11-24 01:32:35 +00:00
Sears Russell
cafd05f9bd moved legacy buffer manager into its own directory 2008-11-13 20:08:37 +00:00
Sears Russell
53a7982f47 fixes numerous blob bugs, allowing linearHashNTA to store arbitrary length key,value pairs. 2008-11-13 04:18:50 +00:00
Sears Russell
46fbfa3c35 Implement blob deallocation; add test case 2008-11-13 00:13:11 +00:00
Sears Russell
e2f604175b move hash iterator code around, update documentation 2008-11-13 00:12:19 +00:00
Sears Russell
dcf07c1d4c Remove obsolete #include 2008-11-13 00:08:27 +00:00
Sears Russell
f5d1229941 Recovery now maintains XactionTable's state in the same way as forward operation. 
Internal transactional2.c state is now declared static.
 2008-11-13 00:06:35 +00:00
Sears Russell
fc5b5a2b29 do away with librw 2008-11-07 03:44:41 +00:00
Sears Russell
eb6ade59e4 fix doxygen warning 2008-11-06 21:06:55 +00:00
Sears Russell
794edb07d5 updated documentation 2008-11-06 07:19:02 +00:00
Sears Russell
2c2c603dd2 all naive linear hash methods now start with TnaiveHash; removed references to ThashAlloc from documentation 2008-11-03 21:42:42 +00:00
Sears Russell
0a551720c8 added optional template arguments to FOR and RLE to let callers specify the size of delta_t and copy_count_t, resepectively. 2008-10-27 23:35:51 +00:00
Sears Russell
0ff6196776 documentation fix 2008-10-27 23:33:22 +00:00
Sears Russell
a639d461b8 fail a bit earlier on error 2008-10-27 23:32:45 +00:00
Sears Russell
c7ba4dfb62 forgot to commit header for new allocation policy function 2008-10-27 23:30:53 +00:00
Sears Russell
470c52c9cc add compile time PACK_STATS option that displays statistics about the number of bytes used per column 2008-10-27 23:27:48 +00:00
Sears Russell
a2a8c73cb5 prevent rose from initiating lots of small merges when the writer process is slow 2008-10-27 23:26:16 +00:00
Sears Russell
5c5b6d0c67 allow callers to truncateNow() to (safely) override normal truncation policy 2008-10-27 23:24:21 +00:00
Sears Russell
108d40ec16 fix warnings on 32-bit platforms 2008-10-09 19:00:17 +00:00
Sears Russell
132a567533 removed dead/unnecessary code; private methods are now static 2008-10-04 07:08:35 +00:00
Sears Russell
c8d368f4b1 Don't include btree header... 2008-10-04 01:37:31 +00:00
Sears Russell
ef487c4dc8 remove btree stub files 2008-10-04 00:53:11 +00:00
Sears Russell
aee7e00c99 Fixed doxygen warning 2008-10-03 17:10:54 +00:00
Sears Russell
6354fe8a2f use pageid instead of int; Tupdate now takes a pageid instead of a recordid 2008-10-03 02:42:25 +00:00
Sears Russell
cf8eb2ca15 fix doxygen warnings 2008-10-01 20:32:02 +00:00
Sears Russell
db1d79efcd Removed Tinstant* and broken logical hash stuff 2008-09-30 15:34:42 +00:00
Sears Russell
ff0887624c Simplified the page locking conventions. 
Added paranoid calls to assertlock() to page.c

Fixed race in abort():

 - pick CLR LSN

 - release page lock

 - someone else updates page

 - lock page

 - apply undo
 2008-09-28 05:50:59 +00:00
Sears Russell
6d17442380 Major refactoring: 
- Changed operations to take only two arguments
     - No more hacks regarding log argument sizes

     - Set pageid = INVALID_PAGE if you want a logical operation

  - Ported operation implementations to new api; exposed + fixed a number of concurrency bugs

  - More fixes to prepare

  - Fixed to nested top actions

  - More coherent operations api / recovery implmentation

  - TnaiveHash* and Tinstant* are (and were already) broken, and are set for removal

  - Removed some instances of fail_unless

  - Fixed design flaws in blob implementation.

  - New naming convention for operation callback functions.
 2008-09-28 03:11:24 +00:00
Sears Russell
41cd975e91 fixed new const warnings from gcc 2008-09-24 06:40:34 +00:00
Sears Russell
944c7e984f fixes / cleanups of Tprepare() 2008-09-24 03:08:32 +00:00
Sears Russell
fb4e91debe removed deferred operation stubs 2008-09-12 23:08:41 +00:00
Sears Russell
e6ec2cb9fc Added "loadUninitializedPage()", which returns an uninitialized page instead of reading preimages from disk. 2008-06-09 01:13:02 +00:00
Sears Russell
aaf2f8bc39 Binary search implementations 2008-06-08 20:45:04 +00:00
Sears Russell
75f857bc26 Added supoort for binary search to page lookups 2008-06-08 20:28:53 +00:00
Sears Russell
651299716e Fixed bugs in gc iterator. 2008-06-08 20:28:23 +00:00
Sears Russell
690ebdea9a Mess with some of the typedefs for better code clarity, and (hopefully) better real-world compression. 2008-05-27 07:33:35 +00:00
Sears Russell
a4c29b95a2 Fix bug due to over-zealous optimization. It was accidentally skipping the max slot id check in some cases. 2008-05-27 07:32:37 +00:00
Sears Russell
decc2f7dfb Port to centos 4.2 2008-05-21 21:15:11 +00:00
Sears Russell
2863487991 Documentation update; added more sections under modules, reduced amount of cruft generated by doxygen. 2008-04-23 01:41:57 +00:00
Sears Russell
2929394b31 Fixed some documentation warnings, and clarified a few points. 2008-04-17 06:29:34 +00:00
Sears Russell
afbd89e3ff All headers now live in the header directory. (stasis/, not src/stasis) 2008-04-13 04:02:57 +00:00
Sears Russell
b1bfe7979f Questionable attempts to appease valgrind. 2008-03-14 03:09:29 +00:00
Sears Russell
ba986bfb02 Fix spurious 'error' in flamebox report. 2008-03-03 06:41:25 +00:00
Sears Russell
e49d51cb7e Clean up includes for new cmake build system. 2008-03-02 23:21:39 +00:00
Sears Russell
146563ca45 Allow third parties to register new iterator types. (made iterator registration non-static; moved constants into constants.h) 2008-03-01 19:34:12 +00:00