greg/stasis-aries-wal
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
1485 commits 3 branches 1 tag 14 MiB
Commit graph

272 commits

Author SHA1 Message Date
Sears Russell
ecbdc9513f add stasis_round_up_to_power_of_two 2010-10-03 15:41:19 +00:00
Sears Russell
b3e5743691 add c decl stuff to crc32.h 2010-10-03 15:40:53 +00:00
Sears Russell
18984c7b44 initial checkin of latency profiler 2010-09-30 19:28:42 +00:00
Sears Russell
6525949f56 add paranoid checks to concurrentHash and concurrentBufferManager; tightened up invariants maintained by concurrentHash 2010-09-30 19:27:18 +00:00
Sears Russell
00cf4eb4ce fix bug in lru backoff, add flag for buffer manager stress testing, add support for buffer manager handles to concurrentBufferManager, fix support in bufferHash 2010-09-28 23:50:07 +00:00
Sears Russell
ae04cee76c exponential backoff and power of two bucket flag for concurrenntWrapper (disabled by default) 2010-09-27 00:11:34 +00:00
Sears Russell
4fe5bcaa72 Refactored doubleLinkedList and lruFast. The linked list API is now less insane, and lruFast is simpler. Added asserts to detect double insertion of nodes into the lists / lru 2010-09-24 17:56:51 +00:00
Sears Russell
a3e7fc0fe2 add hint: stasis_buffer_manager_hint_writes_are_sequential. if set to 1, then only one thread will write back at a time, which leads to greatly improved sequential write throughput 2010-09-23 23:09:43 +00:00
Sears Russell
58c909a2ea add implementations of some more rblookup flags to the stl red black tree adapter 2010-09-23 20:38:50 +00:00
Sears Russell
bb904e85cf add latch-free slotted pages, and a bug-fix in page.c 2010-09-17 01:10:42 +00:00
Sears Russell
b722adff8a added DEBUG() statements 2010-08-05 23:07:52 +00:00
Sears Russell
7a27e39e5b MAX_BUFFER_SIZE is now a runtime (pre-tinit) option; clean up some doxygen warnings 2010-07-16 16:52:08 +00:00
Sears Russell
e2ba421a53 cleaned up buffer manager comments, added some asserts 2010-07-13 18:17:25 +00:00
Sears Russell
ee7aaff9ec add writeback starvation avoidance logic to concurrentBufferManager 2010-07-09 01:55:54 +00:00
Sears Russell
d4804bd926 add rwlc: A read-write lock that can wait on pthread conditions (assuming it holds the write lock) 2010-05-27 01:48:27 +00:00
Sears Russell
dc1f799a37 TarrayListAlloc() and TarrayListExtend() now generate O(1) bytes of log entries; they use the new multipage operation type. 2010-05-11 23:34:13 +00:00
Sears Russell
820f706faf TarrayListExtend now zeros out the contents of the newly allocated records (this is implemented by having fixed.c zero out the records) 2010-05-11 23:27:22 +00:00
Sears Russell
2d645beb42 cleaned up multipage operations a bit; added stasis_segments_enabled flag 2010-05-11 22:01:37 +00:00
Sears Russell
07df7c064a add support for multi-page operations 2010-05-11 00:17:07 +00:00
Sears Russell
31d045efba add link to my dissertation 2010-05-05 19:27:27 +00:00
Sears Russell
85ad4dd316 updated getting started text 2010-05-05 18:54:45 +00:00
Sears Russell
ffa4f18283 add tryToWriteBackPage. concurrentBufferManager is now compatible with FORCE mode transactions. 2010-04-28 00:05:00 +00:00
Sears Russell
c7fa929e71 This fixes in Stasis issue 22. 
Replacement policies now track the pin count of each page that they interact with.  This behavior used to live in bufferHash, and had never been ported to concurrentBufferManager.

For various reasons, the pin counts have to be updated atomically with the LRU state.  This would be very difficult to do inside of concurrentBufferManager, which sometimes needs to add and remove pages to replacement policies without holding any other latches.
 2010-04-20 01:09:32 +00:00
Sears Russell
887832582c expose stasis_handle_t up through the buffer manager; add sequential I/O hinting; fix stupid busy-wait in prefetch 2010-04-12 20:54:10 +00:00
Sears Russell
7c5eb2a7e9 added dup method to io handles 2010-04-08 23:50:01 +00:00
Sears Russell
ff2f2e9e1e add TregionPrefetch() 2010-03-31 22:41:33 +00:00
Sears Russell
3c27ed566e add prefetch primitives 2010-03-29 22:29:30 +00:00
Sears Russell
8a5bbd9a38 silence compiler warning 2010-03-29 22:28:32 +00:00
Sears Russell
6deb34f5b2 fix handling of loadPageForOperation during recovery; initialize rid.size in slottedLast 2010-03-22 19:56:55 +00:00
Sears Russell
5ffd1110e9 Added concept of "transaction fingerprints" 
The region allocator marks freed regions "Condemned" before actually freeing them.  Condemned regions are associated 
with a transaction; before they can be reused / coalesced the transaction that condemned them must complete.

The region allocator used "TisActiveTransaction(xid)" to determine whether a given transaction had completed.  The 
problem is that Stasis now reuses transaction id's, and condemned regions were never reclaimed in practice.  A transaction
fingerprint is a unique identifier (currently transaction id, rec_lsn) that can be used to distinguish transactions that
did not execute concurrently.

I considered an alternative design based upon end-of-transaction actions.  This would have worked as well, but would 
have been a more invasive change.
 2010-02-18 19:12:46 +00:00
Sears Russell
4e100479d1 Add TarrayListDealloc() to arrayList. Remove one region allocation from each array list alloc. Add unit test for dealloc. 
Also, removed some old compensations cruft.
 2010-02-18 04:12:39 +00:00
Sears Russell
98e0cb1fab implement TregionSize(), add TregionForce() 2010-02-12 16:44:28 +00:00
Sears Russell
0afb9177ac C++ compatibility fix 2010-02-11 22:18:49 +00:00
Sears Russell
1ff9afb753 added new constant: INVALID_SIZE 2010-02-11 20:36:33 +00:00
Sears Russell
49979ffa52 unit test for libredblack 2010-01-21 23:48:43 +00:00
Sears Russell
dbf253275f fix nested top action log race; because of a refactoring mistake, it was releasing a mutex without acquiring it... 2010-01-21 20:27:35 +00:00
Sears Russell
250223086c reordered includes for compatibility with gcc3 2010-01-21 00:03:17 +00:00
Sears Russell
f1f1d888e4 avoid use of -Wno-variadic-macros, which causes an error with gcc 3. 2010-01-20 23:51:54 +00:00
Sears Russell
b1f7d1947a more log refactoring. safeWrites now holds a latch across reserve_entry, write_entry, and write_entry_done 2010-01-19 02:14:09 +00:00
Sears Russell
c8cc188b71 unsigned int -> uint32_t 2010-01-19 01:47:13 +00:00
Sears Russell
d71e248291 removed freeLogEntry 2010-01-11 20:41:59 +00:00
Sears Russell
c30aaa3a1a fix compiler warning; -1 is not comparable to entry->type 2010-01-07 21:50:17 +00:00
Sears Russell
1af589dd18 remove malloc and memcpy from the in_memory_log read path 2009-12-31 23:25:56 +00:00
Sears Russell
1409a9eb27 more refactoring; log handles now manage their log entries memory 2009-12-31 23:01:37 +00:00
Sears Russell
a5788688c8 refactor getUpdateArgs; remove a bunch of casts, fix a bunch of warnings 2009-12-31 20:20:47 +00:00
Sears Russell
9aa1d9d652 pull memcpy out of allocUpdateEntry() 2009-12-31 19:30:40 +00:00
Sears Russell
32e40c949d added log modes to sequentialThroughput benchmark, added methods for concurrent appends to log api (and stubbed them out) 2009-12-31 19:02:34 +00:00
Sears Russell
7ff80f37ef add deferred, end of transaction callbacks. addresses issue 15 2009-12-15 06:30:37 +00:00
Sears Russell
efc6a0eabd add missing function declaration 2009-12-10 01:53:15 +00:00
Sears Russell
cd8c63889d clarify documentation 2009-12-09 19:55:27 +00:00
Sears Russell
83a0ec68fe fix rose bit rot, have unintialized page clear the impl pointer (for tidyness), remove some resolved @todos. 2009-12-04 19:33:09 +00:00
Sears Russell
7d1b702239 revert part of opensolaris patch--pack recordids again 2009-11-30 01:01:30 +00:00
Sears Russell
2ff421bff0 initial opensolaris port 2009-11-29 03:22:36 +00:00
Sears Russell
2535f69fbc switch the concurrent hash to use lru->getStaleAndRemove; fix crashes due to partial leak of thread local pages on thread shutdown 2009-11-24 01:43:36 +00:00
Sears Russell
38b999dfed add missing stlredblack.h header 2009-11-24 01:41:55 +00:00
Sears Russell
675bdce24a wrote some wrappers that emulate redblack; default to stl tree for now 2009-11-24 00:24:18 +00:00
Sears Russell
1b0c10b37b clean up handling of uninitialized pages (removes some special case code) 2009-11-09 23:02:27 +00:00
Sears Russell
795e044ab9 documentation fix 2009-11-09 22:58:37 +00:00
Sears Russell
b4a0f2436b concurrent lru implementation for new buffer manager 2009-11-09 20:53:05 +00:00
Sears Russell
8bd0616f97 new concurrent buffer manager 2009-11-09 19:50:00 +00:00
Sears Russell
f5e3ab2d91 concurrent hashtable seems to be working; still need to work out atomicity primitives (eg; options to leave the bucket locked on success) 2009-10-19 21:55:25 +00:00
Sears Russell
1cf16f62f2 add optional support for gcc atomics to transaction table 2009-10-15 22:46:25 +00:00
Sears Russell
e6a718a7f9 improved concurrency for transaction table 2009-10-15 20:11:43 +00:00
Sears Russell
331f680c09 partial rewrite of transaction table to reduce thread contention 2009-10-15 02:33:08 +00:00
Sears Russell
4565aff9de more refactoring; no more globals in transactionTable.c 2009-10-14 21:22:50 +00:00
Sears Russell
dbe3ecf0d0 refactoring transaction table; moved implementation into its own file. still need to remove globals 2009-10-14 18:57:50 +00:00
Sears Russell
20ef74b104 use pthread_rwlock instead of rwlock userspace emulation of it 2009-10-13 00:29:35 +00:00
Sears Russell
20a3fe9df5 add support for soft commits, and making prior soft commits durable 2009-10-12 19:16:41 +00:00
Sears Russell
942ba7fffc bugfixes + support for hard limits on log size 2009-10-08 22:48:58 +00:00
Sears Russell
6a02f2f159 buffer manager implementation is now determined by an overridable factory method 2009-10-05 22:39:09 +00:00
Sears Russell
24650fa190 buffer manager refactoring; started work on new dirty page table writeback interface (to avoid circular dependencies) 2009-10-05 21:19:01 +00:00
Sears Russell
4689582cb5 improve log performance by disabling O_SYNC by default (and use fdatasync instead) 2009-09-12 16:20:13 +00:00
Sears Russell
c4e80b41b5 more include cleaning on alloc, begin cleaning of bufferHash 2009-08-22 21:29:27 +00:00
Sears Russell
1ce1496226 dependency cleanups; removed compenastions from alloc.c 2009-08-22 00:01:02 +00:00
Sears Russell
94c542f8b1 Increase dirtyPageTable concurrency 
Add special-casing of lhtable; special case for buffer manager.
 2009-08-14 06:31:13 +00:00
Sears Russell
dd0bcae074 add thread synchronization logic to ensure that only one thread is performing writeback at a time. This fixes weird I/O interleavings on some machines 2009-08-13 23:18:49 +00:00
Sears Russell
0154f7d29d IMPORTANT API CHANGE: stasis_dirty_page_table_flush_range() no longer force writes to disk. You now need to call forcePageRange() after flush page range returns. 
This commit should significantly improve sequential write thoughput.

I found a segfault bug this morning, but cannot reproduce.  Could be a heisenbug, or could have been a hardware fault (gojira repeatedly kernel paniced, then both problems went away...)
 2009-08-13 05:53:48 +00:00
Sears Russell
0cc65aefaf Remove LRU-ordered writeback. Instead, writeback pages sequentially once some fraction of the page file is dirty. Also. further simplifies writeback code. 2009-08-11 17:23:43 +00:00
Sears Russell
907a24b68e dirty page table flush was pinning pages, then asserting they were not pinned; updated page writeback api to allow correct usage 2009-08-08 01:43:03 +00:00
Sears Russell
c92ee87c32 improved behavior for sequential I/O performance, fixed races during writeback 2009-08-07 18:27:52 +00:00
Sears Russell
a6aa8ed942 added method to access dirtyPageTable 2009-08-05 20:02:33 +00:00
Sears Russell
5b10bcc63b Reimplemented allocationPolicy.c; updated API + added more cases to the unit tests. 2009-08-05 06:35:11 +00:00
Sears Russell
c961b34903 forgot to commit b-tree source files 2009-07-29 21:04:08 +00:00
Sears Russell
ebd6b14dbf single-page btrees are working 2009-07-29 21:03:16 +00:00
Sears Russell
0a308f035c more header cleaning; updated INSTALL 2009-07-26 19:47:05 +00:00
Sears Russell
081c61b414 cleaned up includes, autoconf is now optional, since cmake now creates a config.h 2009-07-26 18:51:45 +00:00
Sears Russell
260310e151 Add page ops for tree implementations. 2009-07-26 06:18:54 +00:00
Sears Russell
7f2edd880d New (more thorough, less frequent) tests for slotted pages. Renamed slotted methods, added microbenchmark for slotted pages. 2009-07-20 17:42:43 +00:00
Sears Russell
9ef3adf40c - Fix compile warning + broken fcn pointer from last commit. 
- Address crashes in Tupdate() and recovery caused by mixing segments and page oriented recovery  (add loadPageForOperation() call)
 2009-07-16 03:05:32 +00:00
Sears Russell
b44f8b17b3 Add getCachedPage() call. This allows dirtyPageTable to write back pages with out accidentally reading them back in from disk with loadPage(). This should improve performance and allow loadPageOfType() to be used safely. 2009-07-14 07:46:47 +00:00
Sears Russell
4493dbb88b Added support for segment based recovery. For now, Stasis doesn't support hybrid recovery, so mixing page deallocation and segments will make recovery corrupt data / segfault. 2009-07-13 17:18:01 +00:00
Sears Russell
d6c91241d6 fixed bug that caused loadPageOfType to ignore type parameter. Recovery is still broken, since it calls loadPage, even when it can't 2009-07-10 01:19:44 +00:00
Sears Russell
eb0661ceeb added segment.c 2009-07-09 21:09:36 +00:00
Sears Russell
ade1dc896e Add loadPageOfType() call. (Untested for now) 2009-07-07 21:33:49 +00:00
Sears Russell
cf49c7dbf1 forgot to svn rm indirect's files 2009-07-07 20:05:15 +00:00
Sears Russell
2593de7a2c Move page type to Page_s struct. This breaks indirect.c (which is poorly thought out, and will be removed shortly), and was the next step on the road to segment-based recovery 2009-07-07 19:20:22 +00:00
Sears Russell
b66328192c in-memory group by implementation 2009-07-01 01:33:24 +00:00
Sears Russell
5efebbbb10 fix doxygen warning 2009-06-28 19:01:13 +00:00
Sears Russell
09af29abad clean up iterator interface; remove Titerator_releaseLock() 2009-06-28 18:54:57 +00:00
Sears Russell
120665ae53 cleaned up arrayList. stasis_record_write no longer takes an LSN (it was ignoring it anyway). Added some missing calls to stasis_page_lsn_write (each time a page is written to, stasis_page_lsn_write should be called; this happens automatically for operations called via tupdate 2009-06-02 18:25:35 +00:00