Commit graph

400 commits

Author SHA1 Message Date
Rusty Sears
4d51fe2212 change more malloc calls to use stasis_malloc/stasis_alloc 2012-11-30 17:15:03 -08:00
Gregory Burd
3a559a3b3c merge in changes from svn[r1572..r1601]
------------------------------------------------------------------------
r1601 | sears.russell@gmail.com | 2012-03-20 18:43:00 -0400 (Tue, 20
Mar 2012) | 1 line

commit bLSM bloom filter to stasis/util, which is where it really
belongs
------------------------------------------------------------------------
r1600 | sears.russell@gmail.com | 2012-03-04 01:58:38 -0500 (Sun, 04
Mar 2012) | 1 line

fix memory leak in skiplist unit test (now it is valgrind clean)
------------------------------------------------------------------------
r1599 | sears.russell@gmail.com | 2012-03-04 01:58:05 -0500 (Sun, 04
Mar 2012) | 1 line

fix typo in finalize type
------------------------------------------------------------------------
r1598 | sears.russell@gmail.com | 2012-03-04 00:59:59 -0500 (Sun, 04
Mar 2012) | 1 line

add comparator and finalizer parameters to skiplist constructor
------------------------------------------------------------------------
r1597 | sears.russell@gmail.com | 2012-03-03 18:23:16 -0500 (Sat, 03
Mar 2012) | 1 line

bugfixes for skiplist
------------------------------------------------------------------------
r1596 | sears.russell@gmail.com | 2012-03-02 15:05:07 -0500 (Fri, 02
Mar 2012) | 1 line

updated concurrentSkipList.  Seeing strange crashes
------------------------------------------------------------------------
r1595 | sears.russell@gmail.com | 2012-03-01 16:51:59 -0500 (Thu, 01
Mar 2012) | 1 line

add progress reports
------------------------------------------------------------------------
r1594 | sears.russell@gmail.com | 2012-02-28 13:17:05 -0500 (Tue, 28
Feb 2012) | 1 line

experimental support for automatic logfile preallocation
------------------------------------------------------------------------
r1593 | sears.russell@gmail.com | 2012-02-28 12:10:01 -0500 (Tue, 28
Feb 2012) | 1 line

add histogram reporting to rawIOPS benchmark
------------------------------------------------------------------------
r1592 | sears.russell@gmail.com | 2012-02-24 16:31:36 -0500 (Fri, 24
Feb 2012) | 1 line

userspace raid 0 implementation
------------------------------------------------------------------------
r1591 | sears.russell@gmail.com | 2012-02-12 01:47:25 -0500 (Sun, 12
Feb 2012) | 1 line

add skiplist unit test, fix compile warnings
------------------------------------------------------------------------
r1590 | sears.russell@gmail.com | 2012-02-12 00:52:52 -0500 (Sun, 12
Feb 2012) | 1 line

fix compile error
------------------------------------------------------------------------
r1589 | sears.russell@gmail.com | 2012-02-12 00:50:21 -0500 (Sun, 12
Feb 2012) | 1 line

fix some bugs in hazard.h surrounding thread list management and
overruns of R under high contention
------------------------------------------------------------------------
r1588 | sears.russell@gmail.com | 2012-02-11 14:23:10 -0500 (Sat, 11
Feb 2012) | 1 line

add hazard pointer for get_lock.  It was implicitly blowing away the
hazard pointer protecting y in the caller
------------------------------------------------------------------------
r1587 | sears.russell@gmail.com | 2012-02-10 18:51:25 -0500 (Fri, 10
Feb 2012) | 1 line

fix null pointer bug
------------------------------------------------------------------------
r1586 | sears.russell@gmail.com | 2012-02-10 18:03:39 -0500 (Fri, 10
Feb 2012) | 1 line

add simple refcounting scheme to concurrentSkipList.  This solves the
problem where a deleted node points to another deleted node, and we
only have a hazard pointer for the first node.
------------------------------------------------------------------------
r1585 | sears.russell@gmail.com | 2012-02-10 14:19:14 -0500 (Fri, 10
Feb 2012) | 1 line

add hazard pointers for update using the smallest free slot first.  The
old method left a race condition, since hazard_scan stops at the first
null pointer.
------------------------------------------------------------------------
r1584 | sears.russell@gmail.com | 2012-02-10 02:45:30 -0500 (Fri, 10
Feb 2012) | 1 line

add hazard pointers for update array
------------------------------------------------------------------------
r1583 | sears.russell@gmail.com | 2012-02-10 00:04:50 -0500 (Fri, 10
Feb 2012) | 1 line

skiplist update: concurrent, but broken
------------------------------------------------------------------------
r1582 | sears.russell@gmail.com | 2012-02-09 17:44:27 -0500 (Thu, 09
Feb 2012) | 1 line

skip list implementation.  Not concurrent yet.
------------------------------------------------------------------------
r1581 | sears.russell@gmail.com | 2012-02-08 13:33:29 -0500 (Wed, 08
Feb 2012) | 1 line

Commit of a bunch of new, unused code: KISS random number generator,
Hazard Pointers, SUX latches (untested) and bit twiddling for
concurrent b-tree
------------------------------------------------------------------------
r1580 | sears.russell@gmail.com | 2012-01-17 19:17:37 -0500 (Tue, 17
Jan 2012) | 1 line

fix typo
------------------------------------------------------------------------
r1579 | sears.russell@gmail.com | 2012-01-11 18:33:31 -0500 (Wed, 11
Jan 2012) | 1 line

static build fixes for linux.  hopefully these do not break macos...
------------------------------------------------------------------------
r1578 | sears.russell@gmail.com | 2012-01-09 19:13:34 -0500 (Mon, 09
Jan 2012) | 1 line

fix cmake under linux
------------------------------------------------------------------------
r1577 | sears.russell@gmail.com | 2012-01-09 18:37:15 -0500 (Mon, 09
Jan 2012) | 1 line

fix linux static binary compilation bugs
------------------------------------------------------------------------
r1576 | sears.russell | 2012-01-09 18:00:08 -0500 (Mon, 09 Jan 2012) |
1 line

port to macos x
------------------------------------------------------------------------
r1575 | sears.russell | 2012-01-09 17:39:43 -0500 (Mon, 09 Jan 2012) |
1 line

add missing _ from sync call name
------------------------------------------------------------------------
r1574 | sears.russell@gmail.com | 2012-01-09 14:26:31 -0500 (Mon, 09
Jan 2012) | 1 line

add -rt flag to static builds
------------------------------------------------------------------------
r1573 | sears.russell@gmail.com | 2011-12-20 23:38:29 -0500 (Tue, 20
Dec 2011) | 1 line

Simple makefile geared toward building libstasis.so and libstasis.a
(and nothing else)
------------------------------------------------------------------------
r1572 | sears.russell@gmail.com | 2011-12-20 22:37:54 -0500 (Tue, 20
Dec 2011) | 1 line

add some missing #include<config.h> lines
2012-04-21 12:52:31 -04:00
Sears Russell
7660467f8e Refactor page implementation to use multiple include trick (so the backing representation can be changed).
Also, fix apparent bug in stasis_page_fixed_set_record_type().

Otherwise, this change should not make any semantic changes.
2011-11-09 21:37:38 +00:00
Sears Russell
0058042987 test + fix allocation of zero byte records 2011-11-07 22:53:55 +00:00
Sears Russell
d6763bc57b remove dead (not even compiled) unit test 2011-10-06 17:10:22 +00:00
Sears Russell
9b9b3b5d03 fix broken assert in unit test 2011-10-06 17:08:45 +00:00
Sears Russell
52cd86f7f6 move myrandom into stasis/util, rename it stasis_util_random64 2011-08-31 19:25:09 +00:00
Sears Russell
9083bca1c3 Implement CLOCK replacement policy, and set it to be the default. Add flags to configure the default replacement policy. 2011-08-23 18:25:26 +00:00
Sears Russell
e692682cf8 remove questionable truncate and append api from handle.c 2011-07-26 19:15:14 +00:00
Sears Russell
7f332f85cc fix deadlock in filePool logger. The deadlock was due to pages being evicted before Tupdate() released the log entry for writeback. Now, we track another timestamp, so that Tupdate() starts with something analogous to a writelock, and downgrades it to a readlock before pinning pages, and invoking callbacks 2011-07-15 17:37:50 +00:00
Sears Russell
9244d4607f fix api violation 2011-07-15 16:29:12 +00:00
Sears Russell
d8621db18f move latching stuff to util/, blobs to operations/ 2011-06-13 11:26:25 +00:00
Sears Russell
a89c09d87c moved in-memory data structures and hash functions to util/ 2011-06-13 10:19:44 +00:00
Sears Russell
87d4563ab5 move compression and latchFree page formats to experimental/ 2011-06-13 08:02:20 +00:00
Sears Russell
377788a13e move a bunch of multiplexer and lsm related stuff into experimental/ directory 2011-06-13 06:58:34 +00:00
Sears Russell
c5a36c032d remove compensations.h (This is a big change, and impacts lots of whitespace. I have left some indentation breakage in place in order to minimize the size of the diff) 2011-06-12 08:13:34 +00:00
Sears Russell
0bce11e0c8 silence compiler warnings about tests for old broken non_blocking handle 2011-04-20 20:46:38 +00:00
Sears Russell
78789a868f re-enable old buffer manager test 2011-04-20 20:44:51 +00:00
Sears Russell
935407be51 fix compiler wranings in check_linearHashNTA 2011-04-20 20:38:22 +00:00
Sears Russell
853394f022 fix compiler warnings regarding min and concurrentRingbuffer 2011-04-20 20:34:39 +00:00
Sears Russell
fbdc102d6c change replacementPolicy to avoid a malloc / free during page pins 2011-04-20 20:25:17 +00:00
Sears Russell
84bbeca165 filePool implementation is more-or-less complete. added remaining functions and latching 2011-04-19 18:51:43 +00:00
Sears Russell
5f3be196fe forgot to commit unit test for filePool 2011-04-18 20:34:19 +00:00
Sears Russell
2f664271b4 partial implementation of filePool log format. force and truncate are unimplemented, as is attempting to read unflushed data. 2011-04-18 20:31:19 +00:00
Sears Russell
175c26e125 add flush and shutdown apis to concurrent ringbuffer, tighten up get_rd_buf api 2011-04-18 20:21:16 +00:00
Sears Russell
e087c9c073 add alternative implementation of min aggregate for small sets. It avoids red black overheads, but some operations are linear in the number of values considered by the aggregate 2011-04-18 20:04:33 +00:00
Sears Russell
304b439217 new data structures: min aggregate and concurrent ringbuffer 2011-04-14 23:03:20 +00:00
Sears Russell
e1201970b6 add firstPendingEntry call to log API, use finer grained latches in Tupdate(). 2011-04-01 17:25:46 +00:00
Sears Russell
de8bf53e98 fix some header #include problems 2011-02-22 19:12:38 +00:00
Sears Russell
fb5791e7e4 one line fix to concurrentHashTable, plus documentation and improved unit tests. 2011-02-15 02:57:19 +00:00
Sears Russell
14d8cebdcc dbug stuff. for now, you have to edit cmakelists.txt to set the location of dbug. 2011-02-14 22:09:57 +00:00
Sears Russell
afc9991156 dbug mode for buffer manager - run with two threads, and a handful of ops/thread 2011-02-07 23:53:20 +00:00
Sears Russell
a4a72b601e short circuit no-op aborted transactions; add unit test for no-op commits and aborts 2011-01-05 19:10:57 +00:00
Sears Russell
0a461faa04 add "raid1" a simple mirroring I/O handle implementation 2010-12-03 03:51:24 +00:00
Sears Russell
2105337d7f disable tests of dead code that are known to fail 2010-10-19 21:05:54 +00:00
Sears Russell
4a5d14f2c6 more non_blocking fixes. now the unit test passes 2010-09-23 19:52:54 +00:00
Sears Russell
bb904e85cf add latch-free slotted pages, and a bug-fix in page.c 2010-09-17 01:10:42 +00:00
Sears Russell
00e474e3fe Fix bug in LinearHashNTA: inserting over an old key, then aborting set deleted the old key instead of restoring it to its original value 2010-09-01 04:01:20 +00:00
Sears Russell
e5528ce218 relaxed page latching requirements; page.c, and dirtyPageTable no longer check / rely on holding rwlatch; removed (now) unnecessary latching from unit tests 2010-08-06 00:19:19 +00:00
Sears Russell
7a27e39e5b MAX_BUFFER_SIZE is now a runtime (pre-tinit) option; clean up some doxygen warnings 2010-07-16 16:52:08 +00:00
Sears Russell
2d645beb42 cleaned up multipage operations a bit; added stasis_segments_enabled flag 2010-05-11 22:01:37 +00:00
Sears Russell
07df7c064a add support for multi-page operations 2010-05-11 00:17:07 +00:00
Sears Russell
04de939e42 fix null pointer derefence in lruFast.c; add unit tests to check for this bug 2010-04-21 21:34:00 +00:00
Sears Russell
c7fa929e71 This fixes in Stasis issue 22.
Replacement policies now track the pin count of each page that they interact with.  This behavior used to live in bufferHash, and had never been ported to concurrentBufferManager.

For various reasons, the pin counts have to be updated atomically with the LRU state.  This would be very difficult to do inside of concurrentBufferManager, which sometimes needs to add and remove pages to replacement policies without holding any other latches.
2010-04-20 01:09:32 +00:00
Sears Russell
5138af6d1c leave 2 pages in reserve for stale pin test (as opposed to one) 2010-04-19 23:09:30 +00:00
Sears Russell
7c5eb2a7e9 added dup method to io handles 2010-04-08 23:50:01 +00:00
Sears Russell
d281282533 fix concurrentWrapper so that it implements the same api as the other replacement policies (it used to livelock when no pages were available. It now returns null). 2010-03-31 22:37:42 +00:00
Sears Russell
3c27ed566e add prefetch primitives 2010-03-29 22:29:30 +00:00
Sears Russell
5ffd1110e9 Added concept of "transaction fingerprints"
The region allocator marks freed regions "Condemned" before actually freeing them.  Condemned regions are associated 
with a transaction; before they can be reused / coalesced the transaction that condemned them must complete.

The region allocator used "TisActiveTransaction(xid)" to determine whether a given transaction had completed.  The 
problem is that Stasis now reuses transaction id's, and condemned regions were never reclaimed in practice.  A transaction
fingerprint is a unique identifier (currently transaction id, rec_lsn) that can be used to distinguish transactions that
did not execute concurrently.

I considered an alternative design based upon end-of-transaction actions.  This would have worked as well, but would 
have been a more invasive change.
2010-02-18 19:12:46 +00:00
Sears Russell
4e100479d1 Add TarrayListDealloc() to arrayList. Remove one region allocation from each array list alloc. Add unit test for dealloc.
Also, removed some old compensations cruft.
2010-02-18 04:12:39 +00:00