greg/stasis-aries-wal
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
1246 commits 3 branches 1 tag 14 MiB
Commit graph

629 commits

Author SHA1 Message Date
Sears Russell
2d136695ff Documentation Fix; FILE_PERM is now in constants.h 2006-10-05 00:47:05 +00:00
Sears Russell
84dcdf228f Documentation Fix 2006-10-05 00:46:18 +00:00
Sears Russell
212e3c4626 No longer compile page/header.c 2006-10-05 00:45:32 +00:00
Sears Russell
b06e3daf1b Should now be silent during normal operation. 2006-10-04 04:41:53 +00:00
Sears Russell
92a1ac3ee5 Recovery now ignores INTERNALLOG entries 2006-10-04 04:41:19 +00:00
Sears Russell
524ece3fa4 Limited "Leaking iterators" warning to once per run. 2006-10-04 04:40:38 +00:00
Sears Russell
6bb126debf Made "GetPage is stuck!" less obtrusive. 2006-10-04 04:39:58 +00:00
Sears Russell
4bd93ee66a Added support for computing size of internal log entries. 2006-10-04 04:38:21 +00:00
Sears Russell
fd7f4074af Added CRC's to log format. 2006-10-04 04:37:03 +00:00
Sears Russell
f01cff0d5b Cleaned up logger API, added truncation support to inMemoryLog.c 2006-09-27 20:30:12 +00:00
Sears Russell
fcf01f5d71 Cleaned up some old comments. 2006-09-27 20:28:44 +00:00
Sears Russell
f620aaedd6 Removed some dead code. 2006-09-27 20:27:49 +00:00
Sears Russell
986ee57020 More fixes to prevent tardis from running out of disk space. 2006-09-25 22:27:13 +00:00
Sears Russell
b615f0bc3e Towards 64-bit pageids. bufferManager can handle 64-bit offsets, but recordid.page is still 32 bit... 2006-08-11 19:31:42 +00:00
Sears Russell
8faf045e7d Fixed numerous bugs in allocationPolicy.c and alloc.c 
allocationPolicy.c was not handling deallocations correctly
Tdealloc() was not calling interpretRid() on its input before passing it into the allocationPolicy
Added locking support for TallocFromPage()
 2006-08-11 02:56:41 +00:00
Sears Russell
ef43a8d54f Added missing declaration of forcePageFile() 2006-08-11 02:42:17 +00:00
Sears Russell
b44551b941 fixed mutex checks 2006-08-11 02:25:17 +00:00
Sears Russell
7dada65a13 Addded "interpretRid" which knows how to call the "dereference*" functions. 2006-08-11 02:24:01 +00:00
Sears Russell
51c5b60047 Fix race in TallocFromPage() 2006-08-11 00:01:12 +00:00
Sears Russell
73436d0d6c Disables fsckSlottedPage() by default. 2006-08-10 23:59:13 +00:00
Sears Russell
7ba91533fe Added assertions to make sure that the region mutex is held when 
appropriate.
 2006-08-10 23:57:49 +00:00
Sears Russell
af175896cb Added logWriter_isDurable. If = 0, then the log will never be forced. 2006-08-10 23:54:46 +00:00
Sears Russell
1b6c828bd2 Added logWriter_isDurable. If set to zero, the log will never be forced 
to disk.
 2006-08-10 23:53:51 +00:00
Sears Russell
3a43450477 Force in-flight page file writes to disk before truncating. (Needed if 
O_DIRECT isn't in use.)

Changed diagnostic messages to go to stderr.
 2006-08-10 23:52:48 +00:00
Sears Russell
f93b2acf69 Now, you can set pageFile_isDurable=0 to disable forcing data to the 
page file.

Also, -DPAGE_FILE_O_DIRECT switches the page file from using a normal
file handle and fdatasync to use O_DIRECT with no fdatasync.
 2006-08-10 23:51:15 +00:00
Sears Russell
6f9c6af99c memoized loadPage() (per thread) 
no longer requires pbl compatibility wrappers
 2006-08-10 23:45:25 +00:00
Sears Russell
7847331771 Minor cleanup, added ability to calculate the global average number of 
buckets traversed per find().  Added a warning for code that actually
relies upon PBL's way of interpreting insert.
 2006-08-10 23:43:53 +00:00
Sears Russell
d3f87b5af4 Added support for locking on deallocation. 2006-08-10 23:39:36 +00:00
Sears Russell
6fcaf34bb5 Needed to protect a linked list with a mutex. It's suprising that this hasn't come up before. 2006-08-08 01:41:45 +00:00
Sears Russell
8a390c40d2 Improved latch profiling, especially for loadPage. 2006-08-04 23:45:27 +00:00
Sears Russell
c413203ec8 Talloc() now reuses space when possible. It still needs to call slottedCompact() intelligentally. Also, allocationPolicy should support an extra tier 
of pages, so it doesn't suggest ones that aren't already in the buffer pool unless it has to.
 2006-07-29 00:56:54 +00:00
Sears Russell
783d1d6ffd Removed the fixed page initialization operations, which weren't being used, and were a slow hack anyway. There is now a more-general (and efficient) 
implementation in alloc.c
 2006-07-29 00:54:01 +00:00
Sears Russell
cdd44a2f2d Initial import of redblack implementation. 2006-07-28 00:01:57 +00:00
Sears Russell
3725cc9b8e Added a function that returns the number of allocated entries in the list 2006-07-28 00:00:32 +00:00
Sears Russell
165b91cf76 compensations_init() was being called too late. 2006-07-27 23:59:21 +00:00
Sears Russell
a3d0f6f6f5 Added support for concurrent transactions (by using nested top actions). 2006-07-27 00:15:29 +00:00
Sears Russell
1988cf297d The "WARNING: Tdeinit() is aborting transaction %d" runtime message now 
lists the xid of the transaction it's aborting instead of "-1".

TisActiveTransaction(xid) now returns 0 for INVALID_XID.  Before the
output was undefined.
 2006-07-27 00:13:20 +00:00
Sears Russell
d1da4e59c0 Fixed segmentation fault when a CLR with a NULLRID is encountered. I 
don't understand why this bug wasn't triggered by linearHashNTA or
linkedListNTA, but is triggered by the new region implementation.
 2006-07-27 00:10:21 +00:00
Sears Russell
7888d647ed Removed unused abort/commit callbacks. 2006-07-26 17:48:36 +00:00
Sears Russell
62abe94bf5 added fsckRegions(), and call it from the unit tests. 2006-07-25 22:23:15 +00:00
Sears Russell
aec9148b83 added call to get the page type. 2006-07-25 22:22:49 +00:00
Sears Russell
41528ef066 Fixed lockManager. lhtable's iterators do not support concurrent modifications. 2006-07-25 19:12:53 +00:00
Sears Russell
8e419845be removed assert() that checks to make sure that Tupdate() didn't pass a page pointer in. 2006-07-25 01:24:49 +00:00
Sears Russell
8aad1b896a pthread.h | rw.h -> latches.h (Otherwise, -DPROFILE_LATCHES doesn't work. 2006-07-25 01:06:01 +00:00
Sears Russell
45a5996215 Documented subtle hack; TallocPage() really needs a way to set page type at recovery. Currently, this just doesn't happen, since page types are lazily set. 
Perhaps Talloc() could handle it?
 2006-07-25 01:05:02 +00:00
Sears Russell
8485cc7fe1 Added missing releasePage(). 2006-07-25 01:03:57 +00:00
Sears Russell
745975112f Removed an assert that assumed pages aren't reused. 2006-07-25 01:03:07 +00:00
Sears Russell
747bc9cdc9 Removed #defines for broken ptrheads stuff. 2006-07-25 01:02:27 +00:00
Sears Russell
9721e86133 Commented out (currently broken) pthreads instrumentation, added readLock tracking to the latch instrumentation. 2006-07-25 01:01:46 +00:00
Sears Russell
2a9cb788a6 if LONG_TEST is defined, check for pinned pages on exit. 
This test is currently implemented inefficiently, otherwise it would always be enabled.
 2006-07-25 01:00:46 +00:00
Sears Russell
a5a234635e readBlob() no longer writes the blob contents to the log. ;) 2006-07-25 00:58:28 +00:00
Sears Russell
0f45b97eda regions now make use of nested top actions, so they're almost correct. (Still need to lock freed regions until end of transaction.) 2006-07-25 00:56:50 +00:00
Sears Russell
86fab6a22b Turn on paranoid slotted page consistency checks for long running tests. 2006-07-22 00:05:40 +00:00
Sears Russell
2af892709f Rewrote slottedCompact; added fsckSlottedPage 2006-07-22 00:01:42 +00:00
Sears Russell
0725af91be Fixed page leak in Talloc(). 2006-07-21 01:07:09 +00:00
Sears Russell
eb13531c0f Partial handling of incomplete log entries. (It detects when the log file does not end at the right byte, but doesn't checksum, or otherwise look for 
sectors that didn't make it to disk.)
 2006-07-21 01:06:35 +00:00
Sears Russell
1ff1d4c128 Re-enabled truncation (accidentally disabled it before...) 2006-07-20 01:30:58 +00:00
Sears Russell
ffd3bd960b switched to region allocator, but TpageAlloc currently grabs its own region... 2006-07-20 01:29:39 +00:00
Sears Russell
7bc60c3ac4 fixed the fixedPageAlloc hack. 2006-07-20 00:49:28 +00:00
Sears Russell
8ed909e79d Fixed the fixedPageAlloc hack. 2006-07-20 00:48:42 +00:00
Sears Russell
1ac3dc6779 More progress on boundary tags. (Still not tested/working yet...) 2006-07-18 23:59:00 +00:00
Sears Russell
bf98e32c73 fixed build 2006-06-22 23:31:42 +00:00
Sears Russell
93e7e96ea3 Started implementing the region allocator 2006-06-22 19:10:02 +00:00
Sears Russell
d26263262b Completely new blob implementation. Slower than the old one was supposed to be, but is correct, and extremeley 
simple.  It simply writes the pre-image and post-image of each blob to the log using TpageSet.
 2006-06-21 00:45:18 +00:00
Sears Russell
c8c9b7af19 the logger understands special record types. (At least partially...); shortened blob tests, since the blob 
implementation is fundamentally broken, and slow
 2006-06-20 23:02:27 +00:00
Sears Russell
6485a4aae9 No more PBL. 2006-06-20 21:44:17 +00:00
Sears Russell
4d79c0a466 Simplified allocation 2006-06-20 21:40:21 +00:00
Sears Russell
e14877821f Fixed blob bug. The reason it's so slow is that it's logging a preimage + postimage of each blob and then synchronously writing to a second copy each time the blob is 
written to(!!!)
 2006-06-20 00:03:38 +00:00
Sears Russell
3e4cac0fe3 Continued cleaning up allocation code. It seems really slow all of a sudden, so this commit probably introduced a performance bug. 2006-06-17 00:25:09 +00:00
Sears Russell
209a6916e5 Updated some comments. 2006-06-16 00:27:02 +00:00
Sears Russell
dc8a3e5d60 Eliminated slottedPreRalloc 2006-06-16 00:05:44 +00:00
Sears Russell
245db32c23 Two programs to create and quickly query a read-only on disk hash table 2006-06-15 22:28:39 +00:00
Sears Russell
50515ffd23 More steps toward factoring page allocation out of record allocation. 2006-06-15 05:31:20 +00:00
Sears Russell
59bd80a2a8 half-hearted attempt to fix -Wextra warnings... 2006-06-13 23:58:04 +00:00
Sears Russell
a3678aad5c Now AM_CFLAGS can be set by modifying GLOBAL_CFLAGS in the root 
Makefile.am
 2006-06-13 22:48:50 +00:00
Sears Russell
f53a46e788 removed heisenbug 2006-06-13 22:47:38 +00:00
Sears Russell
1b3a08d434 added some todos 2006-06-13 22:46:11 +00:00
Sears Russell
1cf3119eb7 renamed __really_do_ralloc to really_do_ralloc 2006-06-13 22:45:30 +00:00
Sears Russell
66ae3b9763 Made it more obvious that a suspicious looking assert is commented out 2006-06-13 22:44:40 +00:00
Sears Russell
24a5da9648 Fixed deprecated volatile static declaration (-> static volatile) 2006-06-13 22:43:58 +00:00
Sears Russell
33af9805ad printf -> fprintf(stderr, and assert(0) -> abort() 2006-06-13 22:43:20 +00:00
Sears Russell
f77e8b240d It was improperly tracking the log's global offset. (Hopefully) 
fixed offset tracking, made it optional, and disabled it.
 2006-06-13 22:42:15 +00:00
Sears Russell
c5add15bd3 assert(0) -> abort 2006-06-13 22:40:53 +00:00
Sears Russell
eab6385ca5 Moved #defines below definition of Page_s. 2006-06-13 22:40:24 +00:00
Sears Russell
f89903d529 moved stdio.h include out of header file. 2006-06-13 22:38:25 +00:00
Sears Russell
e56c46b95c Moved blob_record_t into transactional.h (part of transition to using 
slot type lookup tables.)
 2006-06-13 22:37:29 +00:00
Sears Russell
a11fbdd9e6 Did away with slottedPreRallocFromPage. 2006-06-13 20:00:46 +00:00
Sears Russell
cef9e2c4ce Added settings for longer-running tests 2006-06-12 22:45:41 +00:00
Sears Russell
e0b34633ec Fixed broken min_rec_lsn() semantics; before, if there were not outstanding transactions / writes, 
then these calls would return flushedLSN().  The problem was that flushedLSN wasn't guaranteed to
return a pointed to the beginning of a log entry.
 2006-06-12 21:50:21 +00:00
Sears Russell
7bb0ce0d17 Fixed truncation bug (truncation is more aggressive now), and added more debugging info. 2006-06-12 20:55:49 +00:00
Sears Russell
27f7df6f09 Fixed races on flushedLSN and page->dirty. 2006-06-12 20:05:28 +00:00
Sears Russell
35a5e80871 removed bogus .deps files. 2006-06-01 22:18:50 +00:00
Sears Russell
fd19036843 Added more complete hashtable test; fixed incorrect return value in LH_ENTRY(find) 2006-05-31 00:37:56 +00:00
Sears Russell
0649f179b3 Got rid of the ~1 second delay on exit. 2006-05-30 22:59:22 +00:00
Sears Russell
68dcc2067e No more warnings with -O2. 2006-05-25 22:50:52 +00:00
Sears Russell
8ce9e6127f Fixed return value in removeFromLinkedList 2006-05-25 22:48:15 +00:00
Sears Russell
b5d578f4e0 Fixed remaining warnings, got rid of alTupdate 2006-05-25 20:08:12 +00:00
Sears Russell
fda416090b Fixed a bunch of 'easy' warnings 2006-05-25 00:02:46 +00:00
Sears Russell
cb3a4ae20d Bucket list extension working. 2006-05-24 23:24:31 +00:00
Sears Russell
8b54b0d265 No longer attempts to truncate when long transactions prevent it from 
being useful...
 2006-05-24 23:21:33 +00:00
Sears Russell
827e0d891a The linear hashtable implementation (to replace pbl's) 2006-05-24 02:23:58 +00:00
Sears Russell
2e06cafaa8 2006-05-24 02:19:04 +00:00
Sears Russell
4c168d8fb2 Fixed truncation bugs, and a minor test case bug. 2006-05-22 23:59:02 +00:00
Sears Russell
f06159b3cd forgot to add these... 2006-05-19 20:17:44 +00:00
Sears Russell
ef43d5e4ce Implemented log truncation. 2006-04-14 03:45:26 +00:00
Sears Russell
11c311bc91 Added support for raw pages, implemented posix_memalign workaround, and re-enabled O_DIRECT by default 2006-04-12 01:41:35 +00:00
Sears Russell
33cbbb0e2d In memory logging implementation 2006-04-12 01:40:47 +00:00
Sears Russell
ab541b7dc8 Created bufferPool, moved memory management for page structs to it 2006-04-11 20:35:21 +00:00
Sears Russell
c29b16a6ed Created new module, bufferPool, and moved memory management for page structs to it. 2006-04-11 20:34:52 +00:00
Sears Russell
c1d8906d28 removed memcpy() calls from inMemoryLog; added "const" qualifier to many LogEntry pointers. 2006-04-11 02:20:21 +00:00
Sears Russell
7decae7172 All tests that pass with LOG_TO_FILE now pass with LOG_TO_MEMORY, but LOG_TO_MEMORY is slow, and seems to be 
killing malloc's performance for some reason.  (linearHashNTA spends many seconds in free() between unit
tests...)
 2006-04-08 00:05:09 +00:00
Sears Russell
d3d7f2c788 Implemented "LOG_TO_MEMORY", which mostly works (linear hash NTA goes into an infinite loop....) 2006-04-07 03:40:40 +00:00
Sears Russell
1c979f3052 Factored logWriter.h out of the rest of the system. The only file that directly depends on it is logger2.c, which can 
now dispatch requests to different (hardcoded) log implementations.
 2006-04-06 03:22:57 +00:00
Sears Russell
10e7434d32 corrected slottedFrespace()'s declaration (it was returning an int, but needed to return a size_t 2006-04-05 02:54:14 +00:00
Sears Russell
d9f1df0476 Fixed behavior when __really_do_ralloc() is called, but the record's slot is greater than numslots_ptr(). 
Optimized slotted_compact and __really_do_ralloc() so that they try to put lower numbered slots earlier in the freelist.
 2006-04-05 02:52:40 +00:00
Sears Russell
d896155b82 Fixed freespace measurement during recovery/abort by adding slottedFreespaceForSlot() 
(This version is still broken, because __really_do_ralloc() does not properly increment numslots_ptr
 2006-04-05 00:38:11 +00:00
Sears Russell
041bfce736 The bucket list entries for variable lenth hash tables were too long, causing memory corruption, and wasting space+log bandwidth 2006-04-04 22:34:35 +00:00
Sears Russell
959233566c changed type of lsn variables to "lsn_t" 2006-04-04 22:33:30 +00:00
Sears Russell
8e0bd2ecd2 Extra check for end of iterator. 2006-04-04 22:32:48 +00:00
Sears Russell
2250b7a6c7 More freespace troubles... Should be correct now. 2006-03-28 21:27:10 +00:00
Sears Russell
3c55daede9 The freespace calculations were double counting the new slot's header. 2006-03-28 21:21:47 +00:00
Sears Russell
ced022d0fe partial fixes 2006-03-28 20:15:31 +00:00
Sears Russell
bb37db0857 Partial port to amd64. 2006-03-20 23:11:46 +00:00
Colleen M. Lewis
5bfcbdb076 Bug fixed: TfixedPageAlloc calls fixedPageInitialize with the wrong arguments. 2005-10-31 23:21:18 +00:00
Colleen M. Lewis
345f0cde78 Added the base functionality and test case for the BTree... just a hello world method. but it has the h, c and check files. -colleen ;) 2005-05-05 21:34:12 +00:00
Sears Russell
6b95cefc62 Fixed logWriter abort performance bug, also cleaned up the file, simplified locking, etc. 2005-04-14 04:23:22 +00:00
Sears Russell
a053b19757 removed extraneous abort 2005-04-14 01:43:14 +00:00
Sears Russell
3844b641a3 final transclos benchmark 2005-03-25 16:33:36 +00:00
Sears Russell
08ceca11ce Enabled compactPage in TallocFromPage() 2005-03-24 11:04:06 +00:00
Sears Russell
4c1ca194d0 Major edits throughout the paper. 2005-03-24 10:00:08 +00:00
Sears Russell
cf58e1fb72 Transitive closure working! 2005-03-24 04:43:22 +00:00
Sears Russell
2f99ed4664 Generic consumer wrapper functions, and implementation registrations. 2005-03-23 08:10:30 +00:00
Sears Russell
ee9eb416ea Implemented fifo pool that allocates entries out of a pool (it should be easy to extend this to avoid deadlock, while allowing it to continue to provide backpressure) 2005-03-23 08:07:53 +00:00
Sears Russell
6228931e58 A simple script that interprets the CI data, and allows basic calculations to be performed using command line arguments. 2005-03-21 22:32:28 +00:00
Sears Russell
b38db5b52b Added figures to paper. 2005-03-21 22:31:43 +00:00
Sears Russell
6d4216bc2a Preliminary (broken) transitive closure implementation. 2005-03-21 08:04:29 +00:00
Mike Demmer
c1997d8350 misc fixes to the script 2005-03-20 06:39:44 +00:00
Sears Russell
ef3c30b190 Benchmark updates. 2005-03-20 05:17:25 +00:00
Sears Russell
26d50f19e3 Cleaned up berkeley db tests / lladd logger optimizations + bug fix. 2005-03-19 20:28:30 +00:00
Sears Russell
97bf995022 changes for diff, commit prior to deleteing a lot of old bdb cruft 2005-03-18 23:26:25 +00:00
Sears Russell
d8eb6c9aa7 Major benchmark update; log force merging. 2005-03-17 04:38:55 +00:00
Sears Russell
bf4be359b3 Partially refactored logMemory and multiplexer into a fifo implementation. Added support to fifoPool for 
asynchronous threads running round-robin through the dirty fifos consuming entries in idle ones.
 2005-03-16 00:52:21 +00:00
Sears Russell
dffff067bb multiplexer and fifo are now working. (also added removed some timeouts from tests that were timing out) 2005-03-15 05:30:57 +00:00
Sears Russell
ed9b978bda Another commit on behalf of jimmy. (Fixing the build is hard!) 2005-03-15 01:57:43 +00:00
Sears Russell
602608b60b Adding new files for jimmy... 2005-03-15 01:29:04 +00:00
Jimmy Kittiyachavalit
80ed16c18e Initial implementation of logMemory.c. Haven't yet done any testing, but it seems to compile. However, as of now, the current source code in the cvs repository does not build, dying on check_<somehting (I think ringbuffer.x)>, due to an a reference to an undefined symbol 'mutex'. 2005-03-14 20:54:06 +00:00
Sears Russell
7971df14bf Bugfixes (compiles now) 2005-03-14 08:47:13 +00:00
Sears Russell
9be3c4edd3 Initial upload of (untested) multiplexer implementation. 2005-03-14 00:58:47 +00:00
Sears Russell
ed49949512 This commit breaks the build so that Jimmy and I can sync. Sorry. :( 2005-03-14 00:25:29 +00:00
Sears Russell
1bfb634103 Preliminary benchmark program uploads for SEDA-style response time percentiles vs requests / second 
Also added a few cute little utilities:
  - truncate_log
  - run_recovery

Upped max # of concurrent transactions.  (Current throughput during benchmarking remains ~flat up to 32,000
threads, but past max concurrent transactions was 1,000...)
 2005-03-12 22:11:18 +00:00
Sears Russell
18c772234d new benchmarks revealed that inserts for linkedList, pageOrientedList were taking O(n) time, yielding O(n^2) 
time to build a list, since the lists could not contain duplicate keys.  These operations are now O(1), and
the lists can contain duplicates.
 2005-03-10 20:10:49 +00:00
Sears Russell
f28df29736 Fixed unit tests, added iterator interface, and generic (almost) iterator test harness. 2005-03-10 03:19:04 +00:00
Sears Russell
935de3aeb5 minor changes 2005-03-09 03:24:36 +00:00
Sears Russell
e22d4b8e59 Partial implementation of ringbuffer for in-memory long, among other things. 2005-03-08 07:53:53 +00:00
Sears Russell
a3112ee81c Fixes for freebsd. 2005-03-02 05:47:38 +00:00
Sears Russell
c8a0aa830e Compiles on FreeBSD! 2005-03-02 05:46:29 +00:00
Sears Russell
7833177f05 Fixed the support for more than MAX_TRANSACTION total transactions. 2005-03-02 03:28:00 +00:00
Sears Russell
3e5f7aea8a Bugfixes for OASYS. 2005-03-01 07:32:02 +00:00
Sears Russell
3b88909c5f Fixed compensations test; FreeBSD fix (?) for makefile. 2005-02-28 00:00:17 +00:00
Sears Russell
890a7385d0 Fixeds cht test / finished error checking within LLADD, but haven't tested error handling yet. 2005-02-24 21:12:36 +00:00
Sears Russell
29dacbe7ed Error handling for linked list. 2005-02-22 23:11:03 +00:00
Sears Russell
41fb85eef0 More error handling... 2005-02-22 03:10:54 +00:00
Sears Russell
84a20a3c96 Started adding error handling code to LLADD. 2005-02-16 04:11:14 +00:00
Sears Russell
f4484e35b5 Update for perl script; added return type handling. 2005-02-16 02:13:54 +00:00
Sears Russell
fdf5344ec3 Implemented 'compensations' for C. They're not proper compensations, but they're more flexible than pthread's mechanisms, as they allow the stack to be partially rolled up. (Although calling functions need to either check for errors and return manually, or wrap calls to error-producing callees in a begin_action{ }end_action block. 2005-02-14 02:49:59 +00:00
Sears Russell
2ac1302062 Recovery now supports the lock manager. (And vice-versa ;) 2005-02-10 21:56:32 +00:00
Sears Russell
8bf6ea29ff Added @todo that details requirements for next version of alloc. 2005-02-10 21:55:35 +00:00
Sears Russell
b8a67cbfb2 Numerous lock manager fixes; more realistic test case for lock manager, added support for optional page-level locking.(Still need to report deadlocks back up to the caller.) 2005-02-10 03:51:09 +00:00
Sears Russell
6f022cc577 Some random code cleanups, and implemented a naive lock manager. (Still need to tie it into LLADD...) 2005-02-09 02:53:14 +00:00
Gilad Arnold
815942ed4c - some further API improvements (?). 2005-02-08 23:56:47 +00:00
Gilad Arnold
307f268fdc - working with flags instead of Boolean arguments; replaced some specialized 
function calls with header-file macros.
 2005-02-08 22:06:24 +00:00
Gilad Arnold
b0a9265d1a - enhanced init/shutdown semantics: init may now be re-issued, following a 
predecessing shutdown; note that shutdown does not deallocate objects (either
  persistent or non-persistent), hence re-init would use the same objects as in
  the previous run (assuming they are all intact).
 2005-02-08 08:04:35 +00:00
Sears Russell
258dd5f4c4 Fixed up some return values; tests now pass with assert disabled. 2005-02-08 01:13:57 +00:00
Gilad Arnold
eec80a0e38 - improvement to error handling in pobj: now properly handles resource 
allocation errors (at least most cases...?), including hash/queue
  library calls.
 2005-02-08 00:54:30 +00:00
Gilad Arnold
c94bee33a3 - cosmetic changes to debug logging. 2005-02-07 07:59:40 +00:00
Gilad Arnold
037c164728 - cosmetic fixes in hash module 
- new debug facility to show context-based (stack) debugging messages; turned on
  by corresponding compile switch (HAVE_DEBUG_CONTEXTS, currently hard-coded in
  debug.h)

- added use of dedicated queue module (supporting dynamic allocation and
  growable buffers).

- improvement w.r.t. full error handling in pobj.c: tackled some error cases
  with library calls (malloc, hash module calls, etc) -- but still there's a way
  to go.
 2005-02-07 07:11:52 +00:00
Sears Russell
e33319175f Fixed some cht bugs. Abort is flakey. 2005-02-06 03:48:12 +00:00
Gilad Arnold
f96061a1bd - implementation of growable queues; to replace fix-sized queues used 
in recursive traversal functions.
 2005-02-03 21:22:14 +00:00
Gilad Arnold
91f2c7c382 - moved pobj header internals back into the .c file (sorry Amir) 
- added pobj_is_persistent() call to replace the formerly used macro

- minor rearragement of common contants/macros

- minor fix to makefile
 2005-02-03 21:05:48 +00:00
Gilad Arnold
1fd827a59e - cosmetic: changes all shorthand for 'repository' from 'rep' to 'repo'... 
- handling error return value of hash_insert in two cases (still two other left)
 2005-02-03 20:43:15 +00:00
Gilad Arnold
570dc16f70 - minor: hash_insert returns an error in case of memory allocation failure 2005-02-03 20:40:31 +00:00
Gilad Arnold
c3229723ab - removed std99 and pedantic flags for GCC (caused annoying warnings 
with variable argumented macros).
 2005-02-03 20:39:11 +00:00
Sears Russell
a27787b2fa Cluster hash table now allows transactions to contain multiple requests. 2005-02-03 02:01:23 +00:00
Sears Russell
54ecc0d3c4 page->dirty is now set correctly (assuming LSN's are updated properly). pageFile.c makes use of this fact by ignoring write requests on clean pages. 2005-02-02 02:12:40 +00:00
Sears Russell
a4c13449b8 Bugfix for pageOrientedList. 2005-01-31 02:18:46 +00:00
Sears Russell
1260710a27 Got rid of another broken hash implementation... 2005-01-31 01:34:29 +00:00
Sears Russell
bb69197a65 Fixed blob implementation (it was impossible to distinguish between a blob and a record of length 12), removed jbhash, and ported cht to linearHashNTA. 2005-01-31 01:29:52 +00:00
Sears Russell
7cf5fdee6e Re-implemented pageOrientedListNTA from scratch. It's now more flexible, but relies on features that are not provided by Talloc/Tdealloc, so performance isn't as good as it could be. 2005-01-29 01:17:37 +00:00
Sears Russell
4d04155b0e An optimized, but still slow implementation of pageOrientedList. 2005-01-29 01:09:29 +00:00
Sears Russell
56d98adcbf Working, but slow and non-concurrent variable length key/value support for linear hashtable. 2005-01-28 21:28:23 +00:00
Sears Russell
17f76264aa Implemented pageOrientedList, also did some bug fixing, and added some support functions to alloc.c 2005-01-28 03:32:17 +00:00
Sears Russell
05934d296a Removed O_DIRECT since it breaks on 2.4 kernels. Need to figure out how to 
detect this in the build script so that systems that support O_DIRECT can use it.

(Note:  RedHat kernels silently fail on O_DIRECT)
 2005-01-24 19:58:09 +00:00
Sears Russell
3642716431 CHT cleanup, integrated CHT tests into 'make check' 2005-01-20 22:55:54 +00:00
Sears Russell
b04e71360f Documentation update. 2005-01-20 21:19:47 +00:00
Sears Russell
54edde96c2 Possible fix for older 2.4 kernels. (Also should improve performance...) 2005-01-20 00:26:09 +00:00
Sears Russell
23539f446a More-or-less working version of CHT. Needs a lot of performance/stability/code cleanup work. 2005-01-19 00:51:54 +00:00
Sears Russell
d7b846d370 CHT is starting to work. 2005-01-19 00:47:55 +00:00
Sears Russell
7dacf21069 Rearranged cht, moving some functionality into the 2pc library. Still 
need some test cases.
 2005-01-16 02:30:32 +00:00
Sears Russell
894e30085c libdfa ported to new hash table implementation. (Uses nested top actions.) 2005-01-15 23:55:49 +00:00
Sears Russell
ac0d413e58 - linkedListNTA is now re-entrant. 
- linearHashNTA now uses nested top actions, is re-entrant, and supports a
  non-re-entrant iterator
- 'recursive' nested top actions allow nested top actions to be nested within each
  other.
 2005-01-15 01:45:27 +00:00
Sears Russell
7a990886d2 The linked list implementation now uses nested top actions. (Also, many bugfixes regarding nested top actions were introduced.) 2005-01-14 10:08:10 +00:00
Sears Russell
360f0d15e2 Working, non-concurrent linear hash and linked list implementations. 
The idea is to implement the non-concurrent versions, and then convert
them to concurrent versions using nested top actions.
 2005-01-14 01:52:53 +00:00
Sears Russell
6198522971 Starting to rip the old jbhash implementation out, and replace it with a 
linear hash implementation.  In particular, I've implemented (sort of) an
iterator for linear hash, and (sort of) ported libdfa to linearhash.c

I say 'sort of' since this functionality is broken with this commit.  On the other hand, CVS should build now, and the tests under lladd will pass.  libdfa's new tests don't pass at the moment.
 2005-01-10 03:02:02 +00:00
Sears Russell
c45ed9f5d5 Network configuration parser is working 2005-01-02 01:37:05 +00:00
Sears Russell
6fa948cd74 Partially implemented config file parser. Validates, but does not populate NetworkSetup object yet. 2004-12-30 23:07:47 +00:00
Gilad Arnold
f119717b5e Bugfix: identification of non-persistent objects in recursive updates 
was bogus.
 2004-12-17 02:22:36 +00:00
Gilad Arnold
79f14a1230 Added some debug traces. 2004-12-17 02:20:28 +00:00
Gilad Arnold
819522d008 Bugfix: now reading the full object before comparing with memory image, 
upon recursive update.
 2004-12-17 02:11:25 +00:00
Gilad Arnold
06dcd82a7d Bugfix: size of temporary buffer increases until it can contain the 
currently traversed object.
 2004-12-17 01:50:37 +00:00
Gilad Arnold
6358b74bac Added some debug line. 2004-12-17 01:45:09 +00:00
Gilad Arnold
635561981a Bugfix: dump type info upon set_ref, is it has been (implicitly) change. 2004-12-17 01:32:19 +00:00
Gilad Arnold
879ca42890 Support for TsetRange(), to minimize logging overhead when only portions 
of objects are known to be modified.
 2004-12-17 00:38:52 +00:00
Gilad Arnold
a7b1b70f74 Added persistified recursive updates. Should be tested. 2004-12-16 23:41:57 +00:00
Gilad Arnold
58bed5beca Some changes to pobj.c to support operations for non-persistent (transient) 
objects. Enhancement to test-list program to work with transient objects
as well. Minor fix to header file (typo).
 2004-12-16 23:04:21 +00:00
Gilad Arnold
91eb96a113 Some changes: initial support for (un)persistify, range update, pobj_memset, 
IS_PERSISTENT macro. Most are not yet tested, but compilation is okay...
 2004-12-16 16:50:15 +00:00
Gilad Arnold
6502569a02 Switched to growable (unlimited) persistent objects / static references 
repositories. Has been tested with 15K objects and appears to work. Also
done some significant chnages to auxiliary modules in libpobj, and the
trivial test program (test-list).
 2004-12-16 09:00:07 +00:00
Gilad Arnold
e30aca6138 Temporary fix to avoid a segfault during execution of pobj test 
programs, that's caused by an on-stack allocation of a very big array...
Permanent (future) fix: switch to dynamically allocated hash tables.
 2004-12-15 01:27:17 +00:00
Gilad Arnold
36a0821acf Beginning to add strict typing for objects (minor change). 2004-12-14 19:45:04 +00:00
Gilad Arnold
773df6c690 Added custom memory calls for pobj internal structures. 2004-12-14 07:38:04 +00:00
Gilad Arnold
8137a0914d Disabled some development code in pobj.c, just so it passes compilation. 2004-12-14 01:33:25 +00:00
Gilad Arnold
4329801481 Removed redundant variable from Makefile.am. 2004-12-14 01:24:34 +00:00
Gilad Arnold
fc42d83efc Added pobj library sources, header file, and test programs. Small changes were 
made to src/Makefile.am in order to include src/pobj build, but no corresponding
change was made to test/Makefile.am (will be done in the near future).
 2004-12-13 21:51:11 +00:00
Sears Russell
55b0ddf1b6 Fixed nasty bug that was causing the LLADD header to be lost (!!) 2004-12-06 01:20:48 +00:00
Sears Russell
72070acb67 Implemented TsetRange, fixed bug in linear hash. 2004-12-03 00:27:47 +00:00
Sears Russell
443a90ad7c Implemented TrecordType() 2004-12-01 01:26:25 +00:00
Sears Russell
ded5c92c9d Support for nested top actions, with simple unit test. (Haven't tested for thread safety yet.) 2004-11-29 21:28:13 +00:00
Sears Russell
646f9dfca9 Implemented recover_split(), which is needed by the linear hash to recover properly from crashes. It hasn't been tested at all, and is not currently called by anything. 2004-11-29 02:35:37 +00:00
Sears Russell
2c23fb8a43 Did some misc. cleanup work on linearHash 2004-11-28 01:16:48 +00:00
Sears Russell
164c7cfd69 Linear Hash now survives Tdeinit/Tinit cycle (but not crashes!) 2004-11-24 23:26:15 +00:00
Sears Russell
80d4af4c9b Added getRecordType() 2004-11-24 23:25:36 +00:00
Sears Russell
910649a387 Web-site update, a few minor code changes from before the Freenix deadline. 2004-10-27 01:40:09 +00:00
Sears Russell
aead6a5a83 New benchmark, fixed for PROFILE_LATCHES. 2004-10-20 21:55:00 +00:00
Sears Russell
b59ba35ed4 parameter tweak.. 2004-10-20 06:54:39 +00:00
Sears Russell
70b8aca134 Set some constants to values appropriate for my desktop. 2004-10-20 06:48:29 +00:00
Sears Russell
af9e7adee0 Starting work on the actual graph-generation portion of the benchmarks. :) 2004-10-20 06:21:21 +00:00
Sears Russell
903ba4b97b Bugfix. 2004-10-19 22:58:56 +00:00
Sears Russell
ec99c0e258 Lots of optimizations; re-did on-disk layout for the linear hash table. 2004-10-19 21:16:37 +00:00
Sears Russell
6b265f28b0 Speedups, most notably in the logging subsystem. 2004-10-19 04:45:42 +00:00
Sears Russell
c850ce3a51 The logger now calls fwrite half as often as it used to. This could be improved significantly, but some latching details internal to the logger need to be worked out first. 2004-10-18 19:19:10 +00:00
Sears Russell
ec1276f26f Preliminary version of logical logging linear hash. (No latching yet, and there are some bugs re-opening a closed hash.) 2004-10-18 18:24:54 +00:00
Sears Russell
04cfc7da8e Optimized the 'naive' linear hash table, and moved to sourceforge. :) 2004-10-17 02:17:00 +00:00
Sears Russell
a078f25475 Implemented a linear hash from scratch. Concurrency hasn't been done, but it's probably correct in the single thread case. 2004-10-12 02:44:47 +00:00
Sears Russell
5064e3fac2 Implemented pages that store fixed sized records efficiently, and a java-style ArrayList data structure that efficiently supports (relatively) clustered, O(1) access time expandable arrays. (This will be used for the hash implementation...) 2004-10-06 06:08:09 +00:00
Sears Russell
9268b1d9cf Added support for gcov, and ran gprof. Shaved 10% off the runtime of check_page by optimizing rw.c 2004-10-04 05:24:12 +00:00
Sears Russell
96e7af92a4 Added test cases for Tprepare(), implemented some redo-only operations, and started to clean up record allocation/deallocation. 
Also, numerous bugfixes.
 2004-10-02 07:29:34 +00:00
Sears Russell
811bc5c710 Implemented a freespace manager that should safely allocate space, even in the face of crashes, and can reclaim unused space (unless an application opens more than one simultaneous transaction that performs allocations) 
Fixed some blob bugs (by adding extra fdatasync() calls).

Began factoring out the page management code so that it is an extenstion, and a less integral part of lladd.
 2004-08-21 00:03:30 +00:00
Sears Russell
b4d7883f66 Fixed regressions, major update of page / slotted interfaces in anticipation of moving LLADD towards a generic transactional page system. 2004-08-17 01:46:17 +00:00
Sears Russell
82e3fdb53a Added (untested) support for whole-page operations, lladdhash now works. 2004-08-03 02:04:56 +00:00
Sears Russell
e51759b517 Implemented indirect pages, and fixed some bugs here and there. (Still need to add transactional support for indirect pages...) 2004-07-31 00:27:55 +00:00
Sears Russell
a74c499dd7 Fixed data corruption bug in the freelist implementation, and corrected lsn updates. 2004-07-30 02:07:41 +00:00
Sears Russell
961b63af15 Rewrote / refactored page.h. Now, it uses a freelist to reclaim slotid's within a page, and is implemented as two seperate modules. The first handles lsn's, and (will soon) check the type of each page, and then delegate work to a particular implementation of the second module. 2004-07-30 01:28:39 +00:00
Sears Russell
c649ba7504 O_DIRECT is working now. 2004-07-27 22:04:59 +00:00
Sears Russell
3cbe112926 Moved some utility functions into their own file. 2004-07-27 21:33:00 +00:00
Sears Russell
78eb2cbf6a O_DIRECT (sort of...) 2004-07-27 21:30:54 +00:00
Sears Russell
54ba9b0347 Refactored bufferMananger / pageCache, so that bufferMananger handles loading an unloading pages, and pageCache only implements the replacement policy. 
Also, used memprof to detect and remove all memory leaks (at least all memory leaks that are encountered by check_transactional2), and fixed a number of misuses of the pblHash.
 2004-07-27 01:04:35 +00:00
Sears Russell
e53117a1a0 Implemented (but didn't test) iterator over the records in a page. 2004-07-26 22:01:09 +00:00
Sears Russell
8cb59f0c23 Blobs are now re-entrant. 2004-07-26 20:37:04 +00:00
Sears Russell
422198418a bufferMananger is (really!) re-entrant now, performed big refactoring to prevent deadlocks, check_transactional2 ran overnight without coring, w/ 25 threads, and 10000 allocate/write/reads per thread. (The whole test suite was run in a loop...) 2004-07-23 20:21:44 +00:00
Sears Russell
38ae73c146 Just committing some documentation updates before big refactoring. 2004-07-22 17:48:36 +00:00
Sears Russell
126ae31392 bufferMananger is *really* reentrant now! (I think) However, the same thread may not load a page more than once. So, LLADD has to be altered so that it never calls loadPage() more than once per operation (more efficient), and/or pin/unpin need to be implemented. An easy way to do pin/unpin is to have them pull things out of the cache replacement list, and then re-insert them. 2004-07-21 02:13:28 +00:00
Sears Russell
0ce77903fb Continuing work on multi-threading. r/w access to buffer manager getting close, but still buggy. 2004-07-20 03:40:57 +00:00
Sears Russell
490dd86c09 pageCache.c is now re-entrant. 2004-07-20 00:15:17 +00:00
Sears Russell
12fc5665ab pageCache is now re-entrant, in theory. 2004-07-15 00:42:36 +00:00
Sears Russell
9712e291e6 Moved page.h and some of the logging headers out of the public API. 2004-07-14 21:25:59 +00:00
Sears Russell
2b42451280 Cleaning up bufferManager / page for locking. Want to limit access to the Page struct. 2004-07-14 20:49:18 +00:00
Sears Russell
475b8141d3 page.c is re-entrant (mostly), and now reuses DeRalloced space properly. (For now, BufferManager still is not re-entrant, and also prevents space from being reused.) 2004-07-13 23:48:20 +00:00
Sears Russell
282d284035 Made pageWriteLSN static. 2004-07-09 22:05:33 +00:00
Sears Russell
d564b9ded6 More documentation fixes. 2004-07-06 21:41:33 +00:00
Sears Russell
370ad33210 Fixed groupings for LLADD API docs. 2004-07-06 20:59:36 +00:00
Sears Russell
e1d3bf7a0c Ported LLADD to Fedora, cleaned up autoconf setup, and numerouse #includes that were problematic. 2004-07-06 01:22:18 +00:00
Sears Russell
f8015a5687 Documentation update, added latch profiling tools. 2004-07-04 00:46:49 +00:00
Sears Russell
3349dbc6dc Log truncation. (But no checkpoints, so it doesn't get called... it does pass testing though. :) 2004-07-01 00:53:36 +00:00
Sears Russell
af152bd08e Logwriter can now handler partial log entries correctly (it ignores them) 
Buffer Mananger no longer steals pages pre-maturely
Alloc is an operation, and correct.
Caching broken out into a new module
Wrote a utility to translate a log file into human-readable ascii.
 2004-06-30 01:09:57 +00:00
Sears Russell
c4fac084b8 Cleaned out old cruft, such as commented out code, dead data structures, and dead files. sloccount went from $75,000 to $50,000 on src/lladd. ;) 2004-06-28 22:48:02 +00:00
Sears Russell
fab0e6cbbd Bugfixes ; blobs pass regression. Next stop: Delete old cruft. 2004-06-28 21:10:10 +00:00
Sears Russell
7e2f1fc5b2 Wrote blobmanager, didn't compile it yet. 2004-06-26 02:05:24 +00:00
Sears Russell
0f4dc72e18 blob manager commit. 2004-06-25 18:59:24 +00:00
Sears Russell
e083a333b7 Preliminary implementation of blobManager 2004-06-25 18:56:43 +00:00
Sears Russell
d1f76f0b4a Initial revision 2004-06-24 21:10:31 +00:00