greg/stasis-aries-wal
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Ran ESC-q on whole buffer.

Browse source
This commit is contained in:
Sears Russell 2005-03-29 03:00:26 +00:00
parent a561f990e5
commit 66801e3091
1 changed files with 58 additions and 53 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

99
doc/position-paper/LLADD.txt
View file

@ -25,38 +25,38 @@ collections, and collection implementations can be composed into more
sophisticated data structures. sophisticated data structures.
We have implemented LLADD (/yad/), an extensible transactional storage We have implemented LLADD (/yad/), an extensible transactional storage
implementation that takes a composable and layered approach to transactional implementation that takes a composable and layered approach to
storage. In other work, we show that its performance on transactional storage. In other work, we show that its performance on
traditional workloads is competitive with existing systems and show traditional workloads is competitive with existing systems and show
significant increases in throughput and significant increases in throughput and memory utilization on
memory utilization on specialized workloads.[XXX] specialized workloads.[XXX]
We further argue that because of its natural integration into standard We further argue that because of its natural integration into standard
system software development practices our library can be naturally extended into system software development practices our library can be naturally
networked and distributed domains. Typical write-ahead-logging extended into networked and distributed domains. Typical
protocols implicitly implement machine independent, reorderable log write-ahead-logging protocols implicitly implement machine
entries in order to implement logical undo. These two properties have independent, reorderable log entries in order to implement logical
been crucial in past system software designs, including undo. These two properties have been crucial in past system software
data replication, distribution, and conflict resolution algorithms. Therefore, we designs, including data replication, distribution, and conflict
plan to provide a networked, logical redo log as an application-level resolution algorithms. Therefore, we plan to provide a networked,
primitive, and to explore system designs that leverage these logical redo log as an application-level primitive, and to explore
primitives. system designs that leverage these primitives.
However, our approach assumes that application developers will However, our approach assumes that application developers will
correctly implement new transactional structures even though these data correctly implement new transactional structures even though these
structures are notoriously difficult to implement correctly. In this data structures are notoriously difficult to implement correctly. In
work we present our current attempts to address these concerns. this work we present our current attempts to address these concerns.
For such infrastructure to be generally useful, however, the For such infrastructure to be generally useful, however, the
functionality that it provides should be efficient, reliable functionality that it provides should be efficient, reliable and
and applicable to new application domains. We believe that ease of applicable to new application domains. We believe that ease of
development is a prerequisite to our other goals. development is a prerequisite to our other goals.
Application developers typically have a limited amount of time to Application developers typically have a limited amount of time to
spend implementing and verifying application-specific storage spend implementing and verifying application-specific storage
extensions, and bugs in these extensions affect data durability. extensions, and bugs in these extensions affect data durability.
While the underlying data structure algorithms tend to be While the underlying data structure algorithms tend to be simple and
simple and easily understood, performance tuning and verification of easily understood, performance tuning and verification of
implementation correctness is extremely difficult. implementation correctness is extremely difficult.
Recovery based algorithms must behave correctly during forward Recovery based algorithms must behave correctly during forward
@ -65,8 +65,8 @@ requirement is particularly difficult to verify due to the large
number of materialized page file states that could occur after a number of materialized page file states that could occur after a
crash. crash.
Fortunately, write-ahead-logging schemes such as ARIES make use Fortunately, write-ahead-logging schemes such as ARIES make use of
of nested-top-actions to vastly simplify the problem. Given the nested-top-actions to vastly simplify the problem. Given the
correctness of page based physical undo and redo, logical undo may correctness of page based physical undo and redo, logical undo may
assume that page spanning operations are applied to the data store assume that page spanning operations are applied to the data store
atomically. atomically.
@ -76,28 +76,31 @@ device driver implementations correctly adhere to complex operating
system kernel locking schemes[SLAM]. If we formalize LLADD's latching system kernel locking schemes[SLAM]. If we formalize LLADD's latching
and logging APIs, we believe that analyses such as these will be and logging APIs, we believe that analyses such as these will be
directly applicable, and allow us to verify that data structure directly applicable, and allow us to verify that data structure
behavior during recovery is equivalent to behavior during recovery is equivalent to its behavior on each prefix
its behavior on each prefix of the log produced during normal forward operation. of the log produced during normal forward operation.
By using coarse (one latch per logical operation) latching, By using coarse (one latch per logical operation) latching, we can
we can drastically reduce the size of this space, allowing drastically reduce the size of this space, allowing conventional
conventional state-state based search techniques (such as randomized state-state based search techniques (such as randomized or exhaustive
or exhaustive state-space searches, or simple unit testing techniques) state-space searches, or simple unit testing techniques) to be
to be practical. It has been shown that such coarse grained latching practical. It has been shown that such coarse grained latching can
can yield high performance concurrent data structures if yield high performance concurrent data structures if
semantics-preserving optimizations such as page prefetching are applied[ARIES/IM]. semantics-preserving optimizations such as page prefetching are
applied[ARIES/IM].
A separate approach toward static analysis of LLADD extensions A separate approach toward static analysis of LLADD extensions
involves compiler optimization techniques. Software built on top of layered API's frequently makes involves compiler optimization techniques. Software built on top of
repeated calls to low level functions that must repeat work. A common layered API's frequently makes repeated calls to low level functions
example in LLADD involves loops over data with good locality in the that must repeat work. A common example in LLADD involves loops over
page file. The vast majority of the time, these loops call high level data with good locality in the page file. The vast majority of the
API's that needlessly pin and unpin the same underlying data. time, these loops call high level API's that needlessly pin and unpin
the same underlying data.
The code for each of these high level API calls could be copied into many different The code for each of these high level API calls could be copied into
variants with different pinning/unpinning and latching/unlatching many different variants with different pinning/unpinning and
behavior, but this would greatly complicate the API that application developers latching/unlatching behavior, but this would greatly complicate the
must work with, and complicate any application code that make use of such optimizations. API that application developers must work with, and complicate any
application code that make use of such optimizations.
Compiler optimization techniques such as partial common subexpression Compiler optimization techniques such as partial common subexpression
elimination solve an analogous problem to remove unnecessary algebraic elimination solve an analogous problem to remove unnecessary algebraic
@ -107,11 +110,13 @@ of buffer manager and locking calls made by existing code at runtime.
Our implementation of LLADD is still unstable and inappropriate for Our implementation of LLADD is still unstable and inappropriate for
use on important data. We hope to validate our static analysis tools use on important data. We hope to validate our static analysis tools
by incorporating them into LLADD's development process as we increase by incorporating them into LLADD's development process as we increase
the reliability and overall quality of our implementation and its API's. the reliability and overall quality of our implementation and its
API's.
LLADD provides a set of tools that allow applications to implement custom LLADD provides a set of tools that allow applications to implement
transactional data structures and page layouts. This avoids "impedance custom transactional data structures and page layouts. This avoids
mismatch," simplifying applications and improving performance. "impedance mismatch," simplifying applications and improving
By adding support for automated code verification and transformations we hope performance. By adding support for automated code verification and
to make it easy to produce correct extensions and to allow simple, maintainable transformations we hope to make it easy to produce correct extensions
implementations to compete with carefully crafted, hand-optimized code. and to allow simple, maintainable implementations to compete with
carefully crafted, hand-optimized code.