EDoc update
This commit is contained in:
parent
5d5e400055
commit
c1db285f1e
19 changed files with 203 additions and 51 deletions
|
@ -55,6 +55,6 @@
|
||||||
<hr>
|
<hr>
|
||||||
|
|
||||||
<div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
|
<div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
|
||||||
<p><i>Generated by EDoc, May 20 2015, 11:11:34.</i></p>
|
<p><i>Generated by EDoc, May 20 2015, 12:36:35.</i></p>
|
||||||
</body>
|
</body>
|
||||||
</html>
|
</html>
|
||||||
|
|
|
@ -34,6 +34,6 @@
|
||||||
<hr>
|
<hr>
|
||||||
|
|
||||||
<div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
|
<div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
|
||||||
<p><i>Generated by EDoc, May 20 2015, 11:11:34.</i></p>
|
<p><i>Generated by EDoc, May 20 2015, 12:36:35.</i></p>
|
||||||
</body>
|
</body>
|
||||||
</html>
|
</html>
|
||||||
|
|
|
@ -174,6 +174,6 @@ distributed state of a single Machi Chain Replication chain.</p>
|
||||||
<hr>
|
<hr>
|
||||||
|
|
||||||
<div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
|
<div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
|
||||||
<p><i>Generated by EDoc, May 20 2015, 11:11:34.</i></p>
|
<p><i>Generated by EDoc, May 20 2015, 12:36:35.</i></p>
|
||||||
</body>
|
</body>
|
||||||
</html>
|
</html>
|
||||||
|
|
|
@ -10,27 +10,69 @@
|
||||||
<hr>
|
<hr>
|
||||||
|
|
||||||
<h1>Module machi_chain_repair</h1>
|
<h1>Module machi_chain_repair</h1>
|
||||||
<ul class="index"><li><a href="#description">Description</a></li><li><a href="#index">Function Index</a></li><li><a href="#functions">Function Details</a></li></ul>Erlang API for the Machi FLU TCP protocol version 1, with a
|
<ul class="index"><li><a href="#description">Description</a></li><li><a href="#index">Function Index</a></li><li><a href="#functions">Function Details</a></li></ul>Perform "chain repair", i.e., resynchronization of Machi file
|
||||||
proxy-process style API for hiding messy details such as TCP
|
contents and metadata as servers are (re-)added to the chain.
|
||||||
connection/disconnection with the remote Machi server.
|
|
||||||
|
|
||||||
|
|
||||||
<h2><a name="description">Description</a></h2><p>Erlang API for the Machi FLU TCP protocol version 1, with a
|
<h2><a name="description">Description</a></h2><p>Perform "chain repair", i.e., resynchronization of Machi file
|
||||||
proxy-process style API for hiding messy details such as TCP
|
contents and metadata as servers are (re-)added to the chain.</p>
|
||||||
connection/disconnection with the remote Machi server.</p>
|
|
||||||
|
|
||||||
<p>Machi is intentionally avoiding using distributed Erlang for
|
<p>The implementation here is a very basic one, and is probably a bit
|
||||||
Machi's communication. This design decision makes Erlang-side code
|
slower than the original "demo day" implementation at
|
||||||
more difficult & complex, but it's the price to pay for some
|
<a href="https://github.com/basho/machi/blob/master/prototype/demo-day-hack/file0_repair_server.escript" target="_top"><tt>https://github.com/basho/machi/blob/master/prototype/demo-day-hack/file0_repair_server.escript</tt></a></p>
|
||||||
language independence. Later in Machi's life cycle, we need to
|
|
||||||
(re-)implement some components in a non-Erlang/BEAM-based language.</p>
|
|
||||||
|
|
||||||
This module implements a "man in the middle" proxy between the
|
<p>It's so easy to bikeshed this into a 1 year programming exercise.</p>
|
||||||
Erlang client and Machi server (which is on the "far side" of a TCP
|
|
||||||
connection to somewhere). This proxy process will always execute
|
<p>General TODO note: There are a lot of areas for exploiting parallelism here.
|
||||||
on the same Erlang node as the Erlang client that uses it. The
|
I've set the bikeshed aside for now, but "make repair faster" has a
|
||||||
proxy is intended to be a stable, long-lived process that survives
|
lot of room for exploiting concurrency, overlapping reads & writes,
|
||||||
TCP communication problems with the remote server.
|
etc etc. There are also lots of different trade-offs to make with
|
||||||
|
regard to RAM use vs. disk use.</p>
|
||||||
|
|
||||||
|
<p>There's no reason why repair can't be done:</p>
|
||||||
|
|
||||||
|
<ol>
|
||||||
|
<li> Repair in parallel across multiple repairees ... Optimization.
|
||||||
|
</li>
|
||||||
|
<li> Repair multiple byte ranges concurrently ... Optimization.
|
||||||
|
</li>
|
||||||
|
<li><p> Use bigger chunks than the client originally used to write the file
|
||||||
|
... Optimization ... but it would be the easiest to implement, e.g. use
|
||||||
|
constant-sized 4MB chunks. Unfortuntely, it would also destroy
|
||||||
|
the ability to verify here that the chunk checksums are correct
|
||||||
|
*and* also propagate the correct checksum metadata to the
|
||||||
|
destination FLU.</p>
|
||||||
|
|
||||||
|
As an additional optimization, add a bit of #2 to start the next
|
||||||
|
read while the current write is still in progress.
|
||||||
|
</li>
|
||||||
|
<li> The current method centralizes the "smarts" required to compare
|
||||||
|
checksum differences ... move some computation to each FLU, then use
|
||||||
|
a Merkle- or other-compression-style scheme to reduce the amount of
|
||||||
|
data sent across a network.
|
||||||
|
</li>
|
||||||
|
</ol>
|
||||||
|
|
||||||
|
<p>Most/all of this could be executed in parallel on each FLU relative to
|
||||||
|
its own files. Then, in another TODO option, perhaps build a Merkle tree
|
||||||
|
or other summary of the local files and send that data structure to the
|
||||||
|
repair coordinator.</p>
|
||||||
|
|
||||||
|
<p>Also, as another TODO note, repair_both_present() in the
|
||||||
|
prototype/demo-day code uses an optimization of calculating the MD5
|
||||||
|
checksum of the chunk checksum data as it arrives, and if the two MD5s
|
||||||
|
match, then we consider the two files in sync. If there isn't a match,
|
||||||
|
then we sort the lines and try another MD5, and if they match, then we're
|
||||||
|
in sync. In theory, that's lower overhead than the procedure used here.</p>
|
||||||
|
|
||||||
|
NOTE that one reason I chose the "directives list" method is to have an
|
||||||
|
option, later, of choosing to repair a subset of repairee FLUs if there
|
||||||
|
is a big discrepency between out of sync files: e.g., if FLU x has N
|
||||||
|
bytes out of sync but FLU y has 50N bytes out of sync, then it's likely
|
||||||
|
better to repair x only so that x can return to the UPI list quickly.
|
||||||
|
Also, in the event that all repairees are roughly comparably out of sync,
|
||||||
|
then the repair network traffic can be minimized by reading each chunk
|
||||||
|
only once.
|
||||||
<h2><a name="index">Function Index</a></h2>
|
<h2><a name="index">Function Index</a></h2>
|
||||||
<table width="100%" border="1" cellspacing="0" cellpadding="2" summary="function index"><tr><td valign="top"><a href="#repair-7">repair/7</a></td><td></td></tr>
|
<table width="100%" border="1" cellspacing="0" cellpadding="2" summary="function index"><tr><td valign="top"><a href="#repair-7">repair/7</a></td><td></td></tr>
|
||||||
</table>
|
</table>
|
||||||
|
@ -44,6 +86,6 @@ language independence. Later in Machi's life cycle, we need to
|
||||||
<hr>
|
<hr>
|
||||||
|
|
||||||
<div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
|
<div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
|
||||||
<p><i>Generated by EDoc, May 20 2015, 11:11:34.</i></p>
|
<p><i>Generated by EDoc, May 20 2015, 12:36:35.</i></p>
|
||||||
</body>
|
</body>
|
||||||
</html>
|
</html>
|
||||||
|
|
|
@ -166,6 +166,6 @@ list.</p>
|
||||||
<hr>
|
<hr>
|
||||||
|
|
||||||
<div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
|
<div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
|
||||||
<p><i>Generated by EDoc, May 20 2015, 11:11:34.</i></p>
|
<p><i>Generated by EDoc, May 20 2015, 12:36:35.</i></p>
|
||||||
</body>
|
</body>
|
||||||
</html>
|
</html>
|
||||||
|
|
|
@ -18,6 +18,65 @@
|
||||||
<h2><a name="description">Description</a></h2><p>Erlang API for the Machi client-implemented Chain Replication
|
<h2><a name="description">Description</a></h2><p>Erlang API for the Machi client-implemented Chain Replication
|
||||||
(CORFU-style) protocol.</p>
|
(CORFU-style) protocol.</p>
|
||||||
|
|
||||||
|
<p>See also the docs for <a href="machi_flu1_client.html"><code>machi_flu1_client</code></a> for additional
|
||||||
|
details on data types and operation descriptions.</p>
|
||||||
|
|
||||||
|
<p>The API here is much simpler than the <a href="machi_flu1_client.html"><code>machi_flu1_client</code></a> or
|
||||||
|
<a href="machi_proxy_flu1_client.html"><code>machi_proxy_flu1_client</code></a> APIs. This module's API is a
|
||||||
|
proposed simple-but-complete form for clients who are not
|
||||||
|
interested in being an active participant in a Machi cluster and to
|
||||||
|
have the responsibility for Machi internals, i.e., client-side
|
||||||
|
Chain Replication, client-side read repair, client-side tracking of
|
||||||
|
internal Machi epoch & projection changes, etc.</p>
|
||||||
|
|
||||||
|
<p>This client is implemented as a long-lived Erlang process using
|
||||||
|
<code>gen_server</code>-style OTP code practice. A naive client can expect
|
||||||
|
that this process will manage all transient TCP session
|
||||||
|
disconnections and Machi chain reconfigurations. This client's
|
||||||
|
efforts are best-effort and can require some time to retry
|
||||||
|
operations in certain failure cases, i.e., up to several seconds
|
||||||
|
during a Machi projection & epoch change when a new server is
|
||||||
|
added to the chain.</p>
|
||||||
|
|
||||||
|
<p>Doc TODO: Once this API stabilizes, add all relevant data type details
|
||||||
|
to the EDoc here.</p>
|
||||||
|
|
||||||
|
|
||||||
|
<h4><a name="Missing_API_features">Missing API features</a></h4>
|
||||||
|
|
||||||
|
<p>So far, there is one missing client API feature that ought to be
|
||||||
|
added to Machi in the near future: more flexible checksum
|
||||||
|
management.</p>
|
||||||
|
|
||||||
|
<p>Add a <code>source</code> annotation to all checksums to indicate where the
|
||||||
|
checksum was calculated. For example,</p>
|
||||||
|
|
||||||
|
<ul>
|
||||||
|
|
||||||
|
<li> Calculated by client that performed the original chunk append,
|
||||||
|
</li>
|
||||||
|
|
||||||
|
<li> Calculated by the 1st Machi server to receive an
|
||||||
|
un-checksummed append request
|
||||||
|
</li>
|
||||||
|
|
||||||
|
<li> Re-calculated by Machi to manage fewer checksums of blocks of
|
||||||
|
data larger than the original client-specified chunks.
|
||||||
|
</li>
|
||||||
|
</ul>
|
||||||
|
|
||||||
|
<p>Client-side checksums would be the "strongest" type of
|
||||||
|
checksum, meaning that any data corruption (of the original
|
||||||
|
data and/or of the checksum itself) can be detected after the
|
||||||
|
client-side calculation. There are too many horror stories on
|
||||||
|
The Net about IP PDUs that are corrupted but unnoticed due to
|
||||||
|
weak TCP checksums, buggy hardware, buggy OS drivers, etc.
|
||||||
|
Checksum versioning is also desirable if/when the current checksum
|
||||||
|
implementation changes from SHA-1 to something else.</p>
|
||||||
|
|
||||||
|
|
||||||
|
<h4><a name="Implementation_notes">Implementation notes</a></h4>
|
||||||
|
|
||||||
<p>The major operation processing is implemented in a state machine-like
|
<p>The major operation processing is implemented in a state machine-like
|
||||||
manner. Before attempting an operation <code>X</code>, there's an initial
|
manner. Before attempting an operation <code>X</code>, there's an initial
|
||||||
operation <code>pre-X</code> that takes care of updating the epoch id,
|
operation <code>pre-X</code> that takes care of updating the epoch id,
|
||||||
|
@ -56,7 +115,7 @@ function name.)</p>
|
||||||
<tr><td valign="top"><a href="#list_files-2">list_files/2</a></td><td>Fetch the list of all files on the remote FLU.</td></tr>
|
<tr><td valign="top"><a href="#list_files-2">list_files/2</a></td><td>Fetch the list of all files on the remote FLU.</td></tr>
|
||||||
<tr><td valign="top"><a href="#quit-1">quit/1</a></td><td>Quit & close the connection to remote FLU and stop our
|
<tr><td valign="top"><a href="#quit-1">quit/1</a></td><td>Quit & close the connection to remote FLU and stop our
|
||||||
proxy process.</td></tr>
|
proxy process.</td></tr>
|
||||||
<tr><td valign="top"><a href="#read_chunk-4">read_chunk/4</a></td><td></td></tr>
|
<tr><td valign="top"><a href="#read_chunk-4">read_chunk/4</a></td><td>Read a chunk of data of size <code>Size</code> from <code>File</code> at <code>Offset</code>.</td></tr>
|
||||||
<tr><td valign="top"><a href="#read_chunk-5">read_chunk/5</a></td><td>Read a chunk of data of size <code>Size</code> from <code>File</code> at <code>Offset</code>.</td></tr>
|
<tr><td valign="top"><a href="#read_chunk-5">read_chunk/5</a></td><td>Read a chunk of data of size <code>Size</code> from <code>File</code> at <code>Offset</code>.</td></tr>
|
||||||
<tr><td valign="top"><a href="#start_link-1">start_link/1</a></td><td>Start a local, long-lived process that will be our steady
|
<tr><td valign="top"><a href="#start_link-1">start_link/1</a></td><td>Start a local, long-lived process that will be our steady
|
||||||
& reliable communication proxy with the fickle & flaky
|
& reliable communication proxy with the fickle & flaky
|
||||||
|
@ -144,7 +203,7 @@ function name.)</p>
|
||||||
<h3 class="function"><a name="read_chunk-4">read_chunk/4</a></h3>
|
<h3 class="function"><a name="read_chunk-4">read_chunk/4</a></h3>
|
||||||
<div class="spec">
|
<div class="spec">
|
||||||
<p><tt>read_chunk(PidSpec, File, Offset, Size) -> any()</tt></p>
|
<p><tt>read_chunk(PidSpec, File, Offset, Size) -> any()</tt></p>
|
||||||
</div>
|
</div><p>Read a chunk of data of size <code>Size</code> from <code>File</code> at <code>Offset</code>.</p>
|
||||||
|
|
||||||
<h3 class="function"><a name="read_chunk-5">read_chunk/5</a></h3>
|
<h3 class="function"><a name="read_chunk-5">read_chunk/5</a></h3>
|
||||||
<div class="spec">
|
<div class="spec">
|
||||||
|
@ -165,6 +224,6 @@ function name.)</p>
|
||||||
<hr>
|
<hr>
|
||||||
|
|
||||||
<div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
|
<div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
|
||||||
<p><i>Generated by EDoc, May 20 2015, 11:11:34.</i></p>
|
<p><i>Generated by EDoc, May 20 2015, 12:36:35.</i></p>
|
||||||
</body>
|
</body>
|
||||||
</html>
|
</html>
|
||||||
|
|
|
@ -21,7 +21,18 @@ Please see the EDoc "Overview" for details about the FLU as a
|
||||||
primitive file server process vs. the larger Machi design of a FLU
|
primitive file server process vs. the larger Machi design of a FLU
|
||||||
as a sequencer + file server + chain manager group of processes.</p>
|
as a sequencer + file server + chain manager group of processes.</p>
|
||||||
|
|
||||||
<p>For the moment, this module also implements a rudimentary TCP-based
|
<p>The FLU is named after the CORFU server "FLU" or "FLash Unit" server.</p>
|
||||||
|
|
||||||
|
<h4><a name="Protocol_origins">Protocol origins</a></h4>
|
||||||
|
|
||||||
|
<p>The protocol implemented here is an artisanal, hand-crafted, silly
|
||||||
|
thing that was very quick to put together for a "demo day" proof of
|
||||||
|
concept. It will almost certainly be replaced with something else,
|
||||||
|
both in terms of wire format and better code separation of
|
||||||
|
serialization/deserialization vs. network transport management,
|
||||||
|
etc.</p>
|
||||||
|
|
||||||
|
<p>For the moment, this module implements a rudimentary TCP-based
|
||||||
protocol as the sole supported access method to the server,
|
protocol as the sole supported access method to the server,
|
||||||
sequencer, and projection store. Conceptually, those three
|
sequencer, and projection store. Conceptually, those three
|
||||||
services are independent and ought to have their own protocols. As
|
services are independent and ought to have their own protocols. As
|
||||||
|
@ -30,7 +41,7 @@ compatibility. Furthermore, from the perspective of failure
|
||||||
detection, it is very convenient that all three FLU-related
|
detection, it is very convenient that all three FLU-related
|
||||||
services are accessed using the same single TCP port.</p>
|
services are accessed using the same single TCP port.</p>
|
||||||
|
|
||||||
<p>The FLU is named after the CORFU server "FLU" or "FLash Unit" server.</p>
|
<h4><a name="TODO_items">TODO items</a></h4>
|
||||||
|
|
||||||
<p>TODO There is a major missing feature in this FLU implementation:
|
<p>TODO There is a major missing feature in this FLU implementation:
|
||||||
there is no "write-once" enforcement for any position in a Machi
|
there is no "write-once" enforcement for any position in a Machi
|
||||||
|
@ -90,6 +101,6 @@ replication/chain repair.</p>
|
||||||
<hr>
|
<hr>
|
||||||
|
|
||||||
<div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
|
<div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
|
||||||
<p><i>Generated by EDoc, May 20 2015, 11:11:34.</i></p>
|
<p><i>Generated by EDoc, May 20 2015, 12:36:35.</i></p>
|
||||||
</body>
|
</body>
|
||||||
</html>
|
</html>
|
||||||
|
|
|
@ -13,7 +13,34 @@
|
||||||
<ul class="index"><li><a href="#description">Description</a></li><li><a href="#types">Data Types</a></li><li><a href="#index">Function Index</a></li><li><a href="#functions">Function Details</a></li></ul>Erlang API for the Machi FLU TCP protocol version 1.
|
<ul class="index"><li><a href="#description">Description</a></li><li><a href="#types">Data Types</a></li><li><a href="#index">Function Index</a></li><li><a href="#functions">Function Details</a></li></ul>Erlang API for the Machi FLU TCP protocol version 1.
|
||||||
|
|
||||||
|
|
||||||
<h2><a name="description">Description</a></h2>Erlang API for the Machi FLU TCP protocol version 1.
|
<h2><a name="description">Description</a></h2><p>Erlang API for the Machi FLU TCP protocol version 1.</p>
|
||||||
|
|
||||||
|
<p>This client API handles low-level PDU serialization/deserialization
|
||||||
|
and low-level TCP session management, e.g. open, receive, write,
|
||||||
|
close. The API for higher-level session management and Machi state
|
||||||
|
management can be found in <a href="machi_proxy_flu1_client.html"><code>machi_proxy_flu1_client</code></a> and
|
||||||
|
<a href="machi_cr_client.html"><code>machi_cr_client</code></a>.</p>
|
||||||
|
|
||||||
|
<p>TODO This EDoc was written first, and the EDoc and also <code>-type</code> and
|
||||||
|
<code>-spec</code> definitions for <a href="machi_proxy_flu1_client.html"><code>machi_proxy_flu1_client</code></a> and <a href="machi_cr_client.html"><code>machi_cr_client</code></a> must be improved.</p>
|
||||||
|
|
||||||
|
<h4><a name="Protocol_origins">Protocol origins</a></h4>
|
||||||
|
|
||||||
|
<p>The protocol implemented here is an artisanal, hand-crafted, silly
|
||||||
|
thing that was very quick to put together for a "demo day" proof of
|
||||||
|
concept. It will almost certainly be replaced with something else,
|
||||||
|
both in terms of wire format and better code separation of
|
||||||
|
serialization/deserialization vs. network transport management,
|
||||||
|
etc.</p>
|
||||||
|
|
||||||
|
For the moment, this module implements a rudimentary TCP-based
|
||||||
|
protocol as the sole supported access method to the server,
|
||||||
|
sequencer, and projection store. Conceptually, those three
|
||||||
|
services are independent and ought to have their own protocols. As
|
||||||
|
a practical matter, there is no need for wire protocol
|
||||||
|
compatibility. Furthermore, from the perspective of failure
|
||||||
|
detection, it is very convenient that all three FLU-related
|
||||||
|
services are accessed using the same single TCP port.
|
||||||
<h2><a name="types">Data Types</a></h2>
|
<h2><a name="types">Data Types</a></h2>
|
||||||
|
|
||||||
<h3 class="typedecl"><a name="type-chunk">chunk()</a></h3>
|
<h3 class="typedecl"><a name="type-chunk">chunk()</a></h3>
|
||||||
|
@ -337,6 +364,6 @@
|
||||||
<hr>
|
<hr>
|
||||||
|
|
||||||
<div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
|
<div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
|
||||||
<p><i>Generated by EDoc, May 20 2015, 11:11:34.</i></p>
|
<p><i>Generated by EDoc, May 20 2015, 12:36:35.</i></p>
|
||||||
</body>
|
</body>
|
||||||
</html>
|
</html>
|
||||||
|
|
|
@ -11,12 +11,22 @@
|
||||||
|
|
||||||
<h1>Module machi_flu_psup</h1>
|
<h1>Module machi_flu_psup</h1>
|
||||||
<ul class="index"><li><a href="#description">Description</a></li><li><a href="#index">Function Index</a></li><li><a href="#functions">Function Details</a></li></ul>Supervisor for Machi FLU servers and their related support
|
<ul class="index"><li><a href="#description">Description</a></li><li><a href="#index">Function Index</a></li><li><a href="#functions">Function Details</a></li></ul>Supervisor for Machi FLU servers and their related support
|
||||||
servers.
|
servers.
|
||||||
|
|
||||||
<p><b>Behaviours:</b> <a href="supervisor.html"><tt>supervisor</tt></a>.</p>
|
<p><b>Behaviours:</b> <a href="supervisor.html"><tt>supervisor</tt></a>.</p>
|
||||||
|
|
||||||
<h2><a name="description">Description</a></h2>Supervisor for Machi FLU servers and their related support
|
<h2><a name="description">Description</a></h2><p>Supervisor for Machi FLU servers and their related support
|
||||||
servers.
|
servers.</p>
|
||||||
|
|
||||||
|
<p>Our parent supervisor, <a href="machi_flu_sup.html"><code>machi_flu_sup</code></a>, is responsible for
|
||||||
|
managing FLUs as a single entity. However, the actual
|
||||||
|
implementation of a FLU includes three major Erlang processes (not
|
||||||
|
including support/worker procs): the FLU itself, the FLU's
|
||||||
|
projection store, and the FLU's local chain manager. This
|
||||||
|
supervisor is responsible for managing those three major services
|
||||||
|
as a single "package", to be started & stopped together.</p>
|
||||||
|
|
||||||
|
<img src="/doc/supervisor-2flus.png">
|
||||||
<h2><a name="index">Function Index</a></h2>
|
<h2><a name="index">Function Index</a></h2>
|
||||||
<table width="100%" border="1" cellspacing="0" cellpadding="2" summary="function index"><tr><td valign="top"><a href="#init-1">init/1</a></td><td></td></tr>
|
<table width="100%" border="1" cellspacing="0" cellpadding="2" summary="function index"><tr><td valign="top"><a href="#init-1">init/1</a></td><td></td></tr>
|
||||||
<tr><td valign="top"><a href="#make_mgr_supname-1">make_mgr_supname/1</a></td><td></td></tr>
|
<tr><td valign="top"><a href="#make_mgr_supname-1">make_mgr_supname/1</a></td><td></td></tr>
|
||||||
|
@ -72,6 +82,6 @@
|
||||||
<hr>
|
<hr>
|
||||||
|
|
||||||
<div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
|
<div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
|
||||||
<p><i>Generated by EDoc, May 20 2015, 11:11:34.</i></p>
|
<p><i>Generated by EDoc, May 20 2015, 12:36:35.</i></p>
|
||||||
</body>
|
</body>
|
||||||
</html>
|
</html>
|
||||||
|
|
|
@ -36,6 +36,6 @@
|
||||||
<hr>
|
<hr>
|
||||||
|
|
||||||
<div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
|
<div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
|
||||||
<p><i>Generated by EDoc, May 20 2015, 11:11:34.</i></p>
|
<p><i>Generated by EDoc, May 20 2015, 12:36:35.</i></p>
|
||||||
</body>
|
</body>
|
||||||
</html>
|
</html>
|
||||||
|
|
|
@ -89,6 +89,6 @@ correctly.</p>
|
||||||
<hr>
|
<hr>
|
||||||
|
|
||||||
<div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
|
<div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
|
||||||
<p><i>Generated by EDoc, May 20 2015, 11:11:34.</i></p>
|
<p><i>Generated by EDoc, May 20 2015, 12:36:35.</i></p>
|
||||||
</body>
|
</body>
|
||||||
</html>
|
</html>
|
||||||
|
|
|
@ -164,6 +164,6 @@ module's API.</p>
|
||||||
<hr>
|
<hr>
|
||||||
|
|
||||||
<div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
|
<div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
|
||||||
<p><i>Generated by EDoc, May 20 2015, 11:11:34.</i></p>
|
<p><i>Generated by EDoc, May 20 2015, 12:36:35.</i></p>
|
||||||
</body>
|
</body>
|
||||||
</html>
|
</html>
|
||||||
|
|
|
@ -23,15 +23,18 @@ connection/disconnection with the remote Machi server.</p>
|
||||||
<p>Machi is intentionally avoiding using distributed Erlang for
|
<p>Machi is intentionally avoiding using distributed Erlang for
|
||||||
Machi's communication. This design decision makes Erlang-side code
|
Machi's communication. This design decision makes Erlang-side code
|
||||||
more difficult & complex, but it's the price to pay for some
|
more difficult & complex, but it's the price to pay for some
|
||||||
language independence. Later in Machi's life cycle, we need to
|
language independence. Later in Machi's life cycle, we may (?) need to
|
||||||
(re-)implement some components in a non-Erlang/BEAM-based language.</p>
|
(re-)implement some components in a non-Erlang/BEAM-based language.</p>
|
||||||
|
|
||||||
This module implements a "man in the middle" proxy between the
|
<p>This module implements a "man in the middle" proxy between the
|
||||||
Erlang client and Machi server (which is on the "far side" of a TCP
|
Erlang client and Machi server (which is on the "far side" of a TCP
|
||||||
connection to somewhere). This proxy process will always execute
|
connection to somewhere). This proxy process will always execute
|
||||||
on the same Erlang node as the Erlang client that uses it. The
|
on the same Erlang node as the Erlang client that uses it. The
|
||||||
proxy is intended to be a stable, long-lived process that survives
|
proxy is intended to be a stable, long-lived process that survives
|
||||||
TCP communication problems with the remote server.
|
TCP communication problems with the remote server.</p>
|
||||||
|
|
||||||
|
For a higher level interface, see <a href="machi_cr_client.html"><code>machi_cr_client</code></a>.
|
||||||
|
For a lower level interface, see <a href="machi_flu1_client.html"><code>machi_flu1_client</code></a>.
|
||||||
<h2><a name="index">Function Index</a></h2>
|
<h2><a name="index">Function Index</a></h2>
|
||||||
<table width="100%" border="1" cellspacing="0" cellpadding="2" summary="function index"><tr><td valign="top"><a href="#append_chunk-4">append_chunk/4</a></td><td>Append a chunk (binary- or iolist-style) of data to a file
|
<table width="100%" border="1" cellspacing="0" cellpadding="2" summary="function index"><tr><td valign="top"><a href="#append_chunk-4">append_chunk/4</a></td><td>Append a chunk (binary- or iolist-style) of data to a file
|
||||||
with <code>Prefix</code>.</td></tr>
|
with <code>Prefix</code>.</td></tr>
|
||||||
|
@ -285,6 +288,6 @@ language independence. Later in Machi's life cycle, we need to
|
||||||
<hr>
|
<hr>
|
||||||
|
|
||||||
<div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
|
<div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
|
||||||
<p><i>Generated by EDoc, May 20 2015, 11:11:34.</i></p>
|
<p><i>Generated by EDoc, May 20 2015, 12:36:35.</i></p>
|
||||||
</body>
|
</body>
|
||||||
</html>
|
</html>
|
||||||
|
|
|
@ -18,6 +18,6 @@
|
||||||
the future?<hr>
|
the future?<hr>
|
||||||
|
|
||||||
<div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
|
<div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
|
||||||
<p><i>Generated by EDoc, May 20 2015, 11:11:34.</i></p>
|
<p><i>Generated by EDoc, May 20 2015, 12:36:35.</i></p>
|
||||||
</body>
|
</body>
|
||||||
</html>
|
</html>
|
||||||
|
|
|
@ -34,6 +34,6 @@
|
||||||
<hr>
|
<hr>
|
||||||
|
|
||||||
<div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
|
<div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
|
||||||
<p><i>Generated by EDoc, May 20 2015, 11:11:34.</i></p>
|
<p><i>Generated by EDoc, May 20 2015, 12:36:35.</i></p>
|
||||||
</body>
|
</body>
|
||||||
</html>
|
</html>
|
||||||
|
|
|
@ -163,6 +163,6 @@
|
||||||
<hr>
|
<hr>
|
||||||
|
|
||||||
<div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
|
<div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
|
||||||
<p><i>Generated by EDoc, May 20 2015, 11:11:34.</i></p>
|
<p><i>Generated by EDoc, May 20 2015, 12:36:35.</i></p>
|
||||||
</body>
|
</body>
|
||||||
</html>
|
</html>
|
||||||
|
|
|
@ -337,6 +337,6 @@
|
||||||
<hr>
|
<hr>
|
||||||
|
|
||||||
<div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
|
<div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
|
||||||
<p><i>Generated by EDoc, May 20 2015, 11:11:34.</i></p>
|
<p><i>Generated by EDoc, May 20 2015, 12:36:35.</i></p>
|
||||||
</body>
|
</body>
|
||||||
</html>
|
</html>
|
||||||
|
|
|
@ -180,6 +180,6 @@ the code, is a reminder signal of unfinished work.</p>
|
||||||
|
|
||||||
<hr>
|
<hr>
|
||||||
<div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
|
<div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
|
||||||
<p><i>Generated by EDoc, May 20 2015, 11:11:34.</i></p>
|
<p><i>Generated by EDoc, May 20 2015, 12:36:35.</i></p>
|
||||||
</body>
|
</body>
|
||||||
</html>
|
</html>
|
||||||
|
|
BIN
images/supervisor-2flus.png
Normal file
BIN
images/supervisor-2flus.png
Normal file
Binary file not shown.
After Width: | Height: | Size: 34 KiB |
Loading…
Reference in a new issue