245 lines
12 KiB
HTML
245 lines
12 KiB
HTML
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
|
|
<html>
|
|
<head>
|
|
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
|
|
<title>Module machi_cr_client</title>
|
|
<link rel="stylesheet" type="text/css" href="stylesheet.css" title="EDoc">
|
|
</head>
|
|
<body bgcolor="white">
|
|
<div class="navbar"><a name="#navbar_top"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
|
|
<hr>
|
|
|
|
<h1>Module machi_cr_client</h1>
|
|
<ul class="index"><li><a href="#description">Description</a></li><li><a href="#index">Function Index</a></li><li><a href="#functions">Function Details</a></li></ul>Erlang API for the Machi client-implemented Chain Replication
|
|
(CORFU-style) protocol.
|
|
|
|
<p><b>Behaviours:</b> <a href="gen_server.html"><tt>gen_server</tt></a>.</p>
|
|
|
|
<h2><a name="description">Description</a></h2><p>Erlang API for the Machi client-implemented Chain Replication
|
|
(CORFU-style) protocol.</p>
|
|
|
|
<p>See also the docs for <a href="machi_flu1_client.html"><code>machi_flu1_client</code></a> for additional
|
|
details on data types and operation descriptions.</p>
|
|
|
|
<p>The API here is much simpler than the <a href="machi_flu1_client.html"><code>machi_flu1_client</code></a> or
|
|
<a href="machi_proxy_flu1_client.html"><code>machi_proxy_flu1_client</code></a> APIs. This module's API is a
|
|
proposed simple-but-complete form for clients who are not
|
|
interested in being an active participant in a Machi cluster and to
|
|
have the responsibility for Machi internals, i.e., client-side
|
|
Chain Replication, client-side read repair, client-side tracking of
|
|
internal Machi epoch & projection changes, etc.</p>
|
|
|
|
<p>This client is implemented as a long-lived Erlang process using
|
|
<code>gen_server</code>-style OTP code practice. A naive client can expect
|
|
that this process will manage all transient TCP session
|
|
disconnections and Machi chain reconfigurations. This client's
|
|
efforts are best-effort and can require some time to retry
|
|
operations in certain failure cases, i.e., up to several seconds
|
|
during a Machi projection & epoch change when a new server is
|
|
added to the chain.</p>
|
|
|
|
<p>Doc TODO: Once this API stabilizes, add all relevant data type details
|
|
to the EDoc here.</p>
|
|
|
|
|
|
<h4><a name="Missing_API_features">Missing API features</a></h4>
|
|
|
|
<p>So far, there is one missing client API feature that ought to be
|
|
added to Machi in the near future: more flexible checksum
|
|
management.</p>
|
|
|
|
<p>Add a <code>source</code> annotation to all checksums to indicate where the
|
|
checksum was calculated. For example,</p>
|
|
|
|
<ul>
|
|
|
|
<li> Calculated by client that performed the original chunk append,
|
|
</li>
|
|
|
|
<li> Calculated by the 1st Machi server to receive an
|
|
un-checksummed append request
|
|
</li>
|
|
|
|
<li> Re-calculated by Machi to manage fewer checksums of blocks of
|
|
data larger than the original client-specified chunks.
|
|
</li>
|
|
</ul>
|
|
|
|
<p>Client-side checksums would be the "strongest" type of
|
|
checksum, meaning that any data corruption (of the original
|
|
data and/or of the checksum itself) can be detected after the
|
|
client-side calculation. There are too many horror stories on
|
|
The Net about IP PDUs that are corrupted but unnoticed due to
|
|
weak TCP checksums, buggy hardware, buggy OS drivers, etc.
|
|
Checksum versioning is also desirable if/when the current checksum
|
|
implementation changes from SHA-1 to something else.</p>
|
|
|
|
|
|
<h4><a name="Implementation_notes">Implementation notes</a></h4>
|
|
|
|
<p>The major operation processing is implemented in a state machine-like
|
|
manner. Before attempting an operation <code>X</code>, there's an initial
|
|
operation <code>pre-X</code> that takes care of updating the epoch id,
|
|
restarting client protocol proxies, and if there's any server
|
|
instability (e.g. some server is wedged), then insert some sleep
|
|
time. When the chain appears to have stabilized, then we try the <code>X</code>
|
|
operation again.</p>
|
|
|
|
<p>Function name for the <code>pre-X</code> stuff is usually <code>X()</code>, and the
|
|
function name for the <code>X</code> stuff is usually <code>X2()</code>. (I.e., the <code>X</code>
|
|
stuff follows after <code>pre-X</code> and therefore has a <code>2</code> suffix on the
|
|
function name.)</p>
|
|
|
|
In the case of read repair, there are two stages: find the value to
|
|
perform the repair, then perform the repair writes. In the case of
|
|
the repair writes, the <code>pre-X</code> function is named <code>read_repair3()</code>,
|
|
and the <code>X</code> function is named <code>read_repair4()</code>.
|
|
|
|
<h2><a name="index">Function Index</a></h2>
|
|
<table width="100%" border="1" cellspacing="0" cellpadding="2" summary="function index"><tr><td valign="top"><a href="#append_chunk-3">append_chunk/3</a></td><td>Append a chunk (binary- or iolist-style) of data to a file
|
|
with <code>Prefix</code>.</td></tr>
|
|
<tr><td valign="top"><a href="#append_chunk-4">append_chunk/4</a></td><td>Append a chunk (binary- or iolist-style) of data to a file
|
|
with <code>Prefix</code>.</td></tr>
|
|
<tr><td valign="top"><a href="#append_chunk_extra-4">append_chunk_extra/4</a></td><td>Append a chunk (binary- or iolist-style) of data to a file
|
|
with <code>Prefix</code>.</td></tr>
|
|
<tr><td valign="top"><a href="#append_chunk_extra-5">append_chunk_extra/5</a></td><td>Append a chunk (binary- or iolist-style) of data to a file
|
|
with <code>Prefix</code>.</td></tr>
|
|
<tr><td valign="top"><a href="#checksum_list-2">checksum_list/2</a></td><td>Fetch the list of chunk checksums for <code>File</code>.</td></tr>
|
|
<tr><td valign="top"><a href="#checksum_list-3">checksum_list/3</a></td><td>Fetch the list of chunk checksums for <code>File</code>.</td></tr>
|
|
<tr><td valign="top"><a href="#code_change-3">code_change/3</a></td><td></td></tr>
|
|
<tr><td valign="top"><a href="#handle_call-3">handle_call/3</a></td><td></td></tr>
|
|
<tr><td valign="top"><a href="#handle_cast-2">handle_cast/2</a></td><td></td></tr>
|
|
<tr><td valign="top"><a href="#handle_info-2">handle_info/2</a></td><td></td></tr>
|
|
<tr><td valign="top"><a href="#init-1">init/1</a></td><td></td></tr>
|
|
<tr><td valign="top"><a href="#list_files-1">list_files/1</a></td><td>Fetch the list of all files on the remote FLU.</td></tr>
|
|
<tr><td valign="top"><a href="#list_files-2">list_files/2</a></td><td>Fetch the list of all files on the remote FLU.</td></tr>
|
|
<tr><td valign="top"><a href="#quit-1">quit/1</a></td><td>Quit & close the connection to remote FLU and stop our
|
|
proxy process.</td></tr>
|
|
<tr><td valign="top"><a href="#read_chunk-4">read_chunk/4</a></td><td>Read a chunk of data of size <code>Size</code> from <code>File</code> at <code>Offset</code>.</td></tr>
|
|
<tr><td valign="top"><a href="#read_chunk-5">read_chunk/5</a></td><td>Read a chunk of data of size <code>Size</code> from <code>File</code> at <code>Offset</code>.</td></tr>
|
|
<tr><td valign="top"><a href="#start_link-1">start_link/1</a></td><td>Start a local, long-lived process that will be our steady
|
|
& reliable communication proxy with the fickle & flaky
|
|
remote Machi server.</td></tr>
|
|
<tr><td valign="top"><a href="#terminate-2">terminate/2</a></td><td></td></tr>
|
|
<tr><td valign="top"><a href="#write_chunk-4">write_chunk/4</a></td><td>Write a chunk of data (that has already been
|
|
allocated/sequenced by an earlier append_chunk_extra() call) to
|
|
<code>File</code> at <code>Offset</code>.</td></tr>
|
|
<tr><td valign="top"><a href="#write_chunk-5">write_chunk/5</a></td><td>Read a chunk of data of size <code>Size</code> from <code>File</code> at <code>Offset</code>.</td></tr>
|
|
</table>
|
|
|
|
<h2><a name="functions">Function Details</a></h2>
|
|
|
|
<h3 class="function"><a name="append_chunk-3">append_chunk/3</a></h3>
|
|
<div class="spec">
|
|
<p><tt>append_chunk(PidSpec, Prefix, Chunk) -> any()</tt></p>
|
|
</div><p>Append a chunk (binary- or iolist-style) of data to a file
|
|
with <code>Prefix</code>.</p>
|
|
|
|
<h3 class="function"><a name="append_chunk-4">append_chunk/4</a></h3>
|
|
<div class="spec">
|
|
<p><tt>append_chunk(PidSpec, Prefix, Chunk, Timeout) -> any()</tt></p>
|
|
</div><p>Append a chunk (binary- or iolist-style) of data to a file
|
|
with <code>Prefix</code>.</p>
|
|
|
|
<h3 class="function"><a name="append_chunk_extra-4">append_chunk_extra/4</a></h3>
|
|
<div class="spec">
|
|
<p><tt>append_chunk_extra(PidSpec, Prefix, Chunk, ChunkExtra) -> any()</tt></p>
|
|
</div><p>Append a chunk (binary- or iolist-style) of data to a file
|
|
with <code>Prefix</code>.</p>
|
|
|
|
<h3 class="function"><a name="append_chunk_extra-5">append_chunk_extra/5</a></h3>
|
|
<div class="spec">
|
|
<p><tt>append_chunk_extra(PidSpec, Prefix, Chunk, ChunkExtra, Timeout) -> any()</tt></p>
|
|
</div><p>Append a chunk (binary- or iolist-style) of data to a file
|
|
with <code>Prefix</code>.</p>
|
|
|
|
<h3 class="function"><a name="checksum_list-2">checksum_list/2</a></h3>
|
|
<div class="spec">
|
|
<p><tt>checksum_list(PidSpec, File) -> any()</tt></p>
|
|
</div><p>Fetch the list of chunk checksums for <code>File</code>.</p>
|
|
|
|
<h3 class="function"><a name="checksum_list-3">checksum_list/3</a></h3>
|
|
<div class="spec">
|
|
<p><tt>checksum_list(PidSpec, File, Timeout) -> any()</tt></p>
|
|
</div><p>Fetch the list of chunk checksums for <code>File</code>.</p>
|
|
|
|
<h3 class="function"><a name="code_change-3">code_change/3</a></h3>
|
|
<div class="spec">
|
|
<p><tt>code_change(OldVsn, S, Extra) -> any()</tt></p>
|
|
</div>
|
|
|
|
<h3 class="function"><a name="handle_call-3">handle_call/3</a></h3>
|
|
<div class="spec">
|
|
<p><tt>handle_call(Request, From, S) -> any()</tt></p>
|
|
</div>
|
|
|
|
<h3 class="function"><a name="handle_cast-2">handle_cast/2</a></h3>
|
|
<div class="spec">
|
|
<p><tt>handle_cast(Msg, S) -> any()</tt></p>
|
|
</div>
|
|
|
|
<h3 class="function"><a name="handle_info-2">handle_info/2</a></h3>
|
|
<div class="spec">
|
|
<p><tt>handle_info(Info, S) -> any()</tt></p>
|
|
</div>
|
|
|
|
<h3 class="function"><a name="init-1">init/1</a></h3>
|
|
<div class="spec">
|
|
<p><tt>init(X1) -> any()</tt></p>
|
|
</div>
|
|
|
|
<h3 class="function"><a name="list_files-1">list_files/1</a></h3>
|
|
<div class="spec">
|
|
<p><tt>list_files(PidSpec) -> any()</tt></p>
|
|
</div><p>Fetch the list of all files on the remote FLU.</p>
|
|
|
|
<h3 class="function"><a name="list_files-2">list_files/2</a></h3>
|
|
<div class="spec">
|
|
<p><tt>list_files(PidSpec, Timeout) -> any()</tt></p>
|
|
</div><p>Fetch the list of all files on the remote FLU.</p>
|
|
|
|
<h3 class="function"><a name="quit-1">quit/1</a></h3>
|
|
<div class="spec">
|
|
<p><tt>quit(PidSpec) -> any()</tt></p>
|
|
</div><p>Quit & close the connection to remote FLU and stop our
|
|
proxy process.</p>
|
|
|
|
<h3 class="function"><a name="read_chunk-4">read_chunk/4</a></h3>
|
|
<div class="spec">
|
|
<p><tt>read_chunk(PidSpec, File, Offset, Size) -> any()</tt></p>
|
|
</div><p>Read a chunk of data of size <code>Size</code> from <code>File</code> at <code>Offset</code>.</p>
|
|
|
|
<h3 class="function"><a name="read_chunk-5">read_chunk/5</a></h3>
|
|
<div class="spec">
|
|
<p><tt>read_chunk(PidSpec, File, Offset, Size, Timeout) -> any()</tt></p>
|
|
</div><p>Read a chunk of data of size <code>Size</code> from <code>File</code> at <code>Offset</code>.</p>
|
|
|
|
<h3 class="function"><a name="start_link-1">start_link/1</a></h3>
|
|
<div class="spec">
|
|
<p><tt>start_link(P_srvr_list) -> any()</tt></p>
|
|
</div><p>Start a local, long-lived process that will be our steady
|
|
& reliable communication proxy with the fickle & flaky
|
|
remote Machi server.</p>
|
|
|
|
<h3 class="function"><a name="terminate-2">terminate/2</a></h3>
|
|
<div class="spec">
|
|
<p><tt>terminate(Reason, State) -> any()</tt></p>
|
|
</div>
|
|
|
|
<h3 class="function"><a name="write_chunk-4">write_chunk/4</a></h3>
|
|
<div class="spec">
|
|
<p><tt>write_chunk(PidSpec, File, Offset, Chunk) -> any()</tt></p>
|
|
</div><p>Write a chunk of data (that has already been
|
|
allocated/sequenced by an earlier append_chunk_extra() call) to
|
|
<code>File</code> at <code>Offset</code>.</p>
|
|
|
|
<h3 class="function"><a name="write_chunk-5">write_chunk/5</a></h3>
|
|
<div class="spec">
|
|
<p><tt>write_chunk(PidSpec, File, Offset, Chunk, Timeout) -> any()</tt></p>
|
|
</div><p>Read a chunk of data of size <code>Size</code> from <code>File</code> at <code>Offset</code>.</p>
|
|
<hr>
|
|
|
|
<div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
|
|
<p><i>Generated by EDoc, Jul 1 2015, 18:32:19.</i></p>
|
|
</body>
|
|
</html>
|