| Overview |  |
inet_host() = inet:ip_address() | inet:hostname()
- - -inet_port() = inet:port_number()
- - -| verify_file_checksums_local/3 | |
| verify_file_checksums_local/4 | |
| verify_file_checksums_remote/3 | |
| verify_file_checksums_remote/4 |
verify_file_checksums_local(Sock1::port(), EpochID::machi_flu1_client:epoch_id(), Path::binary() | list()) -> {ok, [tuple()]} | {error, term()}
verify_file_checksums_local(Host::inet_host(), TcpPort::inet_port(), EpochID::machi_flu1_client:epoch_id(), Path::binary() | list()) -> {ok, [tuple()]} | {error, term()}
verify_file_checksums_remote(Sock1::port(), EpochID::machi_flu1_client:epoch_id(), File::binary() | list()) -> {ok, [tuple()]} | {error, term()}
verify_file_checksums_remote(Host::inet_host(), TcpPort::inet_port(), EpochID::machi_flu1_client:epoch_id(), File::binary() | list()) -> {ok, [tuple()]} | {error, term()}
Generated by EDoc, Apr 8 2015, 17:31:11.
- - diff --git a/edoc/machi_app.html b/edoc/machi_app.html deleted file mode 100644 index 8bba636..0000000 --- a/edoc/machi_app.html +++ /dev/null @@ -1,39 +0,0 @@ - - - - -Behaviours: application.
- -| start/2 | |
| stop/1 |
start(StartType, StartArgs) -> any()
-stop(State) -> any()
-Generated by EDoc, Apr 8 2015, 17:31:11.
- - diff --git a/edoc/machi_chain_manager1.html b/edoc/machi_chain_manager1.html deleted file mode 100644 index e2d0bec..0000000 --- a/edoc/machi_chain_manager1.html +++ /dev/null @@ -1,155 +0,0 @@ - - - - -Behaviours: gen_server.
- -The Machi chain manager, Guardian of all things related to -Chain Replication state, status, and data replica safety.
- -The Chain Manager is responsible for managing the state of Machi's -"Chain Replication" state. This role is roughly analogous to the -"Riak Core" application inside of Riak, which takes care of -coordinating replica placement and replica repair.
- -For each primitive data server in the cluster, a Machi FLU, there -is a Chain Manager process that manages its FLU's role within the -Machi cluster's Chain Replication scheme. Each Chain Manager -process executes locally and independently to manage the -distributed state of a single Machi Chain Replication chain.
- - Machi's Chain Manager process performs similar tasks as Riak Core's - claimant. However, Machi has several active Chain Manager - processes, one per FLU server, instead of a single active process - like Core's claimant. Each Chain Manager process acts - independently; each is constrained so that it will reach consensus - via independent computation & action. -code_change(OldVsn, S, Extra) -> any()
-get_all_hosed(P) -> any()
-handle_call(Call, From, Ch_mgr) -> any()
-handle_cast(Cast, Ch_mgr) -> any()
-handle_info(Msg, S) -> any()
-init(X1) -> any()
-make_projection_summary(Projection_v1) -> any()
-ping(Pid) -> any()
-projection_transitions_are_sane(Ps, RelativeToServer) -> any()
-start_link(MyName, All_list, MyFLUPid) -> any()
-start_link(MyName, All_list, MyFLUPid, MgrOpts) -> any()
-stop(Pid) -> any()
-terminate(Reason, S) -> any()
-test_calc_projection(Pid, KeepRunenvP) -> any()
-test_calc_proposed_projection(Pid) -> any()
-test_react_to_env(Pid) -> any()
-test_read_latest_public_projection(Pid, ReadRepairP) -> any()
-test_write_proposed_projection(Pid) -> any()
-Generated by EDoc, Apr 8 2015, 17:31:11.
- - diff --git a/edoc/machi_chash.html b/edoc/machi_chash.html deleted file mode 100644 index 7f2f293..0000000 --- a/edoc/machi_chash.html +++ /dev/null @@ -1,171 +0,0 @@ - - - - -Consistent hashing library. Also known as "random slicing".
- - This code was originally from the Hibari DB source code at - https://github.com/hibari -float_map() = [{owner_name(), float()}]
-A float map subdivides the unit interval, starting at 0.0, to - partitions that are assigned to various owners. The sum of all - floats must be exactly 1.0 (or close enough for floating point - purposes).
- -abstract datatype: float_tree()
-We can't use gb_trees:tree() because 'nil' (the empty tree) is - never valid in our case. But teaching Dialyzer that is difficult.
- -nextfloat_list() = [{float(), brick()}]
-A nextfloat_list - differs from a float_map in two respects: 1) nextfloat_list contains - tuples with the brick name in 2nd position, 2) the float() at each - position I_n > I_m, for all n, m such that n > m. - For example, a nextfloat_list of the float_map example above, - [{0.25, {br1, nd1}}, {0.75, {br2, nd1}}, {1.0, {br3, nd1}].
- -owner_int_range() = {owner_name(), non_neg_integer(), non_neg_integer()}
-Used when "prettying" a float map.
- -owner_name() = term()
-Owner for a range on the unit interval. We are agnostic about its - type.
- -owner_weight() = {owner_name(), weight()}
- - -owner_weight_list() = [owner_weight()]
-A owner_weight_list is a definition of brick assignments over the - unit interval [0.0, 1.0]. The sum of all floats must be 1.0. For - example, [{{br1,nd1}, 0.25}, {{br2,nd1}, 0.5}, {{br3,nd1}, 0.25}].
- -weight() = non_neg_integer()
-For this library, a weight is an integer which specifies the - capacity of a "owner" relative to other owners. For example, if - owner A with a weight of 10, and if owner B has a weight of 20, - then B will be assigned twice as much of the unit interval as A.
- -| hash_binary_via_float_map/2 | Query a float map with a binary (inefficient). |
| hash_binary_via_float_tree/2 | Query a float tree with a binary. |
| make_demo_map1/0 | Create a sample float map. |
| make_demo_map2/0 | Create a sample float map. |
| make_float_map/1 | Create a float map, based on a basic owner weight list. |
| make_float_map/2 | Create a float map, based on an older float map and a new weight -list. |
| make_tree/1 | Create a float tree, which is the rapid lookup data structure - for consistent hash queries. |
| pretty_with_integers/2 | Make a pretty/human-friendly version of a float map that describes
- integer ranges between 1 and Scale. |
| pretty_with_integers/3 | Make a pretty/human-friendly version of a float map (based
- upon a float map created from OldWeights and NewWeights) that
- describes integer ranges between 1 and Scale. |
| query_tree/2 | Low-level function for querying a float tree: the (floating - point) point within the unit interval. |
| sum_map_weights/1 | Create a human-friendly summary of a float map. |
| zzz_usage_details/0 | Various usage examples, see source code below this function - for full details. |
hash_binary_via_float_map(Key::binary(), Map::float_map()) -> {float(), owner_name()}
Query a float map with a binary (inefficient).
- -hash_binary_via_float_tree(Key::binary(), Tree::float_tree()) -> {float(), owner_name()}
Query a float tree with a binary.
- -make_demo_map1() -> float_map()
Create a sample float map.
- -make_demo_map2() -> float_map()
Create a sample float map.
- -make_float_map(NewOwnerWeights::owner_weight_list()) -> float_map()
Create a float map, based on a basic owner weight list.
- -make_float_map(OldFloatMap::float_map(), NewOwnerWeights::owner_weight_list()) -> float_map()
Create a float map, based on an older float map and a new weight -list.
- - The weights in the new weight list may be different than (or the - same as) whatever weights were used to make the older float map. - -make_tree(Map::float_map()) -> float_tree()
Create a float tree, which is the rapid lookup data structure - for consistent hash queries.
- -pretty_with_integers(Map::float_map(), Scale::integer()) -> [owner_int_range()]
Make a pretty/human-friendly version of a float map that describes
- integer ranges between 1 and Scale.
pretty_with_integers(OldWeights::owner_weight_list(), NewWeights::owner_weight_list(), Scale::integer()) -> [owner_int_range()]
Make a pretty/human-friendly version of a float map (based
- upon a float map created from OldWeights and NewWeights) that
- describes integer ranges between 1 and Scale.
query_tree(Val::float(), Tree::float_tree()) -> {float(), owner_name()}
Low-level function for querying a float tree: the (floating - point) point within the unit interval.
- -sum_map_weights(Map::float_map()) -> {{per_owner, float_map()}, {weight_sum, float()}}
Create a human-friendly summary of a float map.
- - The two parts of the summary are: a per-owner total of the unit - interval range(s) owned by each owner, and a total sum of all - per-owner ranges (which should be 1.0 but is not enforced). - -zzz_usage_details() -> any()
-Various usage examples, see source code below this function - for full details.
-Generated by EDoc, Apr 8 2015, 17:31:11.
- - diff --git a/edoc/machi_flu1.html b/edoc/machi_flu1.html deleted file mode 100644 index 3ce6902..0000000 --- a/edoc/machi_flu1.html +++ /dev/null @@ -1,63 +0,0 @@ - - - - -The Machi FLU file server + file location sequencer.
- -This module implements only the Machi FLU file server and its -implicit sequencer. -Please see the EDoc "Overview" for details about the FLU as a -primitive file server process vs. the larger Machi design of a FLU -as a sequencer + file server + chain manager group of processes.
- -For the moment, this module also implements a rudimentary TCP-based -protocol as the sole supported access method to the server, -sequencer, and projection store. Conceptually, those three -services are independent and ought to have their own protocols. As -a practical matter, there is no need for wire protocol -compatibility. Furthermore, from the perspective of failure -detection, it is very convenient that all three FLU-related -services are accessed using the same single TCP port.
- -The FLU is named after the CORFU server "FLU" or "FLash Unit" server.
- - TODO There is one major missing feature in this FLU implementation: - there is no "write-once" enforcement for any position in a Machi - file. At the moment, we rely on correct behavior of the client - & the sequencer to avoid overwriting data. In the Real World, - however, all Machi file data is supposed to be exactly write-once - to avoid problems with bugs, wire protocol corruption, malicious - clients, etc. -| start_link/1 | |
| stop/1 |
start_link(Rest) -> any()
-stop(Pid) -> any()
-Generated by EDoc, Apr 8 2015, 17:31:11.
- - diff --git a/edoc/machi_flu1_client.html b/edoc/machi_flu1_client.html deleted file mode 100644 index f1f6203..0000000 --- a/edoc/machi_flu1_client.html +++ /dev/null @@ -1,278 +0,0 @@ - - - - -chunk() = binary() | iolist()
-client can use either
- -chunk_csum() = {file_offset(), chunk_size(), binary()}
- - -chunk_pos() = {file_offset(), chunk_size(), file_name_s()}
- - -chunk_s() = binary()
-server always uses binary()
- -chunk_size() = non_neg_integer()
- - -epoch_csum() = binary()
- - -epoch_id() = {epoch_num(), epoch_csum()}
- - -epoch_num() = -1 | non_neg_integer()
- - -file_info() = {file_size(), file_name_s()}
- - -file_name() = binary() | list()
- - -file_name_s() = binary()
-server reply
- -file_offset() = non_neg_integer()
- - -file_prefix() = binary() | list()
- - -file_size() = non_neg_integer()
- - -inet_host() = inet:ip_address() | inet:hostname()
- - -inet_port() = inet:port_number()
- - -projection() = #projection_v1{}
- - -projection_type() = public | private
- - -| append_chunk/4 | Append a chunk (binary- or iolist-style) of data to a file
- with Prefix. |
| append_chunk/5 | Append a chunk (binary- or iolist-style) of data to a file
- with Prefix. |
| checksum_list/3 | Fetch the list of chunk checksums for File. |
| checksum_list/4 | Fetch the list of chunk checksums for File. |
| delete_migration/3 | Restricted API: Delete a file after it has been successfully - migrated. |
| delete_migration/4 | Restricted API: Delete a file after it has been successfully - migrated. |
| get_all_projections/2 | Get all projections from the FLU's projection store. |
| get_all_projections/3 | Get all projections from the FLU's projection store. |
| get_latest_epoch/2 | Get the latest epoch number + checksum from the FLU's projection store. |
| get_latest_epoch/3 | Get the latest epoch number + checksum from the FLU's projection store. |
| list_all_projections/2 | Get all epoch numbers from the FLU's projection store. |
| list_all_projections/3 | Get all epoch numbers from the FLU's projection store. |
| list_files/2 | Fetch the list of all files on the remote FLU. |
| list_files/3 | Fetch the list of all files on the remote FLU. |
| quit/1 | Quit & close the connection to remote FLU. |
| read_chunk/5 | Read a chunk of data of size Size from File at Offset. |
| read_chunk/6 | Read a chunk of data of size Size from File at Offset. |
| read_latest_projection/2 | Get the latest projection from the FLU's projection store for ProjType |
| read_latest_projection/3 | Get the latest projection from the FLU's projection store for ProjType |
| read_projection/3 | Read a projection Proj of type ProjType. |
| read_projection/4 | Read a projection Proj of type ProjType. |
| trunc_hack/3 | Restricted API: Truncate a file after it has been successfully - erasure coded. |
| trunc_hack/4 | Restricted API: Truncate a file after it has been successfully - erasure coded. |
| write_chunk/5 | Restricted API: Write a chunk of already-sequenced data to
- File at Offset. |
| write_chunk/6 | Restricted API: Write a chunk of already-sequenced data to
- File at Offset. |
| write_projection/3 | Write a projection Proj of type ProjType. |
| write_projection/4 | Write a projection Proj of type ProjType. |
append_chunk(Sock::port(), EpochID::epoch_id(), Prefix::file_prefix(), Chunk::chunk()) -> {ok, chunk_pos()} | {error, term()}
Append a chunk (binary- or iolist-style) of data to a file
- with Prefix.
append_chunk(Host::inet_host(), TcpPort::inet_port(), EpochID::epoch_id(), Prefix::file_prefix(), Chunk::chunk()) -> {ok, chunk_pos()} | {error, term()}
Append a chunk (binary- or iolist-style) of data to a file
- with Prefix.
checksum_list(Sock::port(), EpochID::epoch_id(), File::file_name()) -> {ok, [chunk_csum()]} | {error, term()}
Fetch the list of chunk checksums for File.
checksum_list(Host::inet_host(), TcpPort::inet_port(), EpochID::epoch_id(), File::file_name()) -> {ok, [chunk_csum()]} | {error, term()}
Fetch the list of chunk checksums for File.
delete_migration(Sock::port(), EpochID::epoch_id(), File::file_name()) -> ok | {error, term()}
Restricted API: Delete a file after it has been successfully - migrated.
- -delete_migration(Host::inet_host(), TcpPort::inet_port(), EpochID::epoch_id(), File::file_name()) -> ok | {error, term()}
Restricted API: Delete a file after it has been successfully - migrated.
- -get_all_projections(Sock::port(), ProjType::projection_type()) -> {ok, [projection()]} | {error, term()}
Get all projections from the FLU's projection store.
- -get_all_projections(Host::inet_host(), TcpPort::inet_port(), ProjType::projection_type()) -> {ok, [projection()]} | {error, term()}
Get all projections from the FLU's projection store.
- -get_latest_epoch(Sock::port(), ProjType::projection_type()) -> {ok, epoch_id()} | {error, term()}
Get the latest epoch number + checksum from the FLU's projection store.
- -get_latest_epoch(Host::inet_host(), TcpPort::inet_port(), ProjType::projection_type()) -> {ok, epoch_id()} | {error, term()}
Get the latest epoch number + checksum from the FLU's projection store.
- -list_all_projections(Sock::port(), ProjType::projection_type()) -> {ok, [non_neg_integer()]} | {error, term()}
Get all epoch numbers from the FLU's projection store.
- -list_all_projections(Host::inet_host(), TcpPort::inet_port(), ProjType::projection_type()) -> {ok, [non_neg_integer()]} | {error, term()}
Get all epoch numbers from the FLU's projection store.
- -list_files(Sock::port(), EpochID::epoch_id()) -> {ok, [file_info()]} | {error, term()}
Fetch the list of all files on the remote FLU.
- -list_files(Host::inet_host(), TcpPort::inet_port(), EpochID::epoch_id()) -> {ok, [file_info()]} | {error, term()}
Fetch the list of all files on the remote FLU.
- -quit(Sock::port()) -> ok
Quit & close the connection to remote FLU.
- -read_chunk(Sock::port(), EpochID::epoch_id(), File::file_name(), Offset::file_offset(), Size::chunk_size()) -> {ok, chunk_s()} | {error, term()}
Read a chunk of data of size Size from File at Offset.
read_chunk(Host::inet_host(), TcpPort::inet_port(), EpochID::epoch_id(), File::file_name(), Offset::file_offset(), Size::chunk_size()) -> {ok, chunk_s()} | {error, term()}
Read a chunk of data of size Size from File at Offset.
read_latest_projection(Sock::port(), ProjType::projection_type()) -> {ok, projection()} | {error, not_written} | {error, term()}
Get the latest projection from the FLU's projection store for ProjType
read_latest_projection(Host::inet_host(), TcpPort::inet_port(), ProjType::projection_type()) -> {ok, projection()} | {error, not_written} | {error, term()}
Get the latest projection from the FLU's projection store for ProjType
read_projection(Sock::port(), ProjType::projection_type(), Epoch::epoch_num()) -> {ok, projection()} | {error, written} | {error, term()}
Read a projection Proj of type ProjType.
read_projection(Host::inet_host(), TcpPort::inet_port(), ProjType::projection_type(), Epoch::epoch_num()) -> {ok, projection()} | {error, written} | {error, term()}
Read a projection Proj of type ProjType.
trunc_hack(Sock::port(), EpochID::epoch_id(), File::file_name()) -> ok | {error, term()}
Restricted API: Truncate a file after it has been successfully - erasure coded.
- -trunc_hack(Host::inet_host(), TcpPort::inet_port(), EpochID::epoch_id(), File::file_name()) -> ok | {error, term()}
Restricted API: Truncate a file after it has been successfully - erasure coded.
- -write_chunk(Sock::port(), EpochID::epoch_id(), File::file_name(), Offset::file_offset(), Chunk::chunk()) -> ok | {error, term()}
Restricted API: Write a chunk of already-sequenced data to
- File at Offset.
write_chunk(Host::inet_host(), TcpPort::inet_port(), EpochID::epoch_id(), File::file_name(), Offset::file_offset(), Chunk::chunk()) -> ok | {error, term()}
Restricted API: Write a chunk of already-sequenced data to
- File at Offset.
write_projection(Sock::port(), ProjType::projection_type(), Proj::projection()) -> ok | {error, written} | {error, term()}
Write a projection Proj of type ProjType.
write_projection(Host::inet_host(), TcpPort::inet_port(), ProjType::projection_type(), Proj::projection()) -> ok | {error, written} | {error, term()}
Write a projection Proj of type ProjType.
Generated by EDoc, Apr 8 2015, 17:31:11.
- - diff --git a/edoc/machi_flu_sup.html b/edoc/machi_flu_sup.html deleted file mode 100644 index 46a3345..0000000 --- a/edoc/machi_flu_sup.html +++ /dev/null @@ -1,41 +0,0 @@ - - - - -Behaviours: supervisor.
- -| init/1 | |
| start_link/0 |
init(X1) -> any()
-start_link() -> any()
-Generated by EDoc, Apr 8 2015, 17:31:11.
- - diff --git a/edoc/machi_projection.html b/edoc/machi_projection.html deleted file mode 100644 index 59f9229..0000000 --- a/edoc/machi_projection.html +++ /dev/null @@ -1,70 +0,0 @@ - - - - -| compare/2 | Compare two projection records for equality (assuming that the - checksum element has been correctly calculated). |
| make_projection_summary/1 | Create a proplist-style summary of a projection record. |
| new/6 | Create a new projection record. |
| new/7 | Create a new projection record. |
| new/8 | Create a new projection record. |
| update_projection_checksum/1 | Update the checksum element of a projection record. |
| update_projection_dbg2/2 | Update the dbg2 element of a projection record. |
compare(Projection_v1::#projection_v1{}, Projection_v1::#projection_v1{}) -> integer()
Compare two projection records for equality (assuming that the - checksum element has been correctly calculated).
- -make_projection_summary(Projection_v1) -> any()
-Create a proplist-style summary of a projection record.
- -new(MyName, All_list, UPI_list, Down_list, Repairing_list, Ps) -> any()
-Create a new projection record.
- -new(EpochNum, MyName, All_list, Down_list, UPI_list, Repairing_list, Dbg) -> any()
-Create a new projection record.
- -new(EpochNum, MyName, All_list0, Down_list, UPI_list, Repairing_list, Dbg, Dbg2) -> any()
-Create a new projection record.
- -update_projection_checksum(P) -> any()
-Update the checksum element of a projection record.
- -update_projection_dbg2(P, Dbg2) -> any()
-Update the dbg2 element of a projection record.
Generated by EDoc, Apr 8 2015, 17:31:11.
- - diff --git a/edoc/machi_projection_store.html b/edoc/machi_projection_store.html deleted file mode 100644 index ed25ba3..0000000 --- a/edoc/machi_projection_store.html +++ /dev/null @@ -1,163 +0,0 @@ - - - - -The Machi write-once projection store service.
- -This API is gen_server-style message passing, intended for use -within a single Erlang node to glue together the projection store -server with the node-local process that implements Machi's TCP -client access protocol (on the "server side" of the TCP connection).
- -All Machi client access to the projection store SHOULD NOT use this -module's API.
- - The projection store is implemented by an Erlang/OTPgen_server
- process that is associated with each FLU. Conceptually, the
- projection store is an array of write-once registers. For each
- projection store register, the key is a 2-tuple of an epoch number
- (non_neg_integer() type) and a projection type (public or
- private type); the value is a projection data structure
- (projection_v1() type).
-| code_change/3 | |
| get_all_projections/2 | Fetch all projection records of type ProjType. |
| get_all_projections/3 | Fetch all projection records of type ProjType. |
| get_latest_epoch/2 | Fetch the latest epoch number + checksum for type ProjType. |
| get_latest_epoch/3 | Fetch the latest epoch number + checksum for type ProjType. |
| handle_call/3 | |
| handle_cast/2 | |
| handle_info/2 | |
| init/1 | |
| list_all_projections/2 | Fetch all projection epoch numbers of type ProjType. |
| list_all_projections/3 | Fetch all projection epoch numbers of type ProjType. |
| read/3 | Fetch the projection record type ProjType for epoch number Epoch . |
| read/4 | Fetch the projection record type ProjType for epoch number Epoch . |
| read_latest_projection/2 | Fetch the latest projection record for type ProjType. |
| read_latest_projection/3 | Fetch the latest projection record for type ProjType. |
| start_link/3 | Start a new projection store server. |
| terminate/2 | |
| write/3 | Write the projection record type ProjType for epoch number Epoch . |
| write/4 | Write the projection record type ProjType for epoch number Epoch . |
code_change(OldVsn, S, Extra) -> any()
-get_all_projections(PidSpec, ProjType) -> any()
-Fetch all projection records of type ProjType.
get_all_projections(PidSpec, ProjType, Timeout) -> any()
-Fetch all projection records of type ProjType.
get_latest_epoch(PidSpec, ProjType) -> any()
-Fetch the latest epoch number + checksum for type ProjType.
get_latest_epoch(PidSpec, ProjType, Timeout) -> any()
-Fetch the latest epoch number + checksum for type ProjType.
- projection.
handle_call(Request, From, S) -> any()
-handle_cast(Msg, S) -> any()
-handle_info(Info, S) -> any()
-init(X1) -> any()
-list_all_projections(PidSpec, ProjType) -> any()
-Fetch all projection epoch numbers of type ProjType.
list_all_projections(PidSpec, ProjType, Timeout) -> any()
-Fetch all projection epoch numbers of type ProjType.
read(PidSpec, ProjType, Epoch) -> any()
-Fetch the projection record type ProjType for epoch number Epoch .
read(PidSpec, ProjType, Epoch, Timeout) -> any()
-Fetch the projection record type ProjType for epoch number Epoch .
read_latest_projection(PidSpec, ProjType) -> any()
-Fetch the latest projection record for type ProjType.
read_latest_projection(PidSpec, ProjType, Timeout) -> any()
-Fetch the latest projection record for type ProjType.
start_link(RegName, DataDir, NotifyWedgeStateChanges) -> any()
-Start a new projection store server.
- - TheDataDir argument should be the same directory as specified
- for use by our companion FLU data server -- all file system paths
- used by this server are intended to be stored underneath a common
- file system parent directory as the FLU data server & sequencer
- servers.
-
-terminate(Reason, S) -> any()
-write(PidSpec, ProjType, Proj) -> any()
-Write the projection record type ProjType for epoch number Epoch .
write(PidSpec, ProjType, Proj, Timeout) -> any()
-Write the projection record type ProjType for epoch number Epoch .
Generated by EDoc, Apr 8 2015, 17:31:11.
- - diff --git a/edoc/machi_proxy_flu1_client.html b/edoc/machi_proxy_flu1_client.html deleted file mode 100644 index 4cf46f4..0000000 --- a/edoc/machi_proxy_flu1_client.html +++ /dev/null @@ -1,222 +0,0 @@ - - - - -Behaviours: gen_server.
- -Erlang API for the Machi FLU TCP protocol version 1, with a -proxy-process style API for hiding messy details such as TCP -connection/disconnection with the remote Machi server.
- -Machi is intentionally avoiding using distributed Erlang for - Machi's communication. This design decision makes Erlang-side code - more difficult & complex, but it's the price to pay for some -language independence. Later in Machi's life cycle, we need to -(re-)implement some components in a non-Erlang/BEAM-based language.
- - This module implements a "man in the middle" proxy between the - Erlang client and Machi server (which is on the "far side" of a TCP - connection to somewhere). This proxy process will always execute - on the same Erlang node as the Erlang client that uses it. The - proxy is intended to be a stable, long-lived process that survives - TCP communication problems with the remote server. -| append_chunk/4 | Append a chunk (binary- or iolist-style) of data to a file
- with Prefix. |
| append_chunk/5 | Append a chunk (binary- or iolist-style) of data to a file
- with Prefix. |
| checksum_list/3 | Fetch the list of chunk checksums for File. |
| checksum_list/4 | Fetch the list of chunk checksums for File. |
| code_change/3 | |
| get_all_projections/2 | Get all projections from the FLU's projection store. |
| get_all_projections/3 | Get all projections from the FLU's projection store. |
| get_latest_epoch/2 | Get the latest epoch number + checksum from the FLU's projection store. |
| get_latest_epoch/3 | Get the latest epoch number + checksum from the FLU's projection store. |
| handle_call/3 | |
| handle_cast/2 | |
| handle_info/2 | |
| init/1 | |
| list_all_projections/2 | Get all epoch numbers from the FLU's projection store. |
| list_all_projections/3 | Get all epoch numbers from the FLU's projection store. |
| list_files/2 | Fetch the list of all files on the remote FLU. |
| list_files/3 | Fetch the list of all files on the remote FLU. |
| quit/1 | Quit & close the connection to remote FLU and stop our - proxy process. |
| read_chunk/5 | Read a chunk of data of size Size from File at Offset. |
| read_chunk/6 | Read a chunk of data of size Size from File at Offset. |
| read_latest_projection/2 | Get the latest projection from the FLU's projection store for ProjType |
| read_latest_projection/3 | Get the latest projection from the FLU's projection store for ProjType |
| read_projection/3 | Read a projection Proj of type ProjType. |
| read_projection/4 | Read a projection Proj of type ProjType. |
| start_link/1 | Start a local, long-lived process that will be our steady - & reliable communication proxy with the fickle & flaky - remote Machi server. |
| terminate/2 | |
| write_projection/3 | Write a projection Proj of type ProjType. |
| write_projection/4 | Write a projection Proj of type ProjType. |
append_chunk(PidSpec, EpochID, Prefix, Chunk) -> any()
-Append a chunk (binary- or iolist-style) of data to a file
- with Prefix.
append_chunk(PidSpec, EpochID, Prefix, Chunk, Timeout) -> any()
-Append a chunk (binary- or iolist-style) of data to a file
- with Prefix.
checksum_list(PidSpec, EpochID, File) -> any()
-Fetch the list of chunk checksums for File.
checksum_list(PidSpec, EpochID, File, Timeout) -> any()
-Fetch the list of chunk checksums for File.
code_change(OldVsn, S, Extra) -> any()
-get_all_projections(PidSpec, ProjType) -> any()
-Get all projections from the FLU's projection store.
- -get_all_projections(PidSpec, ProjType, Timeout) -> any()
-Get all projections from the FLU's projection store.
- -get_latest_epoch(PidSpec, ProjType) -> any()
-Get the latest epoch number + checksum from the FLU's projection store.
- -get_latest_epoch(PidSpec, ProjType, Timeout) -> any()
-Get the latest epoch number + checksum from the FLU's projection store.
- -handle_call(Request, From, S) -> any()
-handle_cast(Msg, S) -> any()
-handle_info(Info, S) -> any()
-init(X1) -> any()
-list_all_projections(PidSpec, ProjType) -> any()
-Get all epoch numbers from the FLU's projection store.
- -list_all_projections(PidSpec, ProjType, Timeout) -> any()
-Get all epoch numbers from the FLU's projection store.
- -list_files(PidSpec, EpochID) -> any()
-Fetch the list of all files on the remote FLU.
- -list_files(PidSpec, EpochID, Timeout) -> any()
-Fetch the list of all files on the remote FLU.
- -quit(PidSpec) -> any()
-Quit & close the connection to remote FLU and stop our - proxy process.
- -read_chunk(PidSpec, EpochID, File, Offset, Size) -> any()
-Read a chunk of data of size Size from File at Offset.
read_chunk(PidSpec, EpochID, File, Offset, Size, Timeout) -> any()
-Read a chunk of data of size Size from File at Offset.
read_latest_projection(PidSpec, ProjType) -> any()
-Get the latest projection from the FLU's projection store for ProjType
read_latest_projection(PidSpec, ProjType, Timeout) -> any()
-Get the latest projection from the FLU's projection store for ProjType
read_projection(PidSpec, ProjType, Epoch) -> any()
-Read a projection Proj of type ProjType.
read_projection(PidSpec, ProjType, Epoch, Timeout) -> any()
-Read a projection Proj of type ProjType.
start_link(P_srvr) -> any()
-Start a local, long-lived process that will be our steady - & reliable communication proxy with the fickle & flaky - remote Machi server.
- -terminate(Reason, S) -> any()
-write_projection(PidSpec, ProjType, Proj) -> any()
-Write a projection Proj of type ProjType.
write_projection(PidSpec, ProjType, Proj, Timeout) -> any()
-Write a projection Proj of type ProjType.
Generated by EDoc, Apr 8 2015, 17:31:11.
- - diff --git a/edoc/machi_sequencer.html b/edoc/machi_sequencer.html deleted file mode 100644 index 0956867..0000000 --- a/edoc/machi_sequencer.html +++ /dev/null @@ -1,23 +0,0 @@ - - - - -Generated by EDoc, Apr 8 2015, 17:31:11.
- - diff --git a/edoc/machi_sup.html b/edoc/machi_sup.html deleted file mode 100644 index aaa08c6..0000000 --- a/edoc/machi_sup.html +++ /dev/null @@ -1,39 +0,0 @@ - - - - -Behaviours: supervisor.
- -| init/1 | |
| start_link/0 |
init(X1) -> any()
-start_link() -> any()
-Generated by EDoc, Apr 8 2015, 17:31:11.
- - diff --git a/edoc/machi_util.html b/edoc/machi_util.html deleted file mode 100644 index f85a247..0000000 --- a/edoc/machi_util.html +++ /dev/null @@ -1,150 +0,0 @@ - - - - -| bin_to_hexstr/1 | Convert a binary() to a hexadecimal string. |
| checksum_chunk/1 | Calculate a checksum for a chunk of file data. |
| connect/2 | Create a TCP connection to a remote Machi server. |
| connect/3 | Create a TCP connection to a remote Machi server. |
| hexstr_to_bin/1 | Convert a hexadecimal string to a binary(). |
| hexstr_to_int/1 | Convert a hexadecimal string to an integer. |
| increment_max_filenum/2 | Increase the file size of a config file, which is used as the - basis for a minimum sequence number. |
| info_msg/2 | Log an 'info' level message. |
| int_to_hexbin/2 | Convert an integer into a hexadecimal string (in binary()
- form) whose length is based on I_size. |
| int_to_hexstr/2 | Convert an integer into a hexadecimal string whose length is
- based on I_size. |
| make_binary/1 | Convert a compatible Erlang data type into a binary() equivalent. |
| make_checksum_filename/2 | Calculate a checksum file path, by common convention. |
| make_data_filename/2 | Calculate a file data file path, by common convention. |
| make_projection_filename/2 | Calculate a projection store file path, by common convention. |
| make_regname/1 | Create a registered name atom for FLU sequencer internal - rendezvous/message passing use. |
| make_string/1 | Convert a compatible Erlang data type into a string() equivalent. |
| read_max_filenum/2 | Read the file size of a config file, which is used as the - basis for a minimum sequence number. |
| verb/1 | Log a verbose message. |
| verb/2 | Log a verbose message. |
bin_to_hexstr(X1) -> any()
-Convert a binary() to a hexadecimal string.
checksum_chunk(Chunk) -> any()
-Calculate a checksum for a chunk of file data.
- -connect(Host::inet:ip_address() | inet:hostname(), Port::inet:port_number()) -> port()
Create a TCP connection to a remote Machi server.
- -connect(Host::inet:ip_address() | inet:hostname(), Port::inet:port_number(), Timeout::timeout()) -> port()
Create a TCP connection to a remote Machi server.
- -hexstr_to_bin(S) -> any()
-Convert a hexadecimal string to a binary().
hexstr_to_int(X) -> any()
-Convert a hexadecimal string to an integer.
- -increment_max_filenum(DataDir, Prefix) -> any()
-Increase the file size of a config file, which is used as the - basis for a minimum sequence number.
- -info_msg(Fmt, Args) -> any()
-Log an 'info' level message.
- -int_to_hexbin(I, I_size) -> any()
-Convert an integer into a hexadecimal string (in binary()
- form) whose length is based on I_size.
int_to_hexstr(I, I_size) -> any()
-Convert an integer into a hexadecimal string whose length is
- based on I_size.
make_binary(X) -> any()
-Convert a compatible Erlang data type into a binary() equivalent.
make_checksum_filename(DataDir, FileName) -> any()
-Calculate a checksum file path, by common convention.
- -make_data_filename(DataDir, File) -> any()
-Calculate a file data file path, by common convention.
- -make_projection_filename(DataDir, File) -> any()
-Calculate a projection store file path, by common convention.
- -make_regname(Prefix) -> any()
-Create a registered name atom for FLU sequencer internal - rendezvous/message passing use.
- -make_string(X) -> any()
-Convert a compatible Erlang data type into a string() equivalent.
read_max_filenum(DataDir, Prefix) -> any()
-Read the file size of a config file, which is used as the - basis for a minimum sequence number.
- -verb(Fmt) -> any()
-Log a verbose message.
- -verb(Fmt, Args) -> any()
-Log a verbose message.
-Generated by EDoc, Apr 8 2015, 17:31:11.
- - diff --git a/edoc/modules-frame.html b/edoc/modules-frame.html deleted file mode 100644 index fbe4efc..0000000 --- a/edoc/modules-frame.html +++ /dev/null @@ -1,24 +0,0 @@ - - - -This EDoc-style documentation will concern itself only with Erlang -function APIs and function & data types. Higher-level design and -commentary will remain outside of the Erlang EDoc system; please see -the "Pointers to Other Machi Documentation" section below for more -details.
- -Readers should beware that this documentation may be out-of-sync with
-the source code. When in doubt, use the make edoc command to
-regenerate all HTML pages.
It is the developer's responsibility to re-generate the documentation -periodically and commit it to the Git repo.
- -The Chain Manager is responsible for managing the state of Machi's -"Chain Replication" state. This role is roughly analogous to the -"Riak Core" application inside of Riak, which takes care of -coordinating replica placement and replica repair.
- -For each primitive data server in the cluster, a Machi FLU, there is a -Chain Manager process that manages its FLU's role within the Machi -cluster's Chain Replication scheme. Each Chain Manager process -executes locally and independently to manage the distributed state of -a single Machi Chain Replication chain.
- -To contrast with Riak Core ... Riak Core's claimant process is - solely responsible for managing certain critical aspects of - Riak Core distributed state. Machi's Chain Manager process - performs similar tasks as Riak Core's claimant. However, Machi - has several active Chain Manager processes, one per FLU server, - instead of a single active process like Core's claimant. Each - Chain Manager process acts independently; each is constrained - so that it will reach consensus via independent computation - & action.
- - Full discussion of this distributed consensus is outside the - scope of this document; see the "Pointers to Other Machi - Documentation" section below for more information. -The FLU is the basic storage server for Machi.
- -append_chunk() protocol message has
- an implicit "sequencer" operation applied by the "head" of the
- Machi Chain Replication chain. If a client wishes to write
- data that has already been assigned a sequencer position, then
- the write_chunk() API function is used.
- For each FLU, there are three independent tasks that are implemented -using three different Erlang processes:
- -machi_flu.erl.
- machi_projection_store.erl.
- machi_chain_manager1.erl.
- From the perspective of failure detection, it is very convenient that -all three FLU-related services (file server, sequencer server, and -projection server) are accessed using the same single TCP port.
- -The projection is a data structure that specifies the current state -of the Machi cluster: all FLUs, which FLUS are considered -up/running or down/crashed/stopped, which FLUs are actively -participants in the Chain Replication protocol, and which FLUs are -under "repair" (i.e., having their data resyncronized when -newly-added to a cluster or when restarting after a crash).
- -The projection store is a storage service that is implemented by an
-Erlang/OTP gen_server process that is associated with each
-FLU. Conceptually, the projection store is an array of
-write-once registers. For each projection store register, the
-key is a 2-tuple of an epoch number (non_neg_integer() type)
-and a projection type (public or private type); the value is
-a projection data structure (projection_v1() type).
Machi is intentionally avoiding using distributed Erlang for Machi's -communication. This design decision makes Erlang-side code more -difficult & complex but allows us the freedom of implementing -parts of Machi in other languages without major -protocol&API&glue code changes later in the product's -lifetime.
- -There are two layers of interface for Machi clients.
- -machi_flu1_client module implements an API that uses a
- TCP socket directly.
- machi_proxy_flu1_client module implements an API that
- uses a local, long-lived gen_server process as a proxy for
- the remote, perhaps disconnected-or-crashed Machi FLU server.
- The types for both modules ought to be the same. However, due to -rapid code churn, some differences might exist. Any major difference -is (almost by definition) a bug: please open a GitHub issue to request -a correction.
- -Any use of the string "TODO" in upper/lower/mixed case, anywhere in -the code, is a reminder signal of unfinished work.
- -../doc/ directory,
-master branch
- of the Machi source repository and therefore may be
- out-of-sync with non-master branch code.
-Generated by EDoc, Apr 8 2015, 17:31:11.
- - diff --git a/edoc/overview.edoc b/edoc/overview.edoc deleted file mode 100644 index 04cf4cc..0000000 --- a/edoc/overview.edoc +++ /dev/null @@ -1,14 +0,0 @@ - -@title Machi: a small village of replicated files - -@doc - -Documentation for Machi is an ongoing challenge. Much of the -high-level design & commentary are outside of the Erlang EDoc system - -Zoom2 zoom zoom zoom boom boom boom boom - -Rumba tango Rumba tango Rumba tango Rumba tango Rumba tango Rumba -tango Rumba tango Rumba tango Rumba tango Rumba tango Rumba tango -Rumba tango Rumba tango Rumba tango Rumba tango Rumba tango Rumba -tango Rumba tango Rumba tango Rumba tango Rumba tango diff --git a/edoc/packages-frame.html b/edoc/packages-frame.html deleted file mode 100644 index 189d01c..0000000 --- a/edoc/packages-frame.html +++ /dev/null @@ -1,11 +0,0 @@ - - - -