Overviewerlang logo

Module machi_flu1_client

Erlang API for the Machi FLU TCP protocol version 1.

Description

Erlang API for the Machi FLU TCP protocol version 1.

This client API handles low-level PDU serialization/deserialization and low-level TCP session management, e.g. open, receive, write, close. The API for higher-level session management and Machi state management can be found in machi_proxy_flu1_client and machi_cr_client.

For the moment, this module implements a Protocol Buffers-based protocol as the sole supported access method to the server, sequencer, and projection store. Conceptually, those three services are independent and ought to have their own protocols. As a practical matter, there is no need for wire protocol compatibility. Furthermore, from the perspective of failure detection, it is very convenient that all three FLU-related services are accessed using the same single TCP port.

TODO This EDoc was written first, and the EDoc and also -type and -spec definitions for machi_proxy_flu1_client and machi_cr_client must be improved.

Data Types

port_wrap()

port_wrap() = {w, atom(), term()}

Function Index

append_chunk/4Append a chunk (binary- or iolist-style) of data to a file with Prefix.
append_chunk/5Append a chunk (binary- or iolist-style) of data to a file with Prefix.
append_chunk_extra/5Append a chunk (binary- or iolist-style) of data to a file with Prefix and also request an additional Extra bytes.
append_chunk_extra/6Append a chunk (binary- or iolist-style) of data to a file with Prefix and also request an additional Extra bytes.
checksum_list/3Fetch the list of chunk checksums for File.
checksum_list/4Fetch the list of chunk checksums for File.
connect/1
connected_p/1
delete_migration/3Restricted API: Delete a file after it has been successfully migrated.
delete_migration/4Restricted API: Delete a file after it has been successfully migrated.
disconnect/1
echo/2Echo -- test protocol round-trip.
echo/3Get all epoch numbers from the FLU's projection store.
get_all_projections/2Get all projections from the FLU's projection store.
get_all_projections/3Get all projections from the FLU's projection store.
get_latest_epochid/2Get the latest epoch number + checksum from the FLU's projection store.
get_latest_epochid/3Get the latest epoch number + checksum from the FLU's projection store.
list_all_projections/2Get all epoch numbers from the FLU's projection store.
list_all_projections/3Get all epoch numbers from the FLU's projection store.
list_files/2Fetch the list of all files on the remote FLU.
list_files/3Fetch the list of all files on the remote FLU.
quit/1Quit & close the connection to remote FLU.
read_chunk/5Read a chunk of data of size Size from File at Offset.
read_chunk/6Read a chunk of data of size Size from File at Offset.
read_latest_projection/2Get the latest projection from the FLU's projection store for ProjType
read_latest_projection/3Get the latest projection from the FLU's projection store for ProjType
read_projection/3Read a projection Proj of type ProjType.
read_projection/4Read a projection Proj of type ProjType.
trunc_hack/3Restricted API: Truncate a file after it has been successfully erasure coded.
trunc_hack/4Restricted API: Truncate a file after it has been successfully erasure coded.
wedge_status/1Fetch the wedge status from the remote FLU.
wedge_status/2Fetch the wedge status from the remote FLU.
write_chunk/5Restricted API: Write a chunk of already-sequenced data to File at Offset.
write_chunk/6Restricted API: Write a chunk of already-sequenced data to File at Offset.
write_projection/3Write a projection Proj of type ProjType.
write_projection/4Write a projection Proj of type ProjType.

Function Details

append_chunk/4

append_chunk(Sock::port_wrap(), EpochID::machi_dt:epoch_id(), Prefix::machi_dt:file_prefix(), Chunk::machi_dt:chunk()) -> {ok, machi_dt:chunk_pos()} | {error, machi_dt:error_general()} | {error, term()}

Append a chunk (binary- or iolist-style) of data to a file with Prefix.

append_chunk/5

append_chunk(Host::machi_dt:inet_host(), TcpPort::machi_dt:inet_port(), EpochID::machi_dt:epoch_id(), Prefix::machi_dt:file_prefix(), Chunk::machi_dt:chunk()) -> {ok, machi_dt:chunk_pos()} | {error, machi_dt:error_general()} | {error, term()}

Append a chunk (binary- or iolist-style) of data to a file with Prefix.

append_chunk_extra/5

append_chunk_extra(Sock::port_wrap(), EpochID::machi_dt:epoch_id(), Prefix::machi_dt:file_prefix(), Chunk::machi_dt:chunk(), ChunkExtra::machi_dt:chunk_size()) -> {ok, machi_dt:chunk_pos()} | {error, machi_dt:error_general()} | {error, term()}

Append a chunk (binary- or iolist-style) of data to a file with Prefix and also request an additional Extra bytes.

For example, if the Chunk size is 1 KByte and Extra is 4K Bytes, then the file offsets that follow Chunk's position for the following 4K will be reserved by the file sequencer for later write(s) by the write_chunk() API.

append_chunk_extra/6

append_chunk_extra(Host::machi_dt:inet_host(), TcpPort::machi_dt:inet_port(), EpochID::machi_dt:epoch_id(), Prefix::machi_dt:file_prefix(), Chunk::machi_dt:chunk(), ChunkExtra::machi_dt:chunk_size()) -> {ok, machi_dt:chunk_pos()} | {error, machi_dt:error_general()} | {error, term()}

Append a chunk (binary- or iolist-style) of data to a file with Prefix and also request an additional Extra bytes.

For example, if the Chunk size is 1 KByte and Extra is 4K Bytes, then the file offsets that follow Chunk's position for the following 4K will be reserved by the file sequencer for later write(s) by the write_chunk() API.

checksum_list/3

checksum_list(Sock::port_wrap(), EpochID::machi_dt:epoch_id(), File::machi_dt:file_name()) -> {ok, binary()} | {error, machi_dt:error_general() | no_such_file | partial_read} | {error, term()}

Fetch the list of chunk checksums for File.

checksum_list/4

checksum_list(Host::machi_dt:inet_host(), TcpPort::machi_dt:inet_port(), EpochID::machi_dt:epoch_id(), File::machi_dt:file_name()) -> {ok, binary()} | {error, machi_dt:error_general() | no_such_file} | {error, term()}

Fetch the list of chunk checksums for File.

Why return a simple binary() type rather than [machi_dt:chunk_summary()]? The two reasons are:
  1. Server overhead: the CPU required to chop up the implementation- specific store into zillions of very small terms is very high.
  2. Protocol encoding and decoding overhead: the cost is non-zero, and the sum of cost of encoding and decoding a zillion small terms is substantial.

For both reasons, the server's protocol response is absurdly simple and very fast: send back a binary() blob to the client. Then it is the client's responsibility to spend the CPU time to parse the blob.

Details of the encoding used inside the binary() blog can be found in the EDoc comments for machi_flu1:decode_csum_file_entry/1.

connect/1

connect(P_srvr) -> any()

connected_p/1

connected_p(X1) -> any()

delete_migration/3

delete_migration(Sock::port_wrap(), EpochID::machi_dt:epoch_id(), File::machi_dt:file_name()) -> ok | {error, machi_dt:error_general() | no_such_file} | {error, term()}

Restricted API: Delete a file after it has been successfully migrated.

delete_migration/4

delete_migration(Host::machi_dt:inet_host(), TcpPort::machi_dt:inet_port(), EpochID::machi_dt:epoch_id(), File::machi_dt:file_name()) -> ok | {error, machi_dt:error_general() | no_such_file} | {error, term()}

Restricted API: Delete a file after it has been successfully migrated.

disconnect/1

disconnect(WS) -> any()

echo/2

echo(Sock::port_wrap(), String::string()) -> string() | {error, term()}

Echo -- test protocol round-trip.

echo/3

echo(Host::machi_dt:inet_host(), TcpPort::machi_dt:inet_port(), String::string()) -> string() | {error, term()}

Get all epoch numbers from the FLU's projection store.

get_all_projections/2

get_all_projections(Sock::port_wrap(), ProjType::machi_dt:projection_type()) -> {ok, [machi_dt:projection()]} | {error, term()}

Get all projections from the FLU's projection store.

get_all_projections/3

get_all_projections(Host::machi_dt:inet_host(), TcpPort::machi_dt:inet_port(), ProjType::machi_dt:projection_type()) -> {ok, [machi_dt:projection()]} | {error, term()}

Get all projections from the FLU's projection store.

get_latest_epochid/2

get_latest_epochid(Sock::port_wrap(), ProjType::machi_dt:projection_type()) -> {ok, machi_dt:epoch_id()} | {error, term()}

Get the latest epoch number + checksum from the FLU's projection store.

get_latest_epochid/3

get_latest_epochid(Host::machi_dt:inet_host(), TcpPort::machi_dt:inet_port(), ProjType::machi_dt:projection_type()) -> {ok, machi_dt:epoch_id()} | {error, term()}

Get the latest epoch number + checksum from the FLU's projection store.

list_all_projections/2

list_all_projections(Sock::port_wrap(), ProjType::machi_dt:projection_type()) -> {ok, [non_neg_integer()]} | {error, term()}

Get all epoch numbers from the FLU's projection store.

list_all_projections/3

list_all_projections(Host::machi_dt:inet_host(), TcpPort::machi_dt:inet_port(), ProjType::machi_dt:projection_type()) -> {ok, [non_neg_integer()]} | {error, term()}

Get all epoch numbers from the FLU's projection store.

list_files/2

list_files(Sock::port_wrap(), EpochID::machi_dt:epoch_id()) -> {ok, [machi_dt:file_info()]} | {error, term()}

Fetch the list of all files on the remote FLU.

list_files/3

list_files(Host::machi_dt:inet_host(), TcpPort::machi_dt:inet_port(), EpochID::machi_dt:epoch_id()) -> {ok, [machi_dt:file_info()]} | {error, term()}

Fetch the list of all files on the remote FLU.

quit/1

quit(Sock::port_wrap()) -> ok

Quit & close the connection to remote FLU.

read_chunk/5

read_chunk(Sock::port_wrap(), EpochID::machi_dt:epoch_id(), File::machi_dt:file_name(), Offset::machi_dt:file_offset(), Size::machi_dt:chunk_size()) -> {ok, machi_dt:chunk_s()} | {error, machi_dt:error_general() | not_written | partial_read} | {error, term()}

Read a chunk of data of size Size from File at Offset.

read_chunk/6

read_chunk(Host::machi_dt:inet_host(), TcpPort::machi_dt:inet_port(), EpochID::machi_dt:epoch_id(), File::machi_dt:file_name(), Offset::machi_dt:file_offset(), Size::machi_dt:chunk_size()) -> {ok, machi_dt:chunk_s()} | {error, machi_dt:error_general() | not_written | partial_read} | {error, term()}

Read a chunk of data of size Size from File at Offset.

read_latest_projection/2

read_latest_projection(Sock::port_wrap(), ProjType::machi_dt:projection_type()) -> {ok, machi_dt:projection()} | {error, not_written} | {error, term()}

Get the latest projection from the FLU's projection store for ProjType

read_latest_projection/3

read_latest_projection(Host::machi_dt:inet_host(), TcpPort::machi_dt:inet_port(), ProjType::machi_dt:projection_type()) -> {ok, machi_dt:projection()} | {error, not_written} | {error, term()}

Get the latest projection from the FLU's projection store for ProjType

read_projection/3

read_projection(Sock::port_wrap(), ProjType::machi_dt:projection_type(), Epoch::machi_dt:epoch_num()) -> {ok, machi_dt:projection()} | {error, not_written} | {error, term()}

Read a projection Proj of type ProjType.

read_projection/4

read_projection(Host::machi_dt:inet_host(), TcpPort::machi_dt:inet_port(), ProjType::machi_dt:projection_type(), Epoch::machi_dt:epoch_num()) -> {ok, machi_dt:projection()} | {error, not_written} | {error, term()}

Read a projection Proj of type ProjType.

trunc_hack/3

trunc_hack(Sock::port_wrap(), EpochID::machi_dt:epoch_id(), File::machi_dt:file_name()) -> ok | {error, machi_dt:error_general() | no_such_file} | {error, term()}

Restricted API: Truncate a file after it has been successfully erasure coded.

trunc_hack/4

trunc_hack(Host::machi_dt:inet_host(), TcpPort::machi_dt:inet_port(), EpochID::machi_dt:epoch_id(), File::machi_dt:file_name()) -> ok | {error, machi_dt:error_general() | no_such_file} | {error, term()}

Restricted API: Truncate a file after it has been successfully erasure coded.

wedge_status/1

wedge_status(Sock::port_wrap()) -> {ok, {boolean(), machi_dt:epoch_id()}} | {error, term()}

Fetch the wedge status from the remote FLU.

wedge_status/2

wedge_status(Host::machi_dt:inet_host(), TcpPort::machi_dt:inet_port()) -> {ok, {boolean(), machi_dt:epoch_id()}} | {error, term()}

Fetch the wedge status from the remote FLU.

write_chunk/5

write_chunk(Sock::port_wrap(), EpochID::machi_dt:epoch_id(), File::machi_dt:file_name(), Offset::machi_dt:file_offset(), Chunk::machi_dt:chunk()) -> ok | {error, machi_dt:error_general()} | {error, term()}

Restricted API: Write a chunk of already-sequenced data to File at Offset.

write_chunk/6

write_chunk(Host::machi_dt:inet_host(), TcpPort::machi_dt:inet_port(), EpochID::machi_dt:epoch_id(), File::machi_dt:file_name(), Offset::machi_dt:file_offset(), Chunk::machi_dt:chunk()) -> ok | {error, machi_dt:error_general()} | {error, term()}

Restricted API: Write a chunk of already-sequenced data to File at Offset.

write_projection/3

write_projection(Sock::port_wrap(), ProjType::machi_dt:projection_type(), Proj::machi_dt:projection()) -> ok | {error, written} | {error, term()}

Write a projection Proj of type ProjType.

write_projection/4

write_projection(Host::machi_dt:inet_host(), TcpPort::machi_dt:inet_port(), ProjType::machi_dt:projection_type(), Proj::machi_dt:projection()) -> ok | {error, written} | {error, term()}

Write a projection Proj of type ProjType.

Overviewerlang logo

Generated by EDoc, Jul 1 2015, 18:32:19.