298 changed files with 5726 additions and 77955 deletions
--- a/.gitignore
+++ b/.gitignore
@ -1,33 +0,0 @@
 prototype/chain-manager/patch.*
 .eqc-info
 .eunit
 deps
 dev
 erl_crash.dump
 eqc
 .concrete/DEV_MODE
 .rebar
 edoc
 # Dialyzer stuff
 .dialyzer-last-run.txt
 .ebin.native
 .local_dialyzer_plt
 dialyzer_unhandled_warnings
 dialyzer_warnings
 *.plt
 # PB artifacts for Erlang
 include/machi_pb.hrl
 # Release packaging
 rel/machi
 rel/vars/dev*vars.config
 # Misc Scott cruft
 *.patch
 current_counterexample.eqc
 foo*
 RUNLOG*
 typescript*
 *.swp
--- a/.travis.yml
+++ b/.travis.yml
@ -1,7 +0,0 @@
 language: erlang
 notifications:
  email: scott@basho.com
 script: "priv/test-for-gh-pr.sh"
 otp_release:
  - 17.5
 ## No, Dialyzer is too different between 17 & 18:  - 18.1
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@ -1,35 +0,0 @@
 # Contributing to the Machi project
 The most helpful way to contribute is by reporting your experience
 through issues. Issues may not be updated while we review internally,
 but they're still incredibly appreciated.
 Pull requests may take multiple engineers for verification and testing. If
 you're passionate enough to want to learn more on how you can get
 hands on in this process, reach out to
 [Matt Brender](mailto:mbrender@basho.com), your developer advocate.
 Thank you for being part of the community! We love you for it. 
 ## If you have a question or wish to provide design feedback/criticism
 Please
 [open a support ticket at GitHub](https://github.com/basho/machi/issues/new)
 to ask questions and to provide feedback about Machi's
 design/documentation/source code.
 ## General development process
 Machi is still a very young project within Basho, with a small team of
 developers; please bear with us as we grow out of "toddler" stage into
 a more mature open source software project.
 * Fork the Machi source repo and/or the sub-projects that are affected
  by your change.  
 * Create a topic branch for your change and checkout that branch.
     git checkout -b some-topic-branch
 * Make your changes and run the test suite if one is provided.
 * Commit your changes and push them to your fork.
 * Open pull-requests for the appropriate projects.
 * Contributors will review your pull request, suggest changes, and merge it when it’s ready and/or offer feedback.
 * To report a bug or issue, please open a new issue against this repository.
--- a/FAQ.md
+++ b/FAQ.md
@ -1,630 +0,0 @@
 # Frequently Asked Questions (FAQ)
 <!-- Formatting: -->
 <!-- All headings omitted from outline are H1 -->
 <!-- All other headings must be on a single line! -->
 <!-- Run: ./priv/make-faq.pl ./FAQ.md > ./tmpfoo; mv ./tmpfoo ./FAQ.md -->
 # Outline
 <!-- OUTLINE -->
 + [1 Questions about Machi in general](#n1)
    + [1.1 What is Machi?](#n1.1)
    + [1.2 What is a Machi chain?](#n1.2)
    + [1.3 What is a Machi cluster?](#n1.3)
    + [1.4 What is Machi like when operating in "eventually consistent" mode?](#n1.4)
    + [1.5 What is Machi like when operating in "strongly consistent" mode?](#n1.5)
    + [1.6 What does Machi's API look like?](#n1.6)
    + [1.7 What licensing terms are used by Machi?](#n1.7)
    + [1.8 Where can I find the Machi source code and documentation?  Can I contribute?](#n1.8)
    + [1.9 What is Machi's expected release schedule, packaging, and operating system/OS distribution support?](#n1.9)
 + [2 Questions about Machi relative to {{something else}}](#n2)
    + [2.1 How is Machi better than Hadoop?](#n2.1)
    + [2.2 How does Machi differ from HadoopFS/HDFS?](#n2.2)
    + [2.3 How does Machi differ from Kafka?](#n2.3)
    + [2.4 How does Machi differ from Bookkeeper?](#n2.4)
    + [2.5 How does Machi differ from CORFU and Tango?](#n2.5)
 + [3 Machi's specifics](#n3)
    + [3.1 What technique is used to replicate Machi's files?  Can other techniques be used?](#n3.1)
    + [3.2 Does Machi have a reliance on a coordination service such as ZooKeeper or etcd?](#n3.2)
    + [3.3 Are there any presentations available about Humming Consensus](#n3.3)
    + [3.4 Is it true that there's an allegory written to describe Humming Consensus?](#n3.4)
    + [3.5 How is Machi tested?](#n3.5)
    + [3.6 Does Machi require shared disk storage? e.g. iSCSI, NBD (Network Block Device), Fibre Channel disks](#n3.6)
    + [3.7 Does Machi require or assume that servers with large numbers of disks must use RAID-0/1/5/6/10/50/60 to create a single block device?](#n3.7)
    + [3.8 What language(s) is Machi written in?](#n3.8)
    + [3.9 Can Machi run on Windows?  Can Machi run on 32-bit platforms?](#n3.9)
    + [3.10 Does Machi use the Erlang/OTP network distribution system (aka "disterl")?](#n3.10)
    + [3.11 Can I use HTTP to write/read stuff into/from Machi?](#n3.11)
 <!-- ENDOUTLINE -->
 <a name="n1">
 ## 1.  Questions about Machi in general
 <a name="n1.1">
 ### 1.1.  What is Machi?
 Very briefly, Machi is a very simple append-only blob/file store.
 Machi is
 "dumber" than many other file stores (i.e., lacking many features
 found in other file stores) such as HadoopFS or a simple NFS or CIFS file
 server.
 However, Machi is a distributed blob/file store, which makes it different
 (and, in some ways, more complicated) than a simple NFS or CIFS file
 server.
 All Machi data is protected by SHA-1 checksums.  By default, these
 checksums are calculated by the client to provide strong end-to-end
 protection against data corruption.  (If the client does not provide a
 checksum, one will be generated by the first Machi server to handle
 the write request.)  Internally, Machi uses these checksums for local
 data integrity checks and for server-to-server file synchronization
 and corrupt data repair.
 As a distributed system, Machi can be configured to operate with
 either eventually consistent mode or strongly consistent mode.  In
 strongly consistent mode, Machi can provide write-once file store
 service in the same style as CORFU.  Machi can be an easy to use tool
 for building fully ordered, log-based distributed systems and
 distributed data structures.
 In eventually consistent mode, Machi can remain available for writes
 during arbitrary network partitions.  When a network partition is
 fixed, Machi can safely merge all file data together without data
 loss.  Similar to the operation of
 Basho's
 [Riak key-value store, Riak KV](http://basho.com/products/riak-kv/),
 Machi can provide file writes during arbitrary network partitions and
 later merge all results together safely when the cluster recovers.
 For a much longer answer, please see the
 [Machi high level design doc](https://github.com/basho/machi/tree/master/doc/high-level-machi.pdf).
 <a name="n1.2">
 ### 1.2.  What is a Machi chain?
 A Machi chain is a small number of machines that maintain a common set
 of replicated files.  A typical chain is of length 2 or 3.  For
 critical data that must be available despite several simultaneous
 server failures, a chain length of 6 or 7 might be used.
 <a name="n1.3">
 ### 1.3.  What is a Machi cluster?
 A Machi cluster is a collection of Machi chains that
 partitions/shards/distributes files (based on file name) across the
 collection of chains.  Machi uses the "random slicing" algorithm (a
 variation of consistent hashing) to define the mapping of file name to
 chain name.
 The cluster management service will be fully decentralized
 and run as a separate software service installed on each Machi
 cluster.  This manager will appear to the local Machi server as simply
 another Machi file client.  The cluster managers will take
 care of file migration as the cluster grows and shrinks in capacity
 and in response to day-to-day changes in workload.
 Though the cluster manager has not yet been implemented,
 its design is fully decentralized and capable of operating despite
 multiple partial failure of its member chains.  We expect this
 design to scale easily to at least one thousand servers.
 Please see the
 [Machi source repository's 'doc' directory for more details](https://github.com/basho/machi/tree/master/doc/).
 <a name="n1.4">
 ### 1.4.  What is Machi like when operating in "eventually consistent" mode?
 Machi's operating mode dictates how a Machi cluster will react to
 network partitions.  A network partition may be caused by:
 * A network failure
 * A server failure
 * An extreme server software "hang" or "pause", e.g. caused by OS
  scheduling problems such as a failing/stuttering disk device.
 The consistency semantics of file operations while in eventual
 consistency mode during and after network partitions are:
 * File write operations are permitted by any client on the "same side"
  of the network partition.
 * File read operations are successful for any file contents where the
  client & server are on the "same side" of the network partition.
    * File read operations will probably fail for any file contents where the
      client & server are on "different sides" of the network partition.
 * After the network partition(s) is resolved, files are merged
  together from "all sides" of the partition(s).
    * Unique files are copied in their entirety.
    * Byte ranges within the same file are merged.  This is possible
      due to Machi's restrictions on file naming and file offset
      assignment.  Both file names and file offsets are always chosen
      by Machi servers according to rules which guarantee safe
      mergeability.  Server-assigned names are a characteristic of a
      "blob store".
 <a name="n1.5">
 ### 1.5.  What is Machi like when operating in "strongly consistent" mode?
 The consistency semantics of file operations while in strongly
 consistency mode during and after network partitions are:
 * File write operations are permitted by any client on the "same side"
  of the network partition if and only if a quorum majority of Machi servers
  are also accessible within that partition.
    * In other words, file write service is unavailable in any
      partition where only a minority of Machi servers are accessible.
 * File read operations are successful for any file contents where the
  client & server are on the "same side" of the network partition.
 * After the network partition(s) is resolved, files are repaired from
  the surviving quorum majority members to out-of-sync minority
  members.
 Machi's design can provide the illusion of quorum minority write
 availability if the cluster is configured to operate with "witness
 servers".  (This feaure partially implemented, as of December 2015.)
 See Section 11 of
 [Machi chain manager high level design doc](https://github.com/basho/machi/tree/master/doc/high-level-chain-mgr.pdf)
 for more details.
 <a name="n1.6">
 ### 1.6.  What does Machi's API look like?
 The Machi API only contains a handful of API operations.  The function
 arguments shown below (in simplifed form) use Erlang-style type annotations.
    append_chunk(Prefix:binary(), Chunk:binary(), CheckSum:binary()).
    append_chunk_extra(Prefix:binary(), Chunk:binary(), CheckSum:binary(), ExtraSpace:non_neg_integer()).
    read_chunk(File:binary(), Offset:non_neg_integer(), Size:non_neg_integer()).
    checksum_list(File:binary()).
    list_files().
 Machi allows the client to choose the prefix of the file name to
 append data to, but the Machi server will always choose the final file
 name and byte offset for each `append_chunk()` operation.  This
 restriction on file naming makes it easy to operate in "eventually
 consistent" mode: files may be written to any server during network
 partitions and can be easily merged together after the partition is
 healed.
 Internally, there is a more complex protocol used by individual
 cluster members to manage file contents and to repair damaged/missing
 files.  See Figure 3 in
 [Machi high level design doc](https://github.com/basho/machi/tree/master/doc/high-level-machi.pdf)
 for more description.
 The definitions of both the "high level" external protocol and "low
 level" internal protocol are in a
 [Protocol Buffers](https://developers.google.com/protocol-buffers/docs/overview)
 definition at [./src/machi.proto](./src/machi.proto).
 <a name="n1.7">
 ### 1.7.  What licensing terms are used by Machi?
 All Machi source code and documentation is licensed by
 [Basho Technologies, Inc.](http://www.basho.com/)
 under the [Apache Public License version 2](https://github.com/basho/machi/tree/master/LICENSE).
 <a name="n1.8">
 ### 1.8.  Where can I find the Machi source code and documentation?  Can I contribute?
 All Machi source code and documentation can be found at GitHub:
 [https://github.com/basho/machi](https://github.com/basho/machi).
 The full URL for this FAQ is [https://github.com/basho/machi/blob/master/FAQ.md](https://github.com/basho/machi/blob/master/FAQ.md).
 There are several "README" files in the source repository.  We hope
 they provide useful guidance for first-time readers.
 If you're interested in contributing code or documentation or
 ideas for improvement, please see our contributing & collaboration
 guidelines at
 [https://github.com/basho/machi/blob/master/CONTRIBUTING.md](https://github.com/basho/machi/blob/master/CONTRIBUTING.md).
 <a name="n1.9">
 ### 1.9.  What is Machi's expected release schedule, packaging, and operating system/OS distribution support?
 Basho expects that Machi's first major product release will take place
 during the 2nd quarter of 2016.
 Basho's official support for operating systems (e.g. Linux, FreeBSD),
 operating system packaging (e.g. CentOS rpm/yum package management,
 Ubuntu debian/apt-get package management), and
 container/virtualization have not yet been chosen.  If you wish to
 provide your opinion, we'd love to hear it.  Please
 [open a support ticket at GitHub](https://github.com/basho/machi/issues/new)
 and let us know.
 <a name="n2">
 ## 2.  Questions about Machi relative to {{something else}}
 <a name="better-than-hadoop">
 <a name="n2.1">
 ### 2.1.  How is Machi better than Hadoop?
 This question is frequently asked by trolls.  If this is a troll
 question, the answer is either, "Nothing is better than Hadoop," or
 else "Everything is better than Hadoop."
 The real answer is that Machi is not a distributed data processing
 framework like Hadoop is.
 See [Hadoop's entry in Wikipedia](https://en.wikipedia.org/wiki/Apache_Hadoop)
 and focus on the description of Hadoop's MapReduce and YARN; Machi
 contains neither.
 <a name="n2.2">
 ### 2.2.  How does Machi differ from HadoopFS/HDFS?
 This is a much better question than the
 [How is Machi better than Hadoop?](#better-than-hadoop)
 question.
 [HadoopFS's entry in Wikipedia](https://en.wikipedia.org/wiki/Apache_Hadoop#HDFS)
 One way to look at Machi is to consider Machi as a distributed file
 store. HadoopFS is also a distributed file store.  Let's compare and
 contrast.
 <table>
 <tr>
 <td> <b>Machi</b>
 <td> <b>HadoopFS (HDFS)</b>
 <tr>
 <td> Not POSIX compliant
 <td> Not POSIX compliant
 <tr>
 <td> Immutable file store with append-only semantics (simplifying
 things a little bit).
 <td> Immutable file store with append-only semantics
 <tr>
 <td> File data may be read concurrently while file is being actively
 appended to.
 <td> File must be closed before a client can read it.
 <tr>
 <td> No concept (yet) of users or authentication (though the initial
 supported release will support basic user + password authentication).
 Machi will probably never natively support directories or ACLs.
 <td> Has concepts of users, directories, and ACLs.
 <tr>
 <td> Machi does not allow clients to name their own files or to specify data
 placement/offset within a file.
 <td> While not POSIX compliant, HDFS allows a fairly flexible API for
 managing file names and file writing position within a file (during a
 file's writable phase). 
 <tr>
 <td> Does not have any file distribution/partitioning/sharding across
 Machi chains: in a single Machi chain, all files are replicated by
 all servers in the chain.  The "random slicing" technique is used
 to distribute/partition/shard files across multiple Machi clusters.
 <td> File distribution/partitioning/sharding is performed
 automatically by the HDFS "name node".
 <tr>
 <td> Machi requires no central "name node" for single chain use or
 for multi-chain cluster use.
 <td> Requires a single "namenode" server to maintain file system contents
 and file content mapping.  (May be deployed with a "secondary
 namenode" to reduce unavailability when the primary namenode fails.)
 <tr>
 <td> Machi uses Chain Replication to manage all file replicas.
 <td> The HDFS name node uses an ad hoc mechanism for replicating file
 contents.  The HDFS file system metadata (file names, file block(s)
 locations, ACLs, etc.) is stored by the name node in the local file
 system and is replicated to any secondary namenode using snapshots.
 <tr>
 <td> Machi replicates files *N* ways where *N* is the length of the
 Chain Replication chain. Typically, *N=2*, but this is configurable.
 <td> HDFS typical replicates file contents *N=3* ways, but this is
 configurable. 
 <tr>
 <td> All Machi file data is protected by SHA-1 checksums generated by
 the client prior to writing by Machi servers.
 <td> Optional file checksum protection may be implemented on the
 server side.
 </table>
 <a name="n2.3">
 ### 2.3.  How does Machi differ from Kafka?
 Machi is rather close to Kafka in spirit, though its implementation is
 quite different.
 <table>
 <tr>
 <td> <b>Machi</b>
 <td> <b>Kafka</b>
 <tr>
 <td> Append-only, strongly consistent file store only
 <td> Append-only, strongly consistent log file store + additional
 services: for example, producer topics & sharding, consumer groups &
 failover, etc.
 <tr>
 <td> Not yet code complete nor "battle tested" in large production
 environments.
 <td> "Battle tested" in large production environments.
 <tr>
 <td> All Machi file data is protected by SHA-1 checksums generated by
 the client prior to writing by Machi servers.
 <td> Each log entry is protected by a 32 bit CRC checksum.
 </table>
 In theory, it should be "quite straightforward" to remove these parts
 of Kafka's code base:
 * local file system I/O for all topic/partition/log files
 * leader/follower file replication, ISR ("In Sync Replica") state
  management, and related log file replication logic
 ... and replace those parts with Machi client API calls.  Those parts
 of Kafka are what Machi has been designed to do from the very
 beginning.
 See also:
 <a href="#corfu-and-tango">How does Machi differ from CORFU and Tango?</a>
 <a name="n2.4">
 ### 2.4.  How does Machi differ from Bookkeeper?
 Sorry, we haven't studied Bookkeeper very deeply or used Bookkeeper
 for any non-trivial project.
 One notable limitation of the Bookkeeper API is that a ledger cannot
 be read by other clients until it has been closed.  Any byte in a
 Machi file that has been written successfully may
 be read immedately by any other Machi client.
 The name "Machi" does not have three consecutive pairs of repeating
 letters.  The name "Bookkeeper" does.
 <a name="corfu-and-tango">
 <a name="n2.5">
 ### 2.5.  How does Machi differ from CORFU and Tango?
 Machi's design borrows very heavily from CORFU.  We acknowledge a deep
 debt to the original Microsoft Research papers that describe CORFU's
 original design and implementation.
 <table>
 <tr>
 <td> <b>Machi</b>
 <td> <b>CORFU</b>
 <tr>
 <td> Writes & reads may be on byte boundaries
 <td> Wries & reads must be on page boundaries, e.g. 4 or 8 KBytes, to
 align with server storage based on flash NVRAM/solid state disk (SSD).
 <tr>
 <td> Provides multiple "logs", where each log has a name and is
 appended to & read from like a file.  A read operation requires a 3-tuple:
 file name, starting byte offset, number of bytes.
 <td> Provides a single "log". A read operation requires only a
 1-tuple: the log page number.  (A protocol option exists to
 request multiple pages in a single read query?)
 <tr>
 <td> Offers service in either strongly consistent mode or eventually
 consistent mode.
 <td> Offers service in strongly consistent mode.
 <tr>
 <td> May be deployed on solid state disk (SSD) or Winchester hard disks.
 <td> Designed for use with solid state disk (SSD) but can also be used
 with Winchester hard disks (with a performance penalty if used as
 suggested by use cases described by the CORFU papers).
 <tr>
 <td> All Machi file data is protected by SHA-1 checksums generated by
 the client prior to writing by Machi servers.
 <td> Depending on server & flash device capabilities, each data page
 may be protected by a checksum (calculated independently by each
 server rather than the client).
 </table>
 See also: the "Recommended reading & related work" and "References"
 sections of the
 [Machi high level design doc](https://github.com/basho/machi/tree/master/doc/high-level-machi.pdf)
 for pointers to the MSR papers related to CORFU.
 Machi does not implement Tango directly.  (Not yet, at least.)
 However, there is a prototype implementation included in the Machi
 source tree.  See
 [the prototype/tango source code directory](https://github.com/basho/machi/tree/master/prototype/tango)
 for details.
 Also, it's worth adding that the original MSR code behind the research
 papers is now available at GitHub:
 [https://github.com/CorfuDB/CorfuDB](https://github.com/CorfuDB/CorfuDB).
 <a name="n3">
 ## 3.  Machi's specifics
 <a name="n3.1">
 ### 3.1.  What technique is used to replicate Machi's files?  Can other techniques be used?
 Machi uses Chain Replication to replicate all file data.  Each byte of
 a file is stored using a "write-once register", which is a mechanism
 to enforce immutability after the byte has been written exactly once.
 In order to ensure availability in the event of *F* failures, Chain
 Replication requires a minimum of *F + 1* servers to be configured.
 Alternative mechanisms could be used to manage file replicas, such as
 Paxos or Raft.  Both Paxos and Raft have some requirements that are
 difficult to adapt to Machi's design goals:
 * Both protocols use quorum majority consensus, which requires a
  minimum of *2F + 1* working servers to tolerate *F* failures.  For
  example, to tolerate 2 server failures, quorum majority protocols
  require a minimum of 5 servers.  To tolerate the same number of
  failures, Chain Replication requires a minimum of only 3 servers.
 * Machi's use of "humming consensus" to manage internal server
  metadata state would also (probably) require conversion to Paxos or
  Raft.  (Or "outsourced" to a service such as ZooKeeper.)
 <a name="n3.2">
 ### 3.2.  Does Machi have a reliance on a coordination service such as ZooKeeper or etcd?
 No.  Machi maintains critical internal cluster information in an
 internal, immutable data service called the "projection store".  The
 contents of the projection store are maintained by a new technique
 called "humming consensus".
 Humming consensus is described in the
 [Machi chain manager high level design doc](https://github.com/basho/machi/tree/master/doc/high-level-chain-mgr.pdf).
 <a name="n3.3">
 ### 3.3.  Are there any presentations available about Humming Consensus
 Scott recently (November 2015) gave a presentation at the
 [RICON 2015 conference](http://ricon.io) about one of the techniques
 used by Machi; "Managing Chain Replication Metadata with
 Humming Consensus" is available online now.
 * [slides (PDF format)](http://ricon.io/speakers/slides/Scott_Fritchie_Ricon_2015.pdf)
 * [video](https://www.youtube.com/watch?v=yR5kHL1bu1Q)
 <a name="n3.4">
 ### 3.4.  Is it true that there's an allegory written to describe Humming Consensus?
 Yes.  In homage to Leslie Lamport's original paper about the Paxos
 protocol, "The Part-time Parliamant", there is an allegorical story
 that describes humming consensus as method to coordinate
 many composers to write a single piece of music.
 The full story, full of wonder and mystery, is called
 ["On “Humming Consensus”, an allegory"](http://www.snookles.com/slf-blog/2015/03/01/on-humming-consensus-an-allegory/).
 There is also a
 [short followup blog posting](http://www.snookles.com/slf-blog/2015/03/20/on-humming-consensus-an-allegory-part-2/).
 <a name="n3.5">
 ### 3.5.  How is Machi tested?
 While not formally proven yet, Machi's implementation of Chain
 Replication and of humming consensus have been extensively tested with
 several techniques:
 * We use an executable model based on the QuickCheck framework for
  property based testing.
 * In addition to QuickCheck alone, we use the PULSE extension to
  QuickCheck is used to test the implementation 
  under extremely skewed & unfair scheduling/timing conditions.
 The model includes simulation of asymmetric network partitions.  For
 example, actor A can send messages to actor B, but B cannot send
 messages to A.  If such a partition happens somewhere in a traditional
 network stack (e.g. a faulty Ethernet cable), any TCP connection
 between A & B will quickly interrupt communication in _both_
 directions.  In the Machi network partition simulator, network
 partitions can be truly one-way only.
 After randomly generating a series of network partitions (which may
 change several times during any single test case) and a random series
 of cluster operations, an event trace of all cluster activity is used
 to verify that no safety-critical rules have been violated.
 All test code is available in the [./test](./test) subdirectory.
 Modules that use QuickCheck will use a file suffix of `_eqc`, for
 example, [./test/machi_ap_repair_eqc.erl](./test/machi_ap_repair_eqc.erl).
 <a name="n3.6">
 ### 3.6.  Does Machi require shared disk storage? e.g. iSCSI, NBD (Network Block Device), Fibre Channel disks
 No, Machi's design assumes that each Machi server is a fully
 independent hardware and assumes only standard local disks (Winchester
 and/or SSD style) with local-only interfaces (e.g. SATA, SCSI, PCI) in
 each machine.
 <a name="n3.7">
 ### 3.7.  Does Machi require or assume that servers with large numbers of disks must use RAID-0/1/5/6/10/50/60 to create a single block device?
 No.  When used with servers with multiple disks, the intent is to
 deploy multiple Machi servers per machine: one Machi server per disk.
 * Pro: disk bandwidth and disk storage capacity can be managed at the
  level of an individual disk.
 * Pro: failure of an individual disk does not risk data loss on other
  disks.
 * Con (or pro, depending on the circumstances): in this configuration,
  Machi would require additional network bandwidth to repair data on a lost
  drive instead of intra-machine disk & bus & memory bandwidth that
  would be required for RAID volume repair
 * Con: replica placement policy, such as "rack awareness", becomes a
  larger problem that must be automated.  For example, a problem of
  placement relative to 12 servers is smaller than a placement problem
  of managing 264 seprate disks (if each of 12 servers has 22 disks).
 <a name="n3.8">
 ### 3.8.  What language(s) is Machi written in?
 So far, Machi is written in Erlang, mostly.  Machi uses at least one
 library, [ELevelDB](https://github.com/basho/eleveldb), that is
 implemented both in C++ and in Erlang, using Erlang NIFs (Native
 Interface Functions) to allow Erlang code to call C++ functions.
 In the event that we encounter a performance problem that cannot be
 solved within the Erlang/OTP runtime environment, all of Machi's
 performance-critical components are small enough to be re-implemented
 in C, Java, or other "gotta go fast fast FAST!!"  programming
 language.  We expect that the Chain Replication manager and other
 critical "control plane" software will remain in Erlang.
 <a name="n3.9">
 ### 3.9.  Can Machi run on Windows?  Can Machi run on 32-bit platforms?
 The ELevelDB NIF does not compile or run correctly on Erlang/OTP
 Windows platforms, nor does it compile correctly on 32-bit platforms.
 Machi should support all 64-bit UNIX-like platforms that are supported
 by Erlang/OTP and ELevelDB.
 <a name="n3.10">
 ### 3.10.  Does Machi use the Erlang/OTP network distribution system (aka "disterl")?
 No, Machi doesn't use Erlang/OTP's built-in distributed message
 passing system.  The code would be *much* simpler if we did use
 "disterl".  However, due to (premature?) worries about performance, we
 wanted to have the option of re-writing some Machi components in C or
 Java or Go or OCaml or COBOL or in-kernel assembly hexadecimal
 bit-twiddling magicSPEED ... without also having to find a replacement
 for disterl.  (Or without having to re-invent disterl's features in
 another language.)
 All wire protocols used by Machi are defined & implemented using
 [Protocol Buffers](https://developers.google.com/protocol-buffers/docs/overview).
 The definition file can be found at [./src/machi.proto](./src/machi.proto).
 <a name="n3.11">
 ### 3.11.  Can I use HTTP to write/read stuff into/from Machi?
 Short answer: No, not yet.
 Longer answer: No, but it was possible as a hack, many months ago, see 
 [primitive/hack'y HTTP interface that is described in this source code commit log](https://github.com/basho/machi/commit/6cebf397232cba8e63c5c9a0a8c02ba391b20fef).
 Please note that commit `6cebf397232cba8e63c5c9a0a8c02ba391b20fef` is
 required to try using this feature: the code has since bit-rotted and
 will not work on today's `master` branch.
 In the long term, we'll probably want the option of an HTTP interface
 that is as well designed and REST'ful as possible.  It's on the
 internal Basho roadmap.  If you'd like to work on a real, not-kludgy
 HTTP interface to Machi,
 [please contact us!](https://github.com/basho/machi/blob/master/CONTRIBUTING.md)
--- a/INSTALLATION.md
+++ b/INSTALLATION.md
@ -1,297 +0,0 @@
 # Installation instructions for Machi
 Machi is still a young enough project that there is no "installation".
 All development is still done using the Erlang/OTP interactive shell
 for experimentation, using `make` to compile, and the shell's
 `l(ModuleName).` command to reload any recompiled modules.
 In the coming months (mid-2015), there are plans to create OS packages
 for common operating systems and OS distributions, such as FreeBSD and
 Linux.  If building RPM, DEB, PKG, or other OS package managers is
 your specialty, we could use your help to speed up the process!  <b>:-)</b>
 ## Development toolchain dependencies
 Machi's dependencies on the developer's toolchain are quite small.
 * Erlang/OTP version 17.0 or later, 32-bit or 64-bit
 * The `make` utility.
    * The GNU version of make is not required.
    * Machi is bundled with a `rebar` package and should be usable on
      any Erlang/OTP 17.x platform.
 Machi does not use any Erlang NIF or port drivers.
 ## Development OS 
 At this time, Machi is 100% Erlang.  Although we have not tested it,
 there should be no good reason why Machi cannot run on Erlang/OTP on
 Windows platforms.  Machi has been developed on OS X and FreeBSD and
 is expected to work on any UNIX-ish platform supported by Erlang/OTP.
 ## Compiling the Machi source
 First, clone the Machi source code, then compile it.  You will
 need Erlang/OTP version 17.x to compile.
    cd /some/nice/dev/place
    git clone https://github.com/basho/machi.git
    cd machi
    make
    make test
 The unit test suite is based on the EUnit framework (bundled with
 Erlang/OTP 17).  The `make test` suite runs on my MacBook in 10
 seconds or less.
 ## Setting up a Machi cluster
 As noted above, everything is done manually at the moment.  Here is a
 rough sketch of day-to-day development workflow.
 ### 1. Run the server
    cd /some/nice/dev/place/machi
    make
    erl -pz ebin deps/*/ebin +A 253 +K true
 This will start an Erlang shell, plus a few extras.
 * Tell the OTP code loader where to find dependent BEAM files.
 * Set a large pool (253) of file I/O worker threads
 * Use a more efficient kernel polling mechanism for network sockets.
    * If your Erlang/OTP package does not support `+K true`, do not
      worry.  It is an optional flag.
 The following commands will start three Machi FLU server processes and
 then tell them to form a single chain.  Internally, each FLU will have
 Erlang registered processes with the names `a`, `b`, and `c`, and
 listen on TCP ports 4444, 4445, and 4446, respectively.  Each will use
 a data directory located in the current directory, e.g. `./data.a`.
 Cut-and-paste the following commands into the CLI at the prompt:
    application:ensure_all_started(machi).        
    machi_flu_psup:start_flu_package(a, 4444, "./data.a", []).
    machi_flu_psup:start_flu_package(b, 4445, "./data.b", []).
    D = orddict:from_list([{a,{p_srvr,a,machi_flu1_client,"localhost",4444,[]}},{b,{p_srvr,b,machi_flu1_client,"localhost",4445,[]}}]).
    machi_chain_manager1:set_chain_members(a_chmgr, D).
    machi_chain_manager1:set_chain_members(b_chmgr, D).
 If you change the TCP ports of any of the processes, you must make the
 same change both in the `machi_flu_psup:start_flu_package()` arguments
 and also in the `D` dictionary.
 The Erlang processes that will be started are arranged in the
 following hierarchy.  See the
 [machi_flu_psup.erl](http://basho.github.io/machi/edoc/machi_flu_psup.html)
 EDoc documentation for a description of each of these processes.
 ![](https://basho.github.io/machi/images/supervisor-2flus.png)
 ### 2. Check the status of the server processes.
 Each Machi FLU is an independent file server.  All replication between
 Machi servers is currently implemented by code on the *client* side.
 (This will change a bit later in 2015.)
 Use the `read_latest_projection` command on the server CLI, e.g.:
    rr("include/machi_projection.hrl").
    machi_projection_store:read_latest_projection(a_pstore, private).
 ... to query the projection store of the local FLU named `a`.
 If you haven't looked at the server-side description of the various
 Machi server-side processes, please take a couple minutes to read
 [machi_flu_psup.erl](http://basho.github.io/machi/edoc/machi_flu_psup.html).
 ### 3. Use the machi_cr_client.erl client
 For development work, I run the client & server on the same Erlang
 VM.  It's just easier that way ... but the Machi client & server use
 TCP to communicate with each other.
 If you are using a separate machine for the client, then compile the
 Machi source on the client machine.  Then run:
    cd /some/nice/dev/place/machi
    make
    erl -pz ebin deps/*/ebin
 (You can add `+K true` if you wish ... but for light development work,
 it doesn't make a big difference.)
 At the CLI, define the dictionary that describes the host & TCP port
 location for each of the Machi servers.  (If you changed the host
 and/or TCP port values when starting the servers, then place make the
 same changes here.
    D = orddict:from_list([{a,{p_srvr,a,machi_flu1_client,"localhost",4444,[]}},{b,{p_srvr,b,machi_flu1_client,"localhost",4445,[]}}]).
 Then start a `machi_cr_client` client process.
    {ok, C1} = machi_cr_client:start_link([P || {_,P} <- orddict:to_list(D)]).
 Please keep in mind that this process is **linked** to your CLI
 process.  If you run a CLI command the throws an exception/exits, then
 this `C1` process will also die!  You can start a new one, using a
 different name, e.g. `C2`.  Or you can start a new one by first
 "forgetting" the CLI's binding for `C1`.
    f(C1).
    {ok, C1} = machi_cr_client:start_link([P || {_,P} <- orddict:to_list(D)]).
 Now, append a small chunk of data to a file with the prefix
 `<<"pre">>`.
    12> {ok, C1} = machi_cr_client:start_link([P || {_,P} <- orddict:to_list(D)]).
    {ok,<0.112.0>}
    13> machi_cr_client:append_chunk(C1, <<"pre">>, <<"Hello, world">>).
    {ok,{1024,12,<<"pre.G6C116EA.3">>}}
 This chunk was written successfully to a file called
 `<<"pre.5BBL16EA.1">>` at byte offset 1024.  Let's fetch it now.  And
 let's see what happens in a couple of error conditions: fetching
 bytes that "straddle" the end of file, bytes that are after the known
 end of file, and bytes from a file that has never been written.
    26> machi_cr_client:read_chunk(C1, <<"pre.G6C116EA.3">>, 1024, 12).
    {ok,<<"Hello, world">>}
    27> machi_cr_client:read_chunk(C1, <<"pre.G6C116EA.3">>, 1024, 777).
    {error,partial_read}
    28> machi_cr_client:read_chunk(C1, <<"pre.G6C116EA.3">>, 889323, 12).
    {error,not_written}
    29> machi_cr_client:read_chunk(C1, <<"no-such-file">>, 1024, 12).
    {error,not_written}
 ### 4. Use the `machi_proxy_flu1_client.erl` client
 The `machi_proxy_flu1_client` module implements a simpler client that
 only uses a single Machi FLU file server.  This client is **not**
 aware of chain replication in any way.
 Let's use this client to verify that the `<<"Hello, world!">> data
 that we wrote in step #3 was truly written to both FLU servers by the
 `machi_cr_client` library.  We start proxy processes for each of the
 FLUs, then we'll query each ... but first we also need to ask (at
 least one of) the servers for the current Machi cluster's Epoch ID.
    {ok, Pa} = machi_proxy_flu1_client:start_link(orddict:fetch(a, D)).
    {ok, Pb} = machi_proxy_flu1_client:start_link(orddict:fetch(b, D)).
    {ok, EpochID0} = machi_proxy_flu1_client:get_epoch_id(Pa).
    machi_proxy_flu1_client:read_chunk(Pa, EpochID0, <<"pre.G6C116EA.3">>, 1024, 12).
    machi_proxy_flu1_client:read_chunk(Pb, EpochID0, <<"pre.G6C116EA.3">>, 1024, 12).
 ### 5. Checking how Chain Replication "read repair" works
 Now, let's cause some trouble: we will write some data only to the
 head of the chain.  By default, all read operations go to the tail of
 the chain.  But, if a value is not written at the tail, then "read
 repair" ought to verify:
 * Perhaps the value truly is not written at any server in the chain.
 * Perhaps the value was partially written, i.e. by a buggy or
  crashed-in-the-middle-of-the-writing-procedure client.
 So, first, let's double-check that the chain is in the order that we
 expect it to be.
    rr("include/machi_projection.hrl").   % In case you didn't do this earlier.
    machi_proxy_flu1_client:read_latest_projection(Pa, private).
 The part of the `#projection_v1` record that we're interested in is
 the `upi`.  This is the list of servers that preserve the Update
 Propagation Invariant property of the Chain Replication algorithm.
 The output should look something like:
    {ok,#projection_v1{
            epoch_number = 1119,
            [...]
            author_server = b,
            all_members = [a,b],
            creation_time = {1432,189599,85392},
            mode = ap_mode,
            upi = [a,b],
            repairing = [],down = [],
            [...]
    }
 So, we see `upi=[a,b]`, which means that FLU `a` is the head of the
 chain and that `b` is the tail.
 Let's append to `a` using the `machi_proxy_flu1_client` to the head
 and then read from both the head and tail.  (If your chain order is
 different, then please exchange `Pa` and `Pb` in all of the commands
 below.)
    16> {ok, {Off1,Size1,File1}} = machi_proxy_flu1_client:append_chunk(Pa, EpochID0, <<"foo">>, <<"Hi, again">>).
    {ok,{1024,9,<<"foo.K63D16M4.1">>}}
    17> machi_proxy_flu1_client:read_chunk(Pa, EpochID0, File1, Off1, Size1).       {ok,<<"Hi, again">>}          
    18> machi_proxy_flu1_client:read_chunk(Pb, EpochID0, File1, Off1, Size1).
    {error,not_written}
 That is correct!  Now, let's read the same file & offset using the
 client that understands chain replication.  Then we will try reading
 directly from FLU `b` again ... we should see something different.
    19> {ok, C2} = machi_cr_client:start_link([P || {_,P} <- orddict:to_list(D)]).
    {ok,<0.113.0>}                
    20> machi_cr_client:read_chunk(C2, File1, Off1, Size1).
    {ok,<<"Hi, again">>}
    21> machi_proxy_flu1_client:read_chunk(Pb, EpochID0, File1, Off1, Size1).
    {ok,<<"Hi, again">>}
 That is correct! The command at prompt #20 automatically performed
 "read repair" on FLU `b`.
 ### 6. Exploring what happens when a server is stopped, data written, and server restarted
 Cut-and-paste the following into your CLI.  We assume that your CLI
 still remembers the value of the `D` dictionary from the previous
 steps above.  We will stop FLU `a`, write one thousand small
 chunks, then restart FLU `a`, then see what happens.
    {ok, C3} = machi_cr_client:start_link([P || {_,P} <- orddict:to_list(D)]).
    machi_flu_psup:stop_flu_package(a).
    [machi_cr_client:append_chunk(C3, <<"foo">>, <<"Lots of stuff">>) || _ <- lists:seq(1,1000)].
    machi_flu_psup:start_flu_package(a, 4444, "./data.a", []).
 About 10 seconds after we restarting `a` with the
 `machi_flu_psup:start_flu_package()` function, this appears on the
 console:
    =INFO REPORT==== 21-May-2015::15:53:46 ===
    Repair start: tail b of [b] -> [a], ap_mode ID {b,{1432,191226,707262}}
    MissingFileSummary [{<<"foo.CYLJ16ZT.1">>,{14024,[a]}}]
    Make repair directives: . done
    Out-of-sync data for FLU a: 0.1 MBytes
    Out-of-sync data for FLU b: 0.0 MBytes
    Execute repair directives: .......... done
    =INFO REPORT==== 21-May-2015::15:53:47 ===
    Repair success: tail b of [b] finished ap_mode repair ID {b,{1432,191226,707262}}: ok
    Stats [{t_in_files,0},{t_in_chunks,1000},{t_in_bytes,13000},{t_out_files,0},{t_out_chunks,1000},{t_out_bytes,13000},{t_bad_chunks,0},{t_elapsed_seconds,0.647}]
 The data repair process, executed by `b`'s chain manager, found 1000
 chunks that were out of sync and copied them to `a` successfully.
 ### 7. Exploring the rest of the client APIs
 Please see the EDoc documentation for the client APIs.  Feel free to
 explore!
 * [Erlang type definitions for the client APIs](http://basho.github.io/machi/edoc/machi_flu1_client.html)
 * [EDoc for machi_cr_client.erl](http://basho.github.io/machi/edoc/machi_cr_client.html)
 * [EDoc for machi_proxy_flu1_client.erl](http://basho.github.io/machi/edoc/machi_proxy_flu1_client.html)
 * [Top level EDoc collection](http://basho.github.io/machi/edoc/)
--- a/178
+++ b/178
@ -1,178 +0,0 @@
                                 Apache License
                           Version 2.0, January 2004
                        http://www.apache.org/licenses/
   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
   1. Definitions.
      "License" shall mean the terms and conditions for use, reproduction,
      and distribution as defined by Sections 1 through 9 of this document.
      "Licensor" shall mean the copyright owner or entity authorized by
      the copyright owner that is granting the License.
      "Legal Entity" shall mean the union of the acting entity and all
      other entities that control, are controlled by, or are under common
      control with that entity. For the purposes of this definition,
      "control" means (i) the power, direct or indirect, to cause the
      direction or management of such entity, whether by contract or
      otherwise, or (ii) ownership of fifty percent (50%) or more of the
      outstanding shares, or (iii) beneficial ownership of such entity.
      "You" (or "Your") shall mean an individual or Legal Entity
      exercising permissions granted by this License.
      "Source" form shall mean the preferred form for making modifications,
      including but not limited to software source code, documentation
      source, and configuration files.
      "Object" form shall mean any form resulting from mechanical
      transformation or translation of a Source form, including but
      not limited to compiled object code, generated documentation,
      and conversions to other media types.
      "Work" shall mean the work of authorship, whether in Source or
      Object form, made available under the License, as indicated by a
      copyright notice that is included in or attached to the work
      (an example is provided in the Appendix below).
      "Derivative Works" shall mean any work, whether in Source or Object
      form, that is based on (or derived from) the Work and for which the
      editorial revisions, annotations, elaborations, or other modifications
      represent, as a whole, an original work of authorship. For the purposes
      of this License, Derivative Works shall not include works that remain
      separable from, or merely link (or bind by name) to the interfaces of,
      the Work and Derivative Works thereof.
      "Contribution" shall mean any work of authorship, including
      the original version of the Work and any modifications or additions
      to that Work or Derivative Works thereof, that is intentionally
      submitted to Licensor for inclusion in the Work by the copyright owner
      or by an individual or Legal Entity authorized to submit on behalf of
      the copyright owner. For the purposes of this definition, "submitted"
      means any form of electronic, verbal, or written communication sent
      to the Licensor or its representatives, including but not limited to
      communication on electronic mailing lists, source code control systems,
      and issue tracking systems that are managed by, or on behalf of, the
      Licensor for the purpose of discussing and improving the Work, but
      excluding communication that is conspicuously marked or otherwise
      designated in writing by the copyright owner as "Not a Contribution."
      "Contributor" shall mean Licensor and any individual or Legal Entity
      on behalf of whom a Contribution has been received by Licensor and
      subsequently incorporated within the Work.
   2. Grant of Copyright License. Subject to the terms and conditions of
      this License, each Contributor hereby grants to You a perpetual,
      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
      copyright license to reproduce, prepare Derivative Works of,
      publicly display, publicly perform, sublicense, and distribute the
      Work and such Derivative Works in Source or Object form.
   3. Grant of Patent License. Subject to the terms and conditions of
      this License, each Contributor hereby grants to You a perpetual,
      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
      (except as stated in this section) patent license to make, have made,
      use, offer to sell, sell, import, and otherwise transfer the Work,
      where such license applies only to those patent claims licensable
      by such Contributor that are necessarily infringed by their
      Contribution(s) alone or by combination of their Contribution(s)
      with the Work to which such Contribution(s) was submitted. If You
      institute patent litigation against any entity (including a
      cross-claim or counterclaim in a lawsuit) alleging that the Work
      or a Contribution incorporated within the Work constitutes direct
      or contributory patent infringement, then any patent licenses
      granted to You under this License for that Work shall terminate
      as of the date such litigation is filed.
   4. Redistribution. You may reproduce and distribute copies of the
      Work or Derivative Works thereof in any medium, with or without
      modifications, and in Source or Object form, provided that You
      meet the following conditions:
      (a) You must give any other recipients of the Work or
          Derivative Works a copy of this License; and
      (b) You must cause any modified files to carry prominent notices
          stating that You changed the files; and
      (c) You must retain, in the Source form of any Derivative Works
          that You distribute, all copyright, patent, trademark, and
          attribution notices from the Source form of the Work,
          excluding those notices that do not pertain to any part of
          the Derivative Works; and
      (d) If the Work includes a "NOTICE" text file as part of its
          distribution, then any Derivative Works that You distribute must
          include a readable copy of the attribution notices contained
          within such NOTICE file, excluding those notices that do not
          pertain to any part of the Derivative Works, in at least one
          of the following places: within a NOTICE text file distributed
          as part of the Derivative Works; within the Source form or
          documentation, if provided along with the Derivative Works; or,
          within a display generated by the Derivative Works, if and
          wherever such third-party notices normally appear. The contents
          of the NOTICE file are for informational purposes only and
          do not modify the License. You may add Your own attribution
          notices within Derivative Works that You distribute, alongside
          or as an addendum to the NOTICE text from the Work, provided
          that such additional attribution notices cannot be construed
          as modifying the License.
      You may add Your own copyright statement to Your modifications and
      may provide additional or different license terms and conditions
      for use, reproduction, or distribution of Your modifications, or
      for any such Derivative Works as a whole, provided Your use,
      reproduction, and distribution of the Work otherwise complies with
      the conditions stated in this License.
   5. Submission of Contributions. Unless You explicitly state otherwise,
      any Contribution intentionally submitted for inclusion in the Work
      by You to the Licensor shall be under the terms and conditions of
      this License, without any additional terms or conditions.
      Notwithstanding the above, nothing herein shall supersede or modify
      the terms of any separate license agreement you may have executed
      with Licensor regarding such Contributions.
   6. Trademarks. This License does not grant permission to use the trade
      names, trademarks, service marks, or product names of the Licensor,
      except as required for reasonable and customary use in describing the
      origin of the Work and reproducing the content of the NOTICE file.
   7. Disclaimer of Warranty. Unless required by applicable law or
      agreed to in writing, Licensor provides the Work (and each
      Contributor provides its Contributions) on an "AS IS" BASIS,
      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
      implied, including, without limitation, any warranties or conditions
      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
      PARTICULAR PURPOSE. You are solely responsible for determining the
      appropriateness of using or redistributing the Work and assume any
      risks associated with Your exercise of permissions under this License.
   8. Limitation of Liability. In no event and under no legal theory,
      whether in tort (including negligence), contract, or otherwise,
      unless required by applicable law (such as deliberate and grossly
      negligent acts) or agreed to in writing, shall any Contributor be
      liable to You for damages, including any direct, indirect, special,
      incidental, or consequential damages of any character arising as a
      result of this License or out of the use or inability to use the
      Work (including but not limited to damages for loss of goodwill,
      work stoppage, computer failure or malfunction, or any and all
      other commercial damages or losses), even if such Contributor
      has been advised of the possibility of such damages.
   9. Accepting Warranty or Additional Liability. While redistributing
      the Work or Derivative Works thereof, You may choose to offer,
      and charge a fee for, acceptance of support, warranty, indemnity,
      or other liability obligations and/or rights consistent with this
      License. However, in accepting such obligations, You may act only
      on Your own behalf and on Your sole responsibility, not on behalf
      of any other Contributor, and only if You agree to indemnify,
      defend, and hold each Contributor harmless for any liability
      incurred by, or claims asserted against, such Contributor by reason
      of your accepting any such warranty or additional liability.
   END OF TERMS AND CONDITIONS
--- a/94
+++ b/94
@ -1,94 +0,0 @@
 REPO            ?= machi
 PKG_REVISION    ?= $(shell git describe --tags)
 PKG_BUILD        = 1
 BASE_DIR         = $(shell pwd)
 ERLANG_BIN       = $(shell dirname $(shell which erl))
 REBAR := $(shell which rebar)
 ifeq ($(REBAR),)
 REBAR = $(BASE_DIR)/rebar
 endif
 OVERLAY_VARS    ?=
 EUNIT_OPTS       = -v
 .PHONY: rel stagedevrel deps package pkgclean edoc
 all: deps compile
 compile:
 	$(REBAR) compile
 ## Make reltool happy by creating a fake entry in the deps dir for
 ## machi, because reltool really wants to have a path with
 ## "machi/ebin" at the end, but we also don't want infinite recursion
 ## if we just symlink "deps/machi" -> ".."
 generate:
 	rm -rf deps/machi
 	mkdir deps/machi
 	ln -s ../../ebin deps/machi
 	ln -s ../../src deps/machi
 	$(REBAR) generate $(OVERLAY_VARS) 2>&1 | grep -v 'command does not apply to directory'
 deps:
 	$(REBAR) get-deps
 clean:
 	$(REBAR) -r clean
 edoc: edoc-clean
 	$(REBAR) skip_deps=true doc
 edoc-clean:
 	rm -f edoc/*.png edoc/*.html edoc/*.css edoc/edoc-info
 pulse: compile
 	@echo Sorry, PULSE test needs maintenance. -SLF
 	#env USE_PULSE=1 $(REBAR) skip_deps=true clean compile
 	#env USE_PULSE=1 $(REBAR) skip_deps=true -D PULSE eunit -v
 ##
 ## Release targets
 ##
 rel: deps compile generate
 relclean:
 	rm -rf rel/$(REPO)
 stage : rel
 	$(foreach dep,$(wildcard deps/*), rm -rf rel/$(REPO)/lib/$(shell basename $(dep))* && ln -sf $(abspath $(dep)) rel/$(REPO)/lib;)
 ##
 ## Developer targets
 ##
 ##  devN - Make a dev build for node N
 ##  stagedevN - Make a stage dev build for node N (symlink libraries)
 ##  devrel - Make a dev build for 1..$DEVNODES
 ##  stagedevrel Make a stagedev build for 1..$DEVNODES
 ##
 ##  Example, make a 68 node devrel cluster
 ##    make stagedevrel DEVNODES=68
 .PHONY : stagedevrel devrel
 DEVNODES ?= 3
 # 'seq' is not available on all *BSD, so using an alternate in awk
 SEQ = $(shell awk 'BEGIN { for (i = 1; i < '$(DEVNODES)'; i++) printf("%i ", i); print i ;exit(0);}')
 $(eval stagedevrel : $(foreach n,$(SEQ),stagedev$(n)))
 $(eval devrel : $(foreach n,$(SEQ),dev$(n)))
 dev% : all
 	mkdir -p dev
 	rel/gen_dev $@ rel/vars/dev_vars.config.src rel/vars/$@_vars.config
 	(cd rel && ../rebar generate target_dir=../dev/$@ overlay_vars=vars/$@_vars.config)
 stagedev% : dev%
 	  $(foreach dep,$(wildcard deps/*), rm -rf dev/$^/lib/$(shell basename $(dep))* && ln -sf $(abspath $(dep)) dev/$^/lib;)
 devclean: clean
 	rm -rf dev
 DIALYZER_APPS = kernel stdlib sasl erts ssl compiler eunit crypto public_key syntax_tools
 PLT = $(HOME)/.machi_dialyzer_plt
 include tools.mk
--- a/6
+++ b/6
@ -1,6 +0,0 @@
 Machi
 Copyright 2007-2015 Basho Technologies
 This product contains code developed at Basho Technologies.
 (http://www.basho.com/)
--- a/README.md
+++ b/README.md
@ -1,174 +0,0 @@
 # Machi: a distributed, decentralized blob/large file store
  [Travis-CI](http://travis-ci.org/basho/machi) :: ![Travis-CI](https://secure.travis-ci.org/basho/machi.png)
 Outline
 1. [Why another blob/file store?](#sec1)
 2. [Where to learn more about Machi](#sec2)
 3. [Development status summary](#sec3)
 4. [Contributing to Machi's development](#sec4)
 <a name="sec1">
 ## 1. Why another blob/file store?
 Our goal is a robust & reliable, distributed, highly available, large
 file and blob store.  Such stores already exist, both in the open source world
 and in the commercial world.  Why reinvent the wheel?  We believe
 there are three reasons, ordered by decreasing rarity.
 1. We want end-to-end checksums for all file data, from the initial
   file writer to every file reader, anywhere, all the time.
 2. We need flexibility to trade consistency for availability:
   e.g. weak consistency in exchange for being available in cases
   of partial system failure.
 3. We want to manage file replicas in a way that's provably correct
   and also easy to test.
 Criteria #3 is difficult to find in the open source world but perhaps
 not impossible.
 If we have app use cases where availability is more important than
 consistency, then systems that meet criteria #2 are also rare.
 Most file stores provide only strong consistency and therefore
 have unavoidable, unavailable behavior when parts of the system
 fail.
 What if we want a file store that is always available to write new
 file data and attempts best-effort file reads?
 If we really do care about data loss and/or data corruption, then we
 really want both #3 and #1.  Unfortunately, systems that meet
 criteria #1 are _very rare_.  (Nonexistant?)
 Why?  This is 2015.  We have decades of research that shows
 that computer hardware can (and
 indeed does) corrupt data at nearly every level of the modern
 client/server application stack.  Systems with end-to-end data
 corruption detection should be ubiquitous today.  Alas, they are not.
 Machi is an effort to change the deplorable state of the world, one
 Erlang function at a time.
 <a name="sec2">
 ## 2. Where to learn more about Machi
 The two major design documents for Machi are now mostly stable.
 Please see the [doc](./doc) directory's [README](./doc) for details.
 We also have a
 [Frequently Asked Questions (FAQ) list](./FAQ.md).
 Scott recently (November 2015) gave a presentation at the
 [RICON 2015 conference](http://ricon.io) about one of the techniques
 used by Machi; "Managing Chain Replication Metadata with
 Humming Consensus" is available online now.
 * [slides (PDF format)](http://ricon.io/speakers/slides/Scott_Fritchie_Ricon_2015.pdf)
 * [video](https://www.youtube.com/watch?v=yR5kHL1bu1Q)
 See later in this document for how to run the Humming Consensus demos,
 including the network partition simulator.
 <a name="sec3">
 ## 3. Development status summary
 Mid-March 2016: The Machi development team has been downsized in
 recent months, and the pace of development has slowed.  Here is a
 summary of the status of Machi's major components.
 * Humming Consensus and the chain manager
  * No new safety bugs have been found by model-checking tests.
  * A new document,
    [Hands-on experiments with Machi and Humming Consensus](doc/humming-consensus-demo.md)
    is now available.  It is a tutorial for setting up a 3 virtual
    machine Machi cluster and how to demonstrate the chain manager's
    reactions to server stops & starts, crashes & restarts, and pauses
    (simulated by `SIGSTOP` and `SIGCONT`).
  * The chain manager can still make suboptimal-but-safe choices for
    chain transitions when a server hangs/pauses temporarily.
    * Recent chain manager changes have made the instability window
      much shorter when the slow/paused server resumes execution.
    * Scott believes that a modest change to the chain manager's
      calculation of a new projection can reduce flapping in this (and
      many other cases) less likely.  Currently, the new local
      projection is calculated using only local state (i.e., the chain
      manager's internal state + the fitness server's state).
      However, if the "latest" projection read from the public
      projection stores were also input to the new projection
      calculation function, then many obviously bad projections can be
      avoided without needing rounds of Humming Consensus to
      demonstrate that a bad projection is bad.
 * FLU/data server process
  * All known correctness bugs have been fixed.
  * Performance has not yet been measured.  Performance measurement
    and enhancements are scheduled to start in the middle of March 2016.
    (This will include a much-needed update to the `basho_bench` driver.)
 * Access protocols and client libraries
  * The protocol used by both external clients and internally (instead
    of using Erlang's native message passing mechanisms) is based on
    Protocol Buffers.
    * (Machi PB protocol specification: ./src/machi.proto)[./src/machi.proto]
    * At the moment, the PB specification contains two protocols.
      Sometime in the near future, the spec will be split to separate
      the external client API (the "high" protocol) from the internal
      communication API (the "low" protocol).
 * Recent conference talks about Machi
  * Erlang Factory San Francisco 2016
    [the slides and video recording](http://www.erlang-factory.com/sfbay2016/scott-lystig-fritchie)
    will be available a few weeks after the conference ends on March
    11, 2016.
  * Ricon 2015
    * [The slides](http://ricon.io/archive/2015/slides/Scott_Fritchie_Ricon_2015.pdf)
    * and the [video recording](https://www.youtube.com/watch?v=yR5kHL1bu1Q&index=13&list=PL9Jh2HsAWHxIc7Tt2M6xez_TOP21GBH6M)
    are now available.
    * If you would like to run the Humming Consensus code (with or without
    the network partition simulator) as described in the RICON 2015
    presentation, please see the
    [Humming Consensus demo doc](./doc/humming_consensus_demo.md).
 <a name="sec4">
 ## 4. Contributing to Machi's development
 ### 4.1 License
 Basho Technologies, Inc. as committed to licensing all work for Machi
 under the
 [Apache Public License version 2](./LICENSE).  All authors of source code
 and documentation who agree with these licensing terms are welcome to
 contribute their ideas in any form: suggested design or features,
 documentation, and source code.
 Machi is still a very young project within Basho, with a small team of
 developers; please bear with us as we grow out of "toddler" stage into
 a more mature open source software project.
 We invite all contributors to review the
 [CONTRIBUTING.md](./CONTRIBUTING.md) document for guidelines for
 working with the Basho development team.
 ### 4.2 Development environment requirements
 All development to date has been done with Erlang/OTP version 17 on OS
 X.  The only known limitations for using R16 are minor type
 specification difference between R16 and 17, but we strongly suggest
 continuing development using version 17.
 We also assume that you have the standard UNIX/Linux developer
 tool chain for C and C++ applications.  Also, we assume
 that Git and GNU Make are available.
 The utility used to compile the Machi source code,
 `rebar`, is pre-compiled and included in the repo.
 For more details, please see the
 [Machi development environment prerequisites doc](./doc/dev-prerequisites.md).
 Machi has a dependency on the
 [ELevelDB](https://github.com/basho/eleveldb) library.  ELevelDB only
 supports UNIX/Linux OSes and 64-bit versions of Erlang/OTP only; we
 apologize to Windows-based and 32-bit-based Erlang developers for this
 restriction.
 ### 4.3 New protocols and features
 If you'd like to work on a protocol such as Thrift, UBF,
 msgpack over UDP, or some other protocol, let us know by
 [opening an issue to discuss it](./issues/new).
--- a/TODO-shortterm.org
+++ b/TODO-shortterm.org
@ -1,114 +0,0 @@
 * To Do list
 ** DONE remove the escript* stuff from machi_util.erl
 ** DONE Add functions to manipulate 1-chain projections
 - Add epoch ID = epoch number + checksum of projection!
  Done via compare() func.
 ** DONE Change all protocol ops to add epoch ID
 ** DONE Add projection store to each FLU.
 *** DONE What should the API look like? (borrow from chain mgr PoC?)
 Yeah, I think that's pretty complete.  Steal it now, worry later.
 *** DONE Choose protocol & TCP port. Share with get/put? Separate?
 Hrm, I like the idea of having a single TCP port to talk to any single
 FLU.
 To make the protocol "easy" to hack, how about using the same basic
 method as append/write where there's a variable size blob.  But we'll
 format that blob as a term_to_binary().  Then dispatch to a single
 func, and pattern match Erlang style in that func.
 *** DONE Do it.
 ** DONE Finish OTP'izing the Chain Manager with FLU & proj store processes
 ** DONE Eliminate the timeout exception for the client: just {error,timeout} ret
 ** DONE Move prototype/chain-manager code to "top" of source tree
 *** DONE Preserve current test code (leave as-is? tiny changes?)
 *** DONE Make chain manager code flexible enough to run "real world" or "sim"
 ** DONE Add projection wedging logic to each FLU.
 ** DONE Implement real data repair, orchestrated by the chain manager
 ** DONE Change all protocol ops to enforce the epoch ID
 - Add no-wedging state to make testing easier?
 ** DONE Adapt the projection-aware, CR-implementing client from demo-day
 ** DONE Add major comment sections to the CR-impl client
 ** DONE Simple basho_bench driver, put some unscientific chalk on the benchtop
 ** TODO Create parallel PULSE test for basic API plus chain manager repair
 ** DONE Add client-side vs. server-side checksum type, expand client API?
 ** TODO Add gproc and get rid of registered name rendezvous
 *** TODO Fixes the atom table leak
 *** TODO Fixes the problem of having active sequencer for the same prefix
         on two FLUS in the same VM
 ** TODO Fix all known bugs/cruft with Chain Manager (list below)
 *** DONE Fix known bugs
 *** DONE Clean up crufty TODO comments and other obvious cruft
 *** TODO Re-add verification step of stable epochs, including inner projections!
 *** TODO Attempt to remove cruft items in flapping_i?
 ** TODO Move the FLU server to gen_server behavior?
 * DONE Chain manager CP mode, Plan B
 ** SKIP Maybe? Change ch_mgr to use middleworker
 **** DONE Is it worthwhile?  Is the parallelism so important?  No, probably.
 **** SKIP Move middleworker func to utility module?
 ** DONE Add new proc to psup group
 *** DONE Name: machi_fitness
 ** DONE ch_mgr keeps its current proc struct: i.e. same 1 proc as today
 ** NO chmgr asks hosed mgr for hosed list @ start of react_to_env
 ** DONE For all hosed, do *async*: try to read latest proj.
 *** NO If OK, inform hosed mgr: status change will be used by next HC iter.
 *** NO If fail, no change, because that server is already known to be hosed
 *** DONE For all non-hosed, continue as the chain manager code does today
 *** DONE Any new errors are added to UpNodes/DownNodes tracking as used today
 *** DONE At end of react loop, if UpNodes list differs, inform hosed mgr.
 * DONE fitness_mon, the fitness monitor
 ** DONE Map key & val sketch
 Logical sketch:
 Map key: ObservingServerName::atom()
 Map val: { ObservingServerLastModTime::now(),
           UnfitList::list(ServerName::atom()),
           AdminDownList::list(ServerName::atom()),
           Props::proplist() }
 Implementation sketch:
 1. Use CRDT map.
 2. If map key is not atom, then atom->string or atom->binary is fine.
 3. For map value, is it possible CRDT LWW type?
 ** DONE Investigate riak_dt data structure definition, manipulating, etc.
 ** DONE Add dependency on riak_dt
 ** DONE Update is an entire dict from Observer O
 *** DONE Merge my pending map + update map + my last mod time + my unfit list
 *** DONE if merged /= pending:
 **** DONE Schedule async tick (more)
 Tick message contains list of servers with differing state as of this
 instant in time... we want to avoid triggering decisions about
 fitness/unfitness for other servers where we might have received less
 than a full time period's worth of waiting.
 **** DONE Spam merged map to All_list -- [Me]
 **** DONE Set pending <- merged
 *** DONE When we receive an async tick
 **** DONE set active map <- pending map for all servers in ticks list
 **** DONE Send ch_mgr a react_to_env tick trigger
 *** DONE react_to_env tick trigger actions
 **** DONE Filter active map to remove stale entries (i.e. no update in 1 hour)
 **** DONE If time since last map spam is too long, spam our *pending* map
 **** DONE Proceed with normal react processing, using *active* map for AllHosed!
--- a/dialyzer.ignore-warnings
+++ b/dialyzer.ignore-warnings
@ -1,15 +0,0 @@
 ### The auto-generated code of machi_pb.beam has some complaints, not fixed yet.
 machi_pb.erl:0:
 ##################################################
 ########  Specific types     #####################
 ##################################################
 Unknown types:
  basho_bench_config:get/2
  machi_partition_simulator:get/1
  hamcrest:matchspec/0
 ##################################################
 ########  Specific messages  #####################
 ##################################################
 machi_chain_manager1.erl:2473: The created fun has no local return
 machi_chain_manager1.erl:2184: The pattern <_P1, P2, Else = {'expected_author2', UPI1_tail, _}> can never match the type <#projection_v1{epoch_number::'undefined' | non_neg_integer(),epoch_csum::'undefined' | binary(),author_server::atom(),chain_name::atom(),all_members::'undefined' | [atom()],witnesses::[atom()],creation_time::'undefined' | {non_neg_integer(),non_neg_integer(),non_neg_integer()},mode::'ap_mode' | 'cp_mode',upi::'undefined' | [atom()],repairing::'undefined' | [atom()],down::'undefined' | [atom()],dbg::'undefined' | [any()],dbg2::'undefined' | [any()],members_dict::'undefined' | [{_,_}]},#projection_v1{epoch_number::'undefined' | non_neg_integer(),epoch_csum::binary(),author_server::atom(),chain_name::atom(),all_members::'undefined' | [atom()],witnesses::[atom()],creation_time::'undefined' | {non_neg_integer(),non_neg_integer(),non_neg_integer()},mode::'ap_mode' | 'cp_mode',upi::'undefined' | [atom()],repairing::'undefined' | [atom()],down::'undefined' | [atom()],dbg::'undefined' | [any()],dbg2::'undefined' | [any()],members_dict::'undefined' | [{_,_}]},'true'>
 machi_chain_manager1.erl:2233: The pattern <_P1 = {'projection_v1', _, _, _, _, _, _, _, 'cp_mode', UPI1, Repairing1, _, _, _, _}, _P2 = {'projection_v1', _, _, _, _, _, _, _, 'cp_mode', UPI2, Repairing2, _, _, _, _}, Else = {'epoch_not_si', EpochX, 'not_gt', EpochY}> can never match the type <#projection_v1{epoch_number::'undefined' | non_neg_integer(),epoch_csum::'undefined' | binary(),author_server::atom(),chain_name::atom(),all_members::'undefined' | [atom()],witnesses::[atom()],creation_time::'undefined' | {non_neg_integer(),non_neg_integer(),non_neg_integer()},mode::'ap_mode' | 'cp_mode',upi::'undefined' | [atom()],repairing::'undefined' | [atom()],down::'undefined' | [atom()],dbg::'undefined' | [any()],dbg2::'undefined' | [any()],members_dict::'undefined' | [{_,_}]},#projection_v1{epoch_number::'undefined' | non_neg_integer(),epoch_csum::binary(),author_server::atom(),chain_name::atom(),all_members::'undefined' | [atom()],witnesses::[atom()],creation_time::'undefined' | {non_neg_integer(),non_neg_integer(),non_neg_integer()},mode::'ap_mode' | 'cp_mode',upi::'undefined' | [atom()],repairing::'undefined' | [atom()],down::'undefined' | [atom()],dbg::'undefined' | [any()],dbg2::'undefined' | [any()],members_dict::'undefined' | [{_,_}]},'true'>
--- a/doc/README.md
+++ b/doc/README.md
@ -1,74 +0,0 @@
 ## Machi Documentation Overview
 For a Web-browsable version of a snapshot of the source doc "EDoc"
 Erlang documentation, please use this link:
 [Machi EDoc snapshot](https://basho.github.io/machi/edoc/).
 ## Documents in this directory
 ### high-level-machi.pdf
 [high-level-machi.pdf](high-level-machi.pdf)
 is an overview of the high level design for
 Machi.  Its abstract:
 > Our goal is a robust & reliable, distributed, highly available large
 > file store based upon write-once registers, append-only files, Chain
 > Replication, and client-server style architecture.  All members of
 > the cluster store all of the files.  Distributed load
 > balancing/sharding of files is outside of the scope of this system.
 > However, it is a high priority that this system be able to integrate
 > easily into systems that do provide distributed load balancing,
 > e.g., Riak Core.  Although strong consistency is a major feature of
 > Chain Replication, this document will focus mainly on eventual
 > consistency features --- strong consistency design will be discussed
 > in a separate document.
 ### high-level-chain-mgr.pdf
 [high-level-chain-mgr.pdf](high-level-chain-mgr.pdf)
 is an overview of the techniques used by
 Machi to manage Chain Replication metadata state.  It also provides an
 introduction to the Humming Consensus algorithm.  Its abstract:
 > Machi is an immutable file store, now in active development by Basho
 > Japan KK.  Machi uses Chain Replication to maintain strong consistency
 > of file updates to all replica servers in a Machi cluster.  Chain
 > Replication is a variation of primary/backup replication where the
 > order of updates between the primary server and each of the backup
 > servers is strictly ordered into a single "chain".  Management of
 > Chain Replication's metadata, e.g., "What is the current order of
 > servers in the chain?", remains an open research problem.  The
 > current state of the art for Chain Replication metadata management
 > relies on an external oracle (e.g., ZooKeeper) or the Elastic
 > Replication algorithm.
 > 
 > This document describes the Machi chain manager, the component
 > responsible for managing Chain Replication metadata state.  The chain
 > manager uses a new technique, based on a variation of CORFU, called
 > "humming consensus".
 > Humming consensus does not require active participation by all or even
 > a majority of participants to make decisions.  Machi's chain manager
 > bases its logic on humming consensus to make decisions about how to
 > react to changes in its environment, e.g. server crashes, network
 > partitions, and changes by Machi cluster admnistrators.  Once a
 > decision is made during a virtual time epoch, humming consensus will
 > eventually discover if other participants have made a different
 > decision during that epoch.  When a differing decision is discovered,
 > new time epochs are proposed in which a new consensus is reached and
 > disseminated to all available participants.
 ### chain-self-management-sketch.org
 [chain-self-management-sketch.org](chain-self-management-sketch.org)
 is a mostly-deprecated draft of
 an introduction to the
 self-management algorithm proposed for Machi.  Most material has been
 moved to the [high-level-chain-mgr.pdf](high-level-chain-mgr.pdf) document.
 ### cluster (directory)
 This directory contains the sketch of the cluster design
 strawman for partitioning/distributing/sharding files across a large
 number of independent Machi chains.
--- a/doc/Using-Basho-Bench.md
+++ b/doc/Using-Basho-Bench.md
@ -1,258 +0,0 @@
 # Using basho_bench to twiddle with Machi
 "Twiddle"?  Really, is that a word?  (Yes, it is a real English word.)
 ## Benchmarking Machi's performance ... no, don't do it.
 Machi isn't ready for benchmark testing.  Its public-facing API isn't
 finished yet.  Its internal APIs aren't quite finished yet either.  So
 any results of "benchmarking" effort is something that has even less
 value **N** months from now than the usual benchmarking effort.
 However, there are uses for a benchmark tool.  For example, one of my
 favorites is to put **stress** on a system.  I don't care about
 average or 99th-percentile latencies, but I **might** care very much
 about behavior.
 * What happens if a Machi system is under moderate load, and then I
  stop one of the servers?  What happens?
    * How quickly do the chain managers react?
    * How quickly do the client libraries within Machi react?
    * How quickly do the external client API libraries react?
 * What happens if a Machi system is under heavy load, for example,
  100% CPU load.  Not all 100% might be the Machi services.  Some CPU
  consumption might be from the load generator, like `basho_bench`
  itself that is running on the same machine as a Machi server.  Or
  perhaps it's a tiny C program that I wrote:
    main()
    { while (1) { ; } }
 ## An example of how adding moderate stress can find weird bugs
 The driver/plug-in module for `basho_bench` is only a few hours old.
 (I'm writing on Wednesday, 2015-05-20.)  But just now, I configured my
 basho_bench config file to try to contact a Machi cluster of three
 nodes ... but really, only one was running.  The client library,
 `machi_cr_client.erl`, has **an extremely simple** method for dealing
 with failed servers.  I know it's simple and dumb, but that's OK in
 many cases.
 However, `basho_bench` and the `machi_cr_client.erl` were acting very,
 very badly.  I couldn't figure it out until I took a peek at my OS's
 `dmesg` output, namely: `dmesg | tail`.  It said things like this:
    Limiting closed port RST response from 690 to 50 packets per second
    Limiting closed port RST response from 367 to 50 packets per second
    Limiting closed port RST response from 101 to 50 packets per second
    Limiting closed port RST response from 682 to 50 packets per second
    Limiting closed port RST response from 467 to 50 packets per second
 Well, isn't that interesting?
 This system was running on a single OS X machine: my MacBook Pro
 laptop, running OS X 10.10 (Yosemite).  I have seen that error
 before.  And I know how to fix it.
 * **Option 1**: Change the client library config to ignore the Machi
    servers that I know will always be down during my experiment.
 * ** Option 2**: Use the following to change my OS's TCP stack RST
    behavior.  (If a TCP port is not being listened to, the OS will
    send a RST packet to signal "connection refused".)
 On OS X, the limit for RST packets is 50/second.  The
 `machi_cr_client.erl` client can generate far more than 50/second, as
 the `Limiting closed port RST response...` messages above show.  So, I
 used some brute-force to change the environment:
    sudo sysctl -w net.inet.icmp.icmplim=20000
 ... and the problem disappeared.
 ## Starting with basho_bench: a step-by-step tutorial
 First, clone the `basho_bench` source code, then compile it.  You will
 need Erlang/OTP version R16B or later to compile.  I recommend using
 Erlang/OTP 17.x, because I've been doing my Machi development using
 17.x.
    cd /some/nice/dev/place
    git clone https://github.com/basho/basho_bench.git
    cd basho_bench
    make
 In order to create graphs of `basho_bench` output, you'll need
 installed one of the following:
 * R (the statistics package)
 * gnuplot
 If you don't have either available on the machine(s) you're testing,
 but you do have R (or gnuplot) on some other machine **Y**, then you can
 copy the output files to machine **Y** and generate the graphs there.
 ## Compiling the Machi source
 First, clone the Machi source code, then compile it.  You will
 need Erlang/OTP version 17.x to compile.
    cd /some/nice/dev/place
    git clone https://github.com/basho/machi.git
    cd machi
    make
 ## Creating a basho_bench test configuration file.
 There are a couple of example `basho_bench` configuration files in the
 Machi `priv` directory.
 * [basho_bench.append-example.config](priv/basho_bench.append-example.config),
  an example for writing Machi files.
 * [basho_bench.read-example.config](priv/basho_bench.read-example.config),
  an example for reading Machi files.
 If you want a test to do both reading & writing ... well, the
 driver/plug-in is not mature enough to do it **well**.  If you really
 want to, refer to the `basho_bench` docs for how to use the
 `operations` config option.
 The `basho_bench` config file is configured in Erlang term format.
 Each configuration item is a 2-tuple followed by a period.  Comments
 begin with a `%` character and continue to the end-of-line.
    %% Mandatory: adjust this code path to top of your compiled Machi source distro
    {code_paths, ["/Users/fritchie/b/src/machi"]}.
    {driver, machi_basho_bench_driver}.
    %% Chose your maximum rate (per worker proc, see 'concurrent' below)
    {mode, {rate, 25}}.
    %% Runtime & reporting interval 
    {duration, 10}.         % minutes
    {report_interval, 1}.   % seconds
    %% Choose your number of worker procs
    {concurrent, 5}.
    %% Here's a chain of (up to) length 3, all on localhost
    {machi_server_info,
     [
      {p_srvr,a,machi_flu1_client,"localhost",4444,[]},
      {p_srvr,b,machi_flu1_client,"localhost",4445,[]},
      {p_srvr,c,machi_flu1_client,"localhost",4446,[]}
     ]}.
    {machi_ets_key_tab_type, set}.   % 'set' or 'ordered_set'
    %% Workload-specific definitions follow....
    %% 10 parts 'append' operation + 0 parts anything else = 100% 'append' ops
    {operations, [{append, 10}]}.
    %% For append, key = Machi file prefix name
    {key_generator, {concat_binary, <<"prefix">>,
                                    {to_binstr, "~w", {uniform_int, 30}}}}.
    %% Increase size of value_generator_source_size if value_generator is big!!
    {value_generator_source_size, 2111000}.
    {value_generator, {fixed_bin, 32768}}.   %  32 KB
 In summary:
 * Yes, you really need to change `code_paths` to be the same as your
  `/some/nice/dev/place/basho_bench` directory ... and that directory
  must be on the same machine(s) that you intend to run `basho_bench`.
 * Each worker process will have a rate limit of 25 ops/sec.
 * The test will run for 10 minutes and report stats every 1 second.
 * There are 5 concurrent worker processes.  Each worker will
  concurrently issue commands from the `operations` list, within the
  workload throttle limit.
 * The Machi cluster is a collection of three servers, all on
  "localhost", and using TCP ports 4444-4446.
 * Don't change the `machi_ets_key_tab_type`
 * Our workload operation mix is 100% `append` operations.
 * The key generator for the `append` operation specifies the file
  prefix that will be chosen (at pseudo-random).  In this case, we'll
  choose uniformly randomly between file prefix `prefix0` and
  `prefix29`.
 * The values that we append will be fixed 32KB length, but they will
  be chosen from a random byte string of 2,111,000 bytes.
 There are many other options for `basho_bench`, especially for the
 `key_generator` and `value_generator` options.  Please see the
 `basho_bench` docs for further information.
 ## Running basho_bench
 You can run `basho_bench` using the command:
    /some/nice/dev/place/basho_bench/basho_bench /path/to/config/file
 ... where `/path/to/config/file` is the path to your config file. (If
 you use an example from the `priv` dir, we recommend that you make a
 copy elsewhere, edit the copy, and then use the copy to run
 `basho_bench`.)
 You'll create a stats output directory, called `tests`, in the current
 working directory. (Add `{results_dir, "/some/output/dir"}.` to change
 the default!)
 Each time `basho_bench` is run, a new output stats directory is
 created in the `tests` directory.  The symbolic link `tests/current`
 will always point to the last `basho_bench` run's output.  But all
 prior results are always accessible!  Take a look in this directory
 for all of the output.
 ## Generating some pretty graphs
 If you are using R, then the following command will create a graph:
    Rscript --vanilla /some/nice/dev/place/basho_bench/basho_bench/priv/summary.r -i $CWD/tests/current
 If the `tests` directory is not in your current working dir (i.e. not
 in `$CWD`), then please alter the command accordingly.
 R will create the final results graph in `$CWD/tests/current/summary.png`.
 If you are using gnuplot, please look at
 `/some/nice/dev/place/basho_bench/basho_bench/Makefile` to see how to
 use gnuplot to create the final results graph.
 ## An example graph
 So, without a lot of context about the **Machi system** or about the
 **basho_bench system** or about the ops being performed, here is an
 example graph that was created by R:
 ![](https://basho.github.io/machi/images/basho_bench.example0.png)
 **Without context??*  How do I remember the context?
 My recommendation is: always keep the `.config` file together with the
 graph file.  In the `tests` directory, `basho_bench` will always make
 a copy of the config file used to generate the test data.
 This config tells you very little about the environment of the load
 generator machine or the Machi cluster, but ... you need to maintain
 that documentation yourself, please!  You'll thank me for that advice,
 someday, 11 months from now when you can't remember the details of
 that important test that you ran so very long ago.
 ## Conclusion
 Really, we don't recommend using `basho_bench` for any serious
 performance measurement of Machi yet: Machi needs more maturity before
 it's reasonable to measure & judge its performance.  But stress
 testing is indeed useful for reasons other than measuring
 Nth-percentile latency of operation `flarfbnitz`.  We hope that this
 tutorial has been helpful!
 If you encounter any difficulty with this tutorial or with Machi,
 please open an issue/ticket at [GH Issues for
 Machi](https://github.com/basho/machi/issues) ... use the green "New
 issue" button.  There are bugs and misfeatures in the `basho_bench`
 plugin, sorry, but please help us fix them.
 > -Scott Lystig Fritchie,
 > Machi Team @ Basho
--- a/doc/chain-self-management-sketch.Diagram1.dia
+++ b/doc/chain-self-management-sketch.Diagram1.dia
--- a/doc/chain-self-management-sketch.Diagram1.pdf
+++ b/doc/chain-self-management-sketch.Diagram1.pdf
--- a/doc/chain-self-management-sketch.org
+++ b/doc/chain-self-management-sketch.org
@ -1,251 +0,0 @@
 -*- mode: org; -*-
 #+TITLE: Machi Chain Self-Management Sketch
 #+AUTHOR: Scott
 #+STARTUP: lognotedone hidestars indent showall inlineimages
 #+SEQ_TODO: TODO WORKING WAITING DONE
 * 1. Abstract
 The high level design of the Machi "chain manager" has moved to the
 [[high-level-chain-manager.pdf][Machi chain manager high level design]] document.
 We try to discuss the network partition simulator that the
 algorithm runs in and how the algorithm behaves in both symmetric and
 asymmetric network partition scenarios.  The symmetric partition cases
 are all working well (surprising in a good way), and the asymmetric
 partition cases are working well (in a damn mystifying kind of way).
 It'd be really, *really* great to get more review of the algorithm and
 the simulator.
 * 2. Copyright
 #+BEGIN_SRC
 %% Copyright (c) 2015 Basho Technologies, Inc.  All Rights Reserved.
 %%
 %% This file is provided to you under the Apache License,
 %% Version 2.0 (the "License"); you may not use this file
 %% except in compliance with the License.  You may obtain
 %% a copy of the License at
 %%
 %%   http://www.apache.org/licenses/LICENSE-2.0
 %%
 %% Unless required by applicable law or agreed to in writing,
 %% software distributed under the License is distributed on an
 %% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
 %% KIND, either express or implied.  See the License for the
 %% specific language governing permissions and limitations
 %% under the License.
 #+END_SRC
 * 3. Document restructuring  
 Much of the text previously appearing in this document has moved to the
 [[high-level-chain-manager.pdf][Machi chain manager high level design]] document.
 * 4. Diagram of the self-management algorithm
 ** WARNING: This section is now deprecated
 The definitive text for this section has moved to the [[high-level-chain-manager.pdf][Machi chain
 manager high level design]] document.
 ** Flowchart notes
 *** Algorithm execution rates / sleep intervals between executions
 Due to the ranking algorithm's preference for author node names that
 are large (lexicographically), nodes with larger node names should
 execute the algorithm more frequently than other nodes.  The reason
 for this is to try to avoid churn: a proposal by a "small" node may
 propose a UPI list of L at epoch 10, and a few moments later a "big"
 node may propose the same UPI list L at epoch 11.  In this case, there
 would be two chain state transitions: the epoch 11 projection would be
 ranked higher than epoch 10's projection.  If the "big" node
 executed more frequently than the "small" node, then it's more likely
 that epoch 10 would be written by the "big" node, which would then
 cause the "small" node to stop at state A40 and avoid any
 externally-visible action.
 *** A simple example race between two participants noting a 3rd's failure
 Assume a chain of three nodes, A, B, and C.  In a projection at epoch
 E.  For all nodes, the P_current projection at epoch E is:
 #+BEGIN_QUOTE
 UPI=[A,B,C], Repairing=[], Down=[]
 #+END_QUOTE
 Now assume that C crashes during epoch E.  The failure detector
 running locally at both A & B eventually notice C's death.  The new
 information triggers a new iteration of the self-management algorithm.
 A calculates its P_newprop (call it P_newprop_a) and writes it to its
 own public projection store.  Meanwhile, B does the same and wins the
 race to write P_newprop_b to its own public projection store.
 At this instant in time, the public projection stores of each node
 looks something like this:
 |-------+--------------+--------------+--------------|
 | Epoch | Node A       | Node B       | Node C       |
 |-------+--------------+--------------+--------------|
 | E     | UPI=[A,B,C]  | UPI=[A,B,C]  | UPI=[A,B,C]  |
 |       | Repairing=[] | Repairing=[] | Repairing=[] |
 |       | Down=[]      | Down=[]      | Down=[]      |
 |       | Author=A     | Author=A     | Author=A     |
 |-------+--------------+--------------+--------------|
 | E+1   | UPI=[A,B]    | UPI=[A,B]    | C is dead,   |
 |       | Repairing=[] | Repairing=[] | unwritten    |
 |       | Down=[C]     | Down=[C]     |              |
 |       | Author=A     | Author=B     |              |
 |-------+--------------+--------------+--------------|
 If we use the CORFU-style projection naming convention, where a
 projection's name is exactly equal to the epoch number, then all
 participants cannot tell the difference between the projection at
 epoch E+1 authored by node A from the projection at epoch E+1 authored
 by node B: the names are the same, i.e., E+1.
 Machi must extend the original CORFU protocols by changing the name of
 the projection.  In Machi's case, the projection is named by this
 2-tuple: 
 #+BEGIN_SRC
 {epoch #, hash of the entire projection (minus hash field itself)}
 #+END_SRC
 This name is used in all relevant APIs where the name is required to
 make a wedge state transition.  In the case of the example & table
 above, all of the UPI & Repairing & Down lists are equal.  However, A
 & B's unanimity is due to the symmetric nature of C's partition: C is
 dead.  In the case of an asymmetric partition of C, it is indeed
 possible for A's version of epoch E+1's UPI list to be different from
 B's UPI list in the same epoch E+1.
 *** A second example, building on the first example
 Building on the first example, let's assume that A & B have reconciled
 their proposals for epoch E+2.  Nodes A & B are running under a
 unanimous proposal at E+2.
 |-------+--------------+--------------+--------------|
 | E+2   | UPI=[A,B]    | UPI=[A,B]    | C is dead,   |
 |       | Repairing=[] | Repairing=[] | unwritten    |
 |       | Down=[C]     | Down=[C]     |              |
 |       | Author=A     | Author=A     |              |
 |-------+--------------+--------------+--------------|
 Now assume that C restarts.  It was dead for a little while, and its
 code is slightly buggy.  Node C decides to make a proposal without
 first consulting its failure detector: let's assume that C believes
 that only C is alive.  Also, C knows that epoch E was the last epoch
 valid before it crashed, so it decides that it will write its new
 proposal at E+2.  The result is a set of public projection stores that
 look like this:
 |-----+--------------+--------------+--------------|
 | E+2 | UPI=[A,B]    | UPI=[A,B]    | UPI=[C]      |
 |     | Repairing=[] | Repairing=[] | Repairing=[] |
 |     | Down=[C]     | Down=[C]     | Down=[A,B]   |
 |     | Author=A     | Author=A     | Author=C     |
 |-----+--------------+--------------+--------------|
 Now we're in a pickle where a client C could read the latest
 projection from node C and get a different view of the world than if
 it had read the latest projection from nodes A or B.
 If running in AP mode, this wouldn't be a big problem: a write to node
 C only (or a write to nodes A & B only) would be reconciled
 eventually.  Also, eventually, one of the nodes would realize that C
 was no longer partitioned and would make a new proposal at epoch E+3.
 If running in CP mode, then any client that attempted to use C's
 version of the E+2 projection would fail: the UPI list does not
 contain a quorum majority of nodes.  (Other discussion of CP mode's
 use of quorum majority for UPI members is out of scope of this
 document.  Also out of scope is the use of "witness servers" to
 augment the quorum majority UPI scheme.)
 * 5. The Network Partition Simulator
 ** Overview
 The function machi_chain_manager1_test:convergence_demo_test()
 executes the following in a simulated network environment within a
 single Erlang VM:
 #+BEGIN_QUOTE
 Test the convergence behavior of the chain self-management algorithm
 for Machi.
  1. Set up 4 FLUs and chain manager pairs.
  2. Create a number of different network partition scenarios, where
     (simulated) partitions may be symmetric or asymmetric.  (At the
     Seattle 2015 meet-up, I called this the "shaking the snow globe"
     phase, where asymmetric network partitions are simulated and are
     calculated at random differently for each simulated node.  During
     this time, the simulated network is wildly unstable.)
  3. Then halt changing the partitions and keep the simulated network
     stable.  The simulated may remain broken (i.e. at least one
     asymmetric partition remains in effect), but at least it's
     stable.
  4. Run a number of iterations of the algorithm in parallel by poking
     each of the manager processes on a random'ish basis to simulate
     the passage of time.
  5. Afterward, fetch the chain transition histories made by each FLU
     and verify that no transition was ever unsafe.
 #+END_QUOTE
 ** Behavior in symmetric network partitions
 The simulator has yet to find an error.  This is both really cool and
 really terrifying: is this *really* working?  No, seriously, where are
 the bugs?  Good question.  Both the algorithm and the simulator need
 review and futher study.
 In fact, it'd be awesome if I could work with someone who has more
 TLA+ experience than I do to work on a formal specification of the
 self-management algorithm and verify its correctness.
 ** Behavior in asymmetric network partitions
 Text has moved to the [[high-level-chain-manager.pdf][Machi chain manager high level design]] document.
 * Prototype notes
 ** Mid-April 2015
 I've finished moving the chain manager plus the inner/nested
 projection code into the top-level 'src' dir of this repo.  The idea
 is working very well under simulation, more than well enough to gamble
 on for initial use.
 Stronger validation work will continue through 2015, ideally using a
 tool like TLA+.
 ** Mid-March 2015
 I've come to realize that the property that causes the nice property
 of "Were my last 2L proposals identical?" also requires that the
 proposals be *stable*.  If a participant notices, "Hey, there's
 flapping happening, so I'll propose a different projection
 P_different", then the very act of proposing P_different disrupts the
 "last 2L proposals identical" cycle the enables us to detect
 flapping.  We kill the goose that's laying our golden egg.
 I've been working on the idea of "nested" projections, namely an
 "outer" and "inner" projection.  Only the "outer projection" is used
 for cycle detection.  The "inner projection" is the same as the outer
 projection when flapping is not detected.  When flapping is detected,
 then the inner projection is one that excludes all nodes that the
 outer projection has identified as victims of asymmetric partition.
 This inner projection technique may or may not work well enough to
 use?  It would require constant flapping of the outer proposal, which
 is going to consume CPU and also chew up projection store keys with
 the flapping churn.  That churn would continue as long as an
 asymmetric partition exists.  The simplest way to cope with this would
 be to reduce proposal rates significantly, say 10x or 50x slower, to
 slow churn down to proposals from several-per-second to perhaps
 several-per-minute?
--- a/doc/cluster/migration-3to4.fig
+++ b/doc/cluster/migration-3to4.fig
@ -1,103 +0,0 @@
 #FIG 3.2  Produced by xfig version 3.2.5b
 Landscape
 Center
 Inches
 Letter  
 94.00
 Single
 -2
 1200 2
 6 7425 2700 8700 3300
 4 0 0 50 -1 2 18 0.0000 4 195 645 7425 2895 After\001
 4 0 0 50 -1 2 18 0.0000 4 255 1215 7425 3210 Migration\001
 -6
 6 7425 450 8700 1050
 4 0 0 50 -1 2 18 0.0000 4 195 780 7425 675 Before\001
 4 0 0 50 -1 2 18 0.0000 4 255 1215 7425 990 Migration\001
 -6
 6 75 1425 6900 2325
 6 4875 1425 6900 2325
 6 5400 1575 6375 2175
 4 0 0 50 -1 2 14 0.0000 4 165 390 5400 1800 Not\001
 4 0 0 50 -1 2 14 0.0000 4 225 945 5400 2100 migrated\001
 -6
 2 2 1 2 0 7 50 -1 -1 6.000 0 0 -1 0 0 5
 	 4950 1500 6825 1500 6825 2250 4950 2250 4950 1500
 -6
 6 2475 1425 4500 2325
 6 3000 1575 3975 2175
 4 0 0 50 -1 2 14 0.0000 4 165 390 3000 1800 Not\001
 4 0 0 50 -1 2 14 0.0000 4 225 945 3000 2100 migrated\001
 -6
 2 2 1 2 0 7 50 -1 -1 6.000 0 0 -1 0 0 5
 	 2550 1500 4425 1500 4425 2250 2550 2250 2550 1500
 -6
 6 75 1425 2100 2325
 6 600 1575 1575 2175
 4 0 0 50 -1 2 14 0.0000 4 165 390 600 1800 Not\001
 4 0 0 50 -1 2 14 0.0000 4 225 945 600 2100 migrated\001
 -6
 2 2 1 2 0 7 50 -1 -1 6.000 0 0 -1 0 0 5
 	 150 1500 2025 1500 2025 2250 150 2250 150 1500
 -6
 -6
 2 1 0 2 0 7 50 -1 -1 6.000 0 0 -1 1 0 2
 	1 1 3.00 60.00 120.00
 	 150 4200 150 3750
 2 1 0 2 0 7 50 -1 -1 6.000 0 0 -1 1 0 2
 	1 1 3.00 60.00 120.00
 	 3750 4200 3750 3750
 2 1 0 2 0 7 50 -1 -1 6.000 0 0 -1 1 0 2
 	1 1 3.00 60.00 120.00
 	 2025 4200 2025 3750
 2 1 0 2 0 7 50 -1 -1 6.000 0 0 -1 1 0 2
 	1 1 3.00 60.00 120.00
 	 7350 4200 7350 3750
 2 1 0 2 0 7 50 -1 -1 6.000 0 0 -1 1 0 2
 	1 1 3.00 60.00 120.00
 	 5550 4200 5550 3750
 2 2 0 3 0 7 50 -1 -1 0.000 0 0 -1 0 0 5
 	 2550 0 2550 1500 150 1500 150 0 2550 0
 2 2 0 3 0 7 50 -1 -1 0.000 0 0 -1 0 0 5
 	 4950 0 4950 1500 2550 1500 2550 0 4950 0
 2 2 0 3 0 7 50 -1 -1 0.000 0 0 -1 0 0 5
 	 7350 0 7350 1500 4950 1500 4950 0 7350 0
 2 2 0 3 0 7 50 -1 -1 0.000 0 0 -1 0 0 5
 	 150 2250 2025 2250 2025 3750 150 3750 150 2250
 2 2 0 3 0 7 50 -1 -1 0.000 0 0 -1 0 0 5
 	 4425 2250 4950 2250 4950 3750 4425 3750 4425 2250
 2 2 0 3 0 7 50 -1 -1 0.000 0 0 -1 0 0 5
 	 4950 2250 6825 2250 6825 3750 4950 3750 4950 2250
 2 2 0 3 0 7 50 -1 -1 0.000 0 0 -1 0 0 5
 	 6825 2250 7350 2250 7350 3750 6825 3750 6825 2250
 2 2 0 3 0 7 50 -1 -1 0.000 0 0 -1 0 0 5
 	 2025 2250 2550 2250 2550 3750 2025 3750 2025 2250
 2 2 0 3 0 7 50 -1 -1 0.000 0 0 -1 0 0 5
 	 2550 2250 4425 2250 4425 3750 2550 3750 2550 2250
 4 0 0 50 -1 2 18 0.0000 4 195 480 75 4500 0.00\001
 4 0 0 50 -1 2 18 0.0000 4 195 480 6825 4500 1.00\001
 4 0 0 50 -1 2 18 0.0000 4 195 480 1725 4500 0.25\001
 4 0 0 50 -1 2 18 0.0000 4 195 480 3525 4500 0.50\001
 4 0 0 50 -1 2 18 0.0000 4 195 480 5250 4500 0.75\001
 4 0 0 50 -1 2 14 0.0000 4 240 1710 450 1275 ~33% total keys\001
 4 0 0 50 -1 2 14 0.0000 4 240 1710 2925 1275 ~33% total keys\001
 4 0 0 50 -1 2 14 0.0000 4 240 1710 5250 1275 ~33% total keys\001
 4 0 0 50 -1 2 14 0.0000 4 180 495 2025 3525 ~8%\001
 4 0 0 50 -1 2 14 0.0000 4 240 1710 300 3525 ~25% total keys\001
 4 0 0 50 -1 2 14 0.0000 4 240 1710 2625 3525 ~25% total keys\001
 4 0 0 50 -1 2 14 0.0000 4 180 495 4425 3525 ~8%\001
 4 0 0 50 -1 2 14 0.0000 4 240 1710 5025 3525 ~25% total keys\001
 4 0 0 50 -1 2 14 0.0000 4 180 495 6825 3525 ~8%\001
 4 0 0 50 -1 2 24 0.0000 4 270 195 2175 3075 4\001
 4 0 0 50 -1 2 24 0.0000 4 270 195 4575 3075 4\001
 4 0 0 50 -1 2 24 0.0000 4 270 195 6975 3075 4\001
 4 0 0 50 -1 2 24 0.0000 4 270 1245 600 600 Chain1\001
 4 0 0 50 -1 2 24 0.0000 4 270 1245 3000 600 Chain2\001
 4 0 0 50 -1 2 24 0.0000 4 270 1245 5400 600 Chain3\001
 4 0 0 50 -1 2 24 0.0000 4 270 285 2100 2625 C\001
 4 0 0 50 -1 2 24 0.0000 4 270 285 4500 2625 C\001
 4 0 0 50 -1 2 24 0.0000 4 270 285 6900 2625 C\001
 4 0 0 50 -1 2 24 0.0000 4 270 1245 525 2850 Chain1\001
 4 0 0 50 -1 2 24 0.0000 4 270 1245 2925 2850 Chain2\001
 4 0 0 50 -1 2 24 0.0000 4 270 1245 5325 2850 Chain3\001
 4 0 0 50 -1 2 18 0.0000 4 240 4350 1350 4875 Cluster locator, on the unit interval\001
--- a/doc/cluster/migration-3to4.png
+++ b/doc/cluster/migration-3to4.png
--- a/doc/cluster/migration-4.png
+++ b/doc/cluster/migration-4.png
--- a/doc/cluster/name-game-sketch.org
+++ b/doc/cluster/name-game-sketch.org
@ -1,481 +0,0 @@
 -*- mode: org; -*-
 #+TITLE: Machi cluster "name game" sketch
 #+AUTHOR: Scott
 #+STARTUP: lognotedone hidestars indent showall inlineimages
 #+SEQ_TODO: TODO WORKING WAITING DONE
 #+COMMENT: M-x visual-line-mode
 #+COMMENT: Also, disable auto-fill-mode
 * 1. "Name Games" with random-slicing style consistent hashing
 Our goal: to distribute lots of files very evenly across a large
 collection of individual, small Machi chains.
 * 2. Assumptions
 ** Basic familiarity with Machi high level design and Machi's "projection"
 The [[https://github.com/basho/machi/blob/master/doc/high-level-machi.pdf][Machi high level design document]] contains all of the basic
 background assumed by the rest of this document.
 ** Analogy: "neighborhood : city :: Machi chain : Machi cluster"
 Analogy: The word "machi" in Japanese means small town or
 neighborhood.  As the Tokyo Metropolitan Area is built from many
 machis and smaller cities, therefore a big, partitioned file store can
 be built out of many small Machi chains.
 ** Familiarity with the Machi chain concept
 It's clear (I hope!) from
 the [[https://github.com/basho/machi/blob/master/doc/high-level-machi.pdf][Machi high level design document]] that Machi alone does not support
 any kind of file partitioning/distribution/sharding across multiple
 small Machi chains.  There must be another layer above a Machi chain to
 provide such partitioning services.
 Using the [[https://github.com/basho/machi/tree/master/prototype/demo-day-hack][cluster quick-and-dirty prototype]] as an
 architecture sketch, let's now assume that we have ~n~ independent Machi
 chains.  We assume that each of these chains has the same
 chain length in the nominal case, e.g. chain length of 3.
 We wish to provide partitioned/distributed file storage
 across all ~n~ chains.  We call the entire collection of ~n~ Machi
 chains a "cluster".
 We may wish to have several types of Machi clusters.  For example:
 + Chain length of 1 for "don't care if it gets lost,
  store stuff very very cheaply" data.
 + Chain length of 2 for normal data.
  + Equivalent to quorum replication's reliability with 3 copies.
 + Chain length of 7 for critical, unreplaceable data.
  + Equivalent to quorum replication's reliability with 15 copies.
 Each of these types of chains will have a name ~N~ in the
 namespace.  The role of the cluster namespace will be demonstrated in
 Section 3 below.
 ** Continue an early assumption: a Machi chain is unaware of clustering
 Let's continue with an assumption that an individual Machi chain
 inside of a cluster is completely unaware of the cluster layer.
 ** The reader is familiar with the random slicing technique
 I'd done something very-very-nearly-like-this for the Hibari database
 6 years ago.  But the Hibari technique was based on stuff I did at
 Sendmail, Inc, in 2000, so this technique feels like old news to me.
 {shrug}
 The following section provides an illustrated example.
 Very quickly, the random slicing algorithm is:
 - Hash a string onto the unit interval [0.0, 1.0)
 - Calculate h(unit interval point, Map) -> bin, where ~Map~ divides
  the unit interval into bins (or partitions or shards).
 Machi's adaptation is in step 1: we do not hash any strings.  Instead, we
 simply choose a number on the unit interval.  This number is called
 the "cluster locator number".
 As described later in this doc, Machi file names are structured into
 several components.  One component of the file name contains the cluster
 locator number; we use the number as-is for step 2 above.
 *** For more information about Random Slicing
 For a comprehensive description of random slicing, please see the
 first two papers.  For a quicker summary, please see the third
 reference.
 #+BEGIN_QUOTE
 Reliable and Randomized Data Distribution Strategies for Large Scale Storage Systems
 Alberto Miranda et al.
 http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.226.5609
                                                  (short version, HIPC'11)
 Random Slicing: Efficient and Scalable Data Placement for Large-Scale
    Storage Systems 
 Alberto Miranda et al.
 DOI: http://dx.doi.org/10.1145/2632230 (long version, ACM Transactions
                              on Storage, Vol. 10, No. 3, Article 9, 2014)
 [[http://hibari.github.io/hibari-doc/hibari-sysadmin-guide.en.html#chain-migration][Hibari Sysadmin Guide, chain migration section]].
 http://hibari.github.io/hibari-doc/hibari-sysadmin-guide.en.html#chain-migration
 #+END_QUOTE
 * 3. A simple illustration
 We use a variation of the Random Slicing hash that we will call
 ~rs_hash_with_float()~.  The Erlang-style function type is shown
 below.
 #+BEGIN_SRC erlang
 %% type specs, Erlang-style
 -spec rs_hash_with_float(float(), rs_hash:map()) -> rs_hash:chain_id().
 #+END_SRC
 I'm borrowing an illustration from the HibariDB documentation here,
 but it fits my purposes quite well.  (I am the original creator of that
 image, and also the use license is compatible.)
 #+CAPTION: Illustration of 'Map', using four Machi chains
 [[./migration-4.png]]
 Assume that we have a random slicing map called ~Map~.  This particular
 ~Map~ maps the unit interval onto 4 Machi chains:
 | Hash range  | Chain ID |
 |-------------+----------|
 | 0.00 - 0.25 | Chain1   |
 | 0.25 - 0.33 | Chain4   |
 | 0.33 - 0.58 | Chain2   |
 | 0.58 - 0.66 | Chain4   |
 | 0.66 - 0.91 | Chain3   |
 | 0.91 - 1.00 | Chain4   |
 Assume that the system chooses a cluster locator of 0.05.
 According to ~Map~, the value of
 ~rs_hash_with_float(0.05,Map) = Chain1~.
 Similarly, ~rs_hash_with_float(0.26,Map) = Chain4~.
 This example should look very similar to Hibari's technique.
 The Hibari documentation has a brief photo illustration of how random
 slicing works, see [[http://hibari.github.io/hibari-doc/hibari-sysadmin-guide.en.html#chain-migration][Hibari Sysadmin Guide, chain migration]].
 * 4. Use of the cluster namespace: name separation plus chain type
 Let us assume that the cluster framework provides several different types
 of chains:
 | Chain length | Namespace    | Consistency Mode | Comment                          |
 |--------------+--------------+------------------+----------------------------------|
 |            3 | ~normal~     | eventual         | Normal storage redundancy & cost |
 |            2 | ~reduced~    | eventual         | Reduced cost storage             |
 |            1 | ~risky~      | eventual         | Really, really cheap storage     |
 |            7 | ~paranoid~   | eventual         | Safety-critical storage          |
 |            3 | ~sequential~ | strong           | Strong consistency               |
 |--------------+--------------+------------------+----------------------------------|
 The client may want to choose the amount of redundancy that its
 application requires: normal, reduced cost, or perhaps even a single
 copy.  The cluster namespace is used by the client to signal this
 intention.
 Further, the cluster administrators may wish to use the namespace to
 provide separate storage for different applications.  Jane's
 application may use the namespace "jane-normal" and Bob's app uses
 "bob-reduced".  Administrators may definine separate groups of
 chains on separate servers to serve these two applications.
 * 5. In its lifetime, a file may be moved to different chains
 The cluster management scheme may decide that files need to migrate to
 other chains -- i.e., file that is initially created on chain ID ~X~
 has been moved to chain ID ~Y~.
 + For storage load or I/O load balancing reasons.
 + Because a chain is being decommissioned by the sysadmin.
 * 6. Floating point is not required ... it is merely convenient for explanation
 NOTE: Use of floating point terms is not required.  For example,
 integer arithmetic could be used, if using a sufficiently large
 interval to create an even & smooth distribution of hashes across the
 expected maximum number of chains.
 For example, if the maximum cluster size would be 4,000 individual
 Machi chains, then a minimum of 12 bits of integer space is required
 to assign one integer per Machi chain.  However, for load balancing
 purposes, a finer grain of (for example) 100 integers per Machi
 chain would permit file migration to move increments of
 approximately 1% of single Machi chain's storage capacity.  A
 minimum of 12+7=19 bits of hash space would be necessary to accommodate
 these constraints.
 It is likely that Machi's final implementation will choose a 24 bit
 integer (or perhaps 32 bits) to represent the cluster locator.
 * 7. Proposal: Break the opacity of Machi file names, slightly.
 Machi assigns file names based on:
 ~ClientSuppliedPrefix ++ "^" ++ SomeOpaqueFileNameSuffix~
 What if some parts of the system could peek inside of the opaque file name
 suffix in order to look at the cluster location information that we might
 code in the filename suffix?
 We break the system into parts that speak two levels of protocols,
 "high" and "low".
 + The high level protocol is used outside of the Machi cluster
 + The low level protocol is used inside of the Machi cluster
 Both protocols are based on a Protocol Buffers specification and
 implementation.  Other protocols, such as HTTP, will be added later.
 #+BEGIN_SRC
     +-----------------------+
     | Machi external client |
     | e.g. Riak CS          |
     +-----------------------+
          ^
          | Machi "high" API
          | ProtoBuffs protocol     Machi cluster boundary: outside
 .........................................................................
          |                         Machi cluster boundary: inside
          v
     +--------------------------+    +------------------------+
     | Machi "high" API service |    | Machi HTTP API service |
     +--------------------------+    +------------------------+
          ^                                       |
          |              +------------------------+
          v              v
     +------------------------+
     | Cluster bridge service |
     +------------------------+
          ^
          | Machi "low" API
          | ProtoBuffs protocol                                             
          +----------------------------------------+----+----+
          |                                        |    |    |  
          v                                        v    v    v  
       +-------------------------+              ... other chains...
       | Chain C1 (logical view) |
       |  +--------------+       |  
       |  | FLU server 1 |       |  
       |  |  +--------------+    |  
       |  +--| FLU server 2 |    |  
       |     +--------------+    |  In reality, API bridge talks directly
       +-------------------------+  to each FLU server in a chain.       
 #+END_SRC
 ** The notation we use
 - ~N~   = the cluster namespace, chosen by the client.
 - ~p~   = file prefix, chosen by the client.
 - ~L~   = the cluster locator (a number, type is implementation-dependent)
 - ~Map~ = a mapping of cluster locators to chains
 - ~T~   = the target chain ID/name
 - ~u~   = a unique opaque file name suffix, e.g. a GUID string
 - ~F~   = a Machi file name, i.e., a concatenation of ~p^L^N^u~
 ** The details: cluster file append
 0. Cluster client chooses ~N~ and ~p~ (i.e., cluster namespace and
   file prefix) and sends the append request to a Machi cluster member
   via the Protocol Buffers "high" API.
 1. Cluster bridge chooses ~T~ (i.e., target chain), based on criteria
   such as disk utilization percentage.
 2. Cluster bridge knows the cluster ~Map~ for namespace ~N~.
 3. Cluster bridge choose some cluster locator value ~L~ such that
   ~rs_hash_with_float(L,Map) = T~ (see algorithm below).
 4. Cluster bridge sends its request to chain
   ~T~: ~append_chunk(p,L,N,...) -> {ok,p^L^N^u,ByteOffset}~
 5. Cluster bridge forwards the reply tuple to the client.
 6. Client stores/uses the file name ~F = p^L^N^u~.
 ** The details: Cluster file read
 0. Cluster client sends the read request to a Machi cluster member via
   the Protocol Buffers "high" API.
 1. Cluster bridge parses the file name ~F~  to find
   the values of ~L~ and ~N~ (recall, ~F = p^L^N^u~).
 2. Cluster bridge knows the Cluster ~Map~ for type ~N~.
 3. Cluster bridge calculates ~rs_hash_with_float(L,Map) = T~
 4. Cluster bridge sends request to chain ~T~:
   ~read_chunk(F,...) ->~ ... reply
 5. Cluster bridge forwards the reply to the client.
 ** The details: calculating 'L' (the cluster locator number) to match a desired target chain
 1. We know ~Map~, the current cluster mapping for a cluster namespace ~N~.
 2. We look inside of ~Map~, and we find all of the unit interval ranges
   that map to our desired target chain ~T~.  Let's call this list
   ~MapList = [Range1=(start,end],Range2=(start,end],...]~.
 3. In our example, ~T=Chain2~.  The example ~Map~ contains a single
   unit interval range for ~Chain2~, ~[(0.33,0.58]]~.
 4. Choose a uniformly random number ~r~ on the unit interval.
 5. Calculate the cluster locator ~L~ by mapping ~r~ onto the concatenation
   of the cluster hash space range intervals in ~MapList~.  For example,
   if ~r=0.5~, then ~L = 0.33 + 0.5*(0.58-0.33) = 0.455~, which is
   exactly in the middle of the ~(0.33,0.58]~ interval.
 ** A bit more about the cluster namespaces's meaning and use
 For use by Riak CS, for example, we'd likely start with the following
 namespaces ... working our way down the list as we add new features
 and/or re-implement existing CS features.
 - "standard" = Chain length = 3, eventually consistency mode
 - "reduced" = Chain length = 2, eventually consistency mode.
 - "stanchion7" = Chain length = 7, strong consistency mode.  Perhaps
  use this namespace for the metadata required to re-implement the
  operations that are performed by today's Stanchion application.
 We want the cluster framework to:
 - provide means of creating and managing
  chains of different types, e.g., chain length, consistency mode.
 - manage the mapping of cluster namespace
  names to the chains in the system.
 - provide query functions to map a cluster
  namespace name to a cluster map,
  e.g. ~get_cluster_latest_map("reduced") -> Map{generation=7,...}~.
 * 8. File migration (a.k.a. rebalancing/reparitioning/resharding/redistribution)
 ** What is "migration"?
 This section describes Machi's file migration.  Other storage systems
 call this process as "rebalancing", "repartitioning", "resharding" or
 "redistribution".
 For Riak Core applications, it is called "handoff" and "ring resizing"
 (depending on the context).
 See also the [[http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/HdfsUserGuide.html#Balancer][Hadoop file balancer]] for another example of a data
 migration process.
 As discussed in section 5, the client can have good reason for wanting
 to have some control of the initial location of the file within the
 chain.  However, the chain manager has an ongoing interest in
 balancing resources throughout the lifetime of the file.  Disks will
 get full, hardware will change, read workload will fluctuate,
 etc etc.
 This document uses the word "migration" to describe moving data from
 one Machi chain to another chain within a cluster system.
 A simple variation of the Random Slicing hash algorithm can easily
 accommodate Machi's need to migrate files without interfering with
 availability.  Machi's migration task is much simpler due to the
 immutable nature of Machi file data.
 ** Change to Random Slicing
 The map used by the Random Slicing hash algorithm needs a few simple
 changes to make file migration straightforward.
 - Add a "generation number", a strictly increasing number (similar to
  a Machi chain's "epoch number") that reflects the history of
  changes made to the Random Slicing map
 - Use a list of Random Slicing maps instead of a single map, one map
  per chance that files may not have been migrated yet out of
  that map.
 As an example:
 #+CAPTION: Illustration of 'Map', using four Machi chains
 [[./migration-3to4.png]]
 And the new Random Slicing map for some cluster namespace ~N~ might look
 like this:
 | Generation number / Namespace | 7 / reduced |
 |-------------------------------+-------------|
 | SubMap                        | 1           |
 |-------------------------------+-------------|
 | Hash range                    | Chain ID    |
 |-------------------------------+-------------|
 | 0.00 - 0.33                   | Chain1      |
 | 0.33 - 0.66                   | Chain2      |
 | 0.66 - 1.00                   | Chain3      |
 |-------------------------------+-------------|
 | SubMap                        | 2           |
 |-------------------------------+-------------|
 | Hash range                    | Chain ID    |
 |-------------------------------+-------------|
 | 0.00 - 0.25                   | Chain1      |
 | 0.25 - 0.33                   | Chain4      |
 | 0.33 - 0.58                   | Chain2      |
 | 0.58 - 0.66                   | Chain4      |
 | 0.66 - 0.91                   | Chain3      |
 | 0.91 - 1.00                   | Chain4      |
 When a new Random Slicing map contains a single submap, then its use
 is identical to the original Random Slicing algorithm.  If the map
 contains multiple submaps, then the access rules change a bit:
 - Write operations always go to the newest/largest submap.
 - Read operations attempt to read from all unique submaps.
  - Skip searching submaps that refer to the same chain ID.
    - In this example, unit interval value 0.10 is mapped to Chain1
      by both submaps.
  - Read from newest/largest submap to oldest/smallest submap.
  - If not found in any submap, search a second time (to handle races
    with file copying between submaps).
  - If the requested data is found, optionally copy it directly to the
    newest submap.   (This is a variation of read repair (RR). RR here
    accelerates the migration process and can reduce the number of
    operations required to query servers in multiple submaps).
 The cluster manager is responsible for:
 - Managing the various generations of the cluster Random Slicing maps for
  all namespaces.
 - Distributing namespace maps to cluster bridges.
 - Managing the processes that are responsible for copying "cold" data,
  i.e., files data that is not regularly accessed, to its new submap
  location.
 - When migration of a file to its new chain is confirmed successful,
  delete it from the old chain.
 In example map #7, the cluster manager will copy files with unit interval
 assignments in ~(0.25,0.33]~, ~(0.58,0.66]~, and ~(0.91,1.00]~ from their
 old locations in chain IDs Chain1/2/3 to their new chain,
 Chain4.  When the cluster manager is satisfied that all such files have
 been copied to Chain4, then the cluster manager can create and
 distribute a new map, such as:
 | Generation number / Namespace | 8 / reduced |
 |-------------------------------+-------------|
 | SubMap                        | 1           |
 |-------------------------------+-------------|
 | Hash range                    | Chain ID    |
 |-------------------------------+-------------|
 | 0.00 - 0.25                   | Chain1      |
 | 0.25 - 0.33                   | Chain4      |
 | 0.33 - 0.58                   | Chain2      |
 | 0.58 - 0.66                   | Chain4      |
 | 0.66 - 0.91                   | Chain3      |
 | 0.91 - 1.00                   | Chain4      |
 The HibariDB system performs data migrations in almost exactly this
 manner.  However, one important
 limitation of HibariDB is not being able to
 perform more than one migration at a time.  HibariDB's data is
 mutable.  Mutation causes many problems when migrating data
 across two submaps; three or more submaps was too complex to implement
 quickly and correctly.
 Fortunately for Machi, its file data is immutable and therefore can
 easily manage many migrations in parallel, i.e., its submap list may
 be several maps long, each one for an in-progress file migration.
 * 9. Other considerations for FLU/sequencer implementations
 ** Append to existing file when possible
 The sequencer should always assign new offsets to the latest/newest
 file for any prefix, as long as all prerequisites are also true,
 - The epoch has not changed.  (In AP mode, epoch change -> mandatory
  file name suffix change.)
 - The cluster locator number is stable.
 - The latest file for prefix ~p~ is smaller than maximum file size for
  a FLU's configuration.
 The stability of the cluster locator number is an implementation detail that
 must be managed by the cluster bridge.
 Reuse of the same file is not possible if the bridge always chooses a
 different cluster locator number ~L~ or if the client always uses a unique
 file prefix ~p~.  The latter is a sign of a misbehaved client; the
 former is a poorly-implemented bridge.
 * 10. Acknowledgments
 The original source for the "migration-4.png" and "migration-3to4.png" images
 come from the [[http://hibari.github.io/hibari-doc/images/migration-3to4.png][HibariDB documentation]].
--- a/doc/dev-clone-compile.md
+++ b/doc/dev-clone-compile.md
@ -1,30 +0,0 @@
 # Clone and compile Machi
 Clone the Machi source repo and compile the source and test code.  Run
 the following commands at your login shell:
    cd /tmp
    git clone https://github.com/basho/machi.git
    cd machi
    git checkout master
    make          # or 'gmake' if GNU make uses an alternate name
 Then run the unit test suite.  This may take up to two minutes or so
 to finish.
    make test
 At the end, the test suite should report that all tests passed.  The
 actual number of tests shown in the "All `X` tests passed" line may be
 different than the example below.
    [... many lines omitted ...]
    module 'event_logger'
    module 'chain_mgr_legacy'
    =======================================================
      All 90 tests passed.
 If you had a test failure, a likely cause may be a limit on the number
 of file descriptors available to your user process.  (Recent releases
 of OS X have a limit of 1024 file descriptors, which may be too slow.)
 The output of the `limit -n` will tell you your file descriptor limit.
--- a/doc/dev-prerequisites.md
+++ b/doc/dev-prerequisites.md
@ -1,38 +0,0 @@
 ## Machi developer environment prerequisites
 1. Machi requires an 64-bit variant of UNIX: OS X, FreeBSD, Linux, or
   Solaris machine is a standard developer environment for C and C++
   applications (64-bit versions).
 2. You'll need the `git` source management utility.
 3. You'll need the 64-bit Erlang/OTP 17 runtime environment.  Please
   don't use earlier or later versions until we have a chance to fix
   the compilation warnings that versions R16B and 18 will trigger.
   Also, please verify that you are not using a 32-bit Erlang/OTP
   runtime package.
 For `git` and the Erlang runtime, please use your OS-specific
 package manager to install these.  If your package manager doesn't
 have 64-bit Erlang/OTP version 17 available, then we recommend using the
 [precompiled packages available at Erlang Solutions](https://www.erlang-solutions.com/resources/download.html).
 Also, please verify that you have enough file descriptors available to
 your user processes.  The output of `ulimit -n` should report at least
 4,000 file descriptors available.  If your limit is lower (a frequent
 problem for OS X users), please increase it to at least 4,000.
 # Using Vagrant to set up a developer environment for Machi
 The Machi source directory contains a `Vagrantfile` for creating an
 Ubuntu Linux-based virtual machine for compiling and running Machi.
 This file is in the
 [$SRC_TOP/priv/humming-consensus-demo.vagrant](../priv/humming-consensus-demo.vagrant)
 directory.
 If used as-is, the virtual machine specification is modest.
 * 1 virtual CPU
 * 512MB virtual memory
 * 768MB swap space
 * 79GB sparse virtual disk image.  After installing prerequisites and
  compiling Machi, the root file system uses approximately 2.7 GBytes.
--- a/doc/flu-and-chain-lifecycle.org
+++ b/doc/flu-and-chain-lifecycle.org
@ -1,617 +0,0 @@
 FLU and Chain Life Cycle Management -*- mode: org; -*-
 #+STARTUP: lognotedone hidestars indent showall inlineimages
 #+COMMENT: To generate the outline section: egrep '^\*[*]* ' doc/flu-and-chain-lifecycle.org | egrep -v '^\* Outline' | sed -e 's/^\*\*\* /    + /' -e 's/^\*\* /  + /' -e 's/^\* /+ /'
 * FLU and Chain Life Cycle Management
 In an ideal world, we (the Machi development team) would have a full
 vision of how Machi would be managed, down to the last detail of
 beautiful CLI character and network protocol bit.  Our vision isn't
 complete yet, so we are working one small step at a time.
 * Outline
 + FLU and Chain Life Cycle Management
 + Terminology review
  + Terminology: Machi run-time components/services/thingies
  + Terminology: Machi chain data structures
  + Terminology: Machi cluster data structures
 + Overview of administrative life cycles
  + Cluster administrative life cycle
  + Chain administrative life cycle
  + FLU server administrative life cycle
 + Quick admin: declarative management of Machi FLU and chain life cycles
  + Quick admin uses the "rc.d" config scheme for life cycle management
  + Quick admin's declarative "language": an Erlang-flavored AST
    + Term 'host': define a new host for FLU services
    + Term 'flu': define a new FLU
    + Term 'chain': define or reconfigure a chain
  + Executing quick admin AST files via the 'machi-admin' utility
    + Checking the syntax of an AST file
    + Executing an AST file
  + Using quick admin to manage multiple machines
 + The "rc.d" style configuration file scheme
  + Riak had a similar configuration file editing problem (and its solution)
  + Machi's "rc.d" file scheme.
  + FLU life cycle management using "rc.d" style files
    + The key configuration components of a FLU
  + Chain life cycle management using "rc.d" style files
    + The key configuration components of a chain
 * Terminology review
 ** Terminology: Machi run-time components/services/thingies
 + FLU: a basic Machi server, responsible for managing a collection of
  files.
 + Chain: a small collection of FLUs that maintain replicas of the same
  collection of files.  A chain is usually small, 1-3 servers, where
  more than 3 would be used only in cases when availability of
  certain data is critical despite failures of several machines.
  + The length of a chain is directly proportional to its
    replication factor, e.g., a chain length=3 will maintain
    (nominally) 3 replicas of each file.
  + To maintain file availability when ~F~ failures have occurred, a
    chain must be at least ~F+1~ members long.  (In comparison, the
    quorum replication technique requires ~2F+1~ members in the
    general case.)
 + Cluster: A collection of Machi chains that are used to store files
  in a horizontally partitioned/sharded/distributed manner.
 ** Terminology: Machi data structures
 + Projection: used to define a single chain: the chain's consistency
  mode (strong or eventual consistency), all members (from an
  administrative point of view), all active members (from a runtime,
  automatically-managed point of view), repairing/file-syncing members
  (also runtime, auto-managed), and so on
 + Epoch: A version number of a projection.  The epoch number is used
  by both clients & servers to manage transitions from one projection
  to another, e.g., when the chain is temporarily shortened by the
  failure of a member FLU server.
 ** Terminology: Machi cluster data structures
 + Namespace: A collection of human-friendly names that are mapped to
  groups of Machi chains that provide the same type of storage
  service: consistency mode, replication policy, etc.
  + A single namespace name, e.g. ~normal-ec~, is paired with a single
    cluster map (see below).
  + Example: ~normal-ec~ might be a collection of Machi chains in
    eventually-consistent mode that are of length=3.
  + Example: ~risky-ec~ might be a collection of Machi chains in
    eventually-consistent mode that are of length=1.
  + Example: ~mgmt-critical~ might be a collection of Machi chains in
    strongly-consistent mode that are of length=7.
 + Cluster map: Encodes the rules which partition/shard/distribute
  the files stored in a particular namespace across a group of chains
  that collectively store the namespace's files.
 + Chain weight: A value assigned to each chain within a cluster map
  structure that defines the relative storage capacity of a chain
  within the namespace.  For example, a chain weight=150 has 50% more
  capacity than a chain weight=100.
 + Cluster map epoch: The version number assigned to a cluster map.
 * Overview of administrative life cycles
 ** Cluster administrative life cycle
 + Cluster is first created
 + Adds namespaces (e.g. consistency policy + chain length policy) to
  the cluster
 + Chains are added to/removed from a namespace to increase/decrease the
  namespace's storage capacity.
 + Adjust chain weights within a namespace, e.g., to shift files
  within the namespace to chains with greater storage capacity
  resources and/or runtime I/O resources.
 A cluster "file migration" is the process of moving files from one
 namespace member chain to another for purposes of shifting &
 re-balancing storage capacity and/or runtime I/O capacity.
 ** Chain administrative life cycle
 + A chain is created with an initial FLU membership list.
 + Chain may be administratively modified zero or more times to
  add/remove member FLU servers.
 + A chain may be decommissioned.
 See also: http://basho.github.io/machi/edoc/machi_lifecycle_mgr.html
 ** FLU server administrative life cycle
 + A FLU is created after an administrator chooses the FLU's runtime
  location is selected by the administrator: which machine/virtual
  machine, IP address and TCP port allocation, etc.
 + An unassigned FLU may be added to a chain by chain administrative
  policy.
 + A FLU that is assigned to a chain may be removed from that chain by
  chain administrative policy.
  + In the current implementation, the FLU's Erlang processes will be
    halted.  Then the FLU's data and metadata files will be moved to
    another area of the disk for safekeeping.  Later, a "garbage
    collection" process can be used for reclaiming disk space used by
    halted FLU servers.
 See also: http://basho.github.io/machi/edoc/machi_lifecycle_mgr.html
 * Quick admin: declarative management of Machi FLU and chain life cycles
 The "quick admin" scheme is a temporary (?) tool for managing Machi
 FLU server and chain life cycles in a declarative manner.  The API is
 described in this section.
 ** Quick admin uses the "rc.d" config scheme for life cycle management
 As described at the top of
 http://basho.github.io/machi/edoc/machi_lifecycle_mgr.html, the "rc.d"
 config files do not manage "policy".  "Policy" is doing the right
 thing with a Machi cluster from a systems administrator's
 point of view.  The "rc.d" config files can only implement decisions
 made according to policy.
 The "quick admin" tool is a first attempt at automating policy
 decisions in a safe way (we hope) that is also easy to implement (we
 hope) with a variety of systems management tools, e.g. Chef, Puppet,
 Ansible, Saltstack, or plain-old-human-at-a-keyboard.
 ** Quick admin's declarative "language": an Erlang-flavored AST
 The "language" that an administrator uses to express desired policy
 changes is not (yet) a true language.  As a quick implementation hack,
 the current language is an Erlang-flavored abstract syntax tree
 (AST).  The tree isn't very deep, either, frequently just one
 element tall.  (Not much of a tree, is it?)
 There are three terms in the language currently:
 + ~host~, define a new host that can execute FLU servers
 + ~flu~, define a new FLU
 + ~chain~, define a new chain or re-configure an existing chain with
  the same name
 *** Term 'host': define a new host for FLU services
 In this context, a host is a machine, virtual machine, or container
 that can execute the Machi application and can therefore provide FLU
 services, i.e. file service, Humming Consensus management.
 Two formats may be used to define a new host:
 #+BEGIN_SRC
 {host, Name, Props}.
 {host, Name, AdminI, ClientI, Props}.
 #+END_SRC
 The shorter tuple is shorthand notation for the latter.  If the
 shorthand form is used, then it will be converted automatically to the
 long form as:
 #+BEGIN_SRC
 {host, Name, AdminI=Name, ClientI=Name, Props}.
 #+END_SRC
 Type information, description, and restrictions:
 + ~Name::string()~ The ~Name~ attribute must be unique.  Note that it
  is possible to define two different hosts, one using a DNS hostname
  and one using an IP address.  The user must avoid this
  double-definition because it is not enforced by quick admin.
  + The ~Name~ field is used for cross-reference purposes with other
    terms, e.g., ~flu~ and ~chain~.
  + There is no syntax yet for removing a host definition.
 + ~AdminI::string()~ A DNS hostname or IP address for cluster
  administration purposes, e.g. SSH access.
  + This field is unused at the present time.
 + ~ClientI::string()~ A DNS hostname or IP address for Machi's client
  protocol access, e.g., Protocol Buffers network API service.
  + This field is unused at the present time.
 + ~props::proplist()~ is an Erlang-style property list for specifying
  additional configuration options, debugging information, sysadmin
  comments, etc.
 + A full-featured admin tool should also include managing several
  other aspects of configuration related to a "host".  For example,
  for any single IP address, quick admin assumes that there will be
  exactly one Erlang VM that is running the Machi application.  Of
  course, it is possible to have dozens of Erlang VMs on the same
  (let's assume for clarity) hardware machine and all running Machi
  ... but there are additional aspects of such a machine that quick
  admin does not account for
  + multiple IP addresses per machine
  + multiple Machi package installation paths
  + multiple Machi config files (e.g. cuttlefish config, ~etc.conf~,
    ~vm.args~)
  + multiple data directories/file system mount points
    + This is also a management problem for quick admin for a single
      Machi package on a machine to take advantage of bulk data
      storage using multiple multiple file system mount points.
  + multiple Erlang VM host names, required for distributed Erlang,
    which is used for communication with ~machi~ and ~machi-admin~
    command line utilities.
  + and others....
 *** Term 'flu': define a new FLU
 A new FLU is defined relative to a previously-defined ~host~ entities;
 an exception will be thrown if the ~host~ cannot be cross-referenced.
 #+BEGIN_SRC
 {flu, Name, HostName, Port, Props}
 #+END_SRC
 Type information, description, and restrictions:
 + ~Name::atom()~ The name of the FLU, as a human-friendly name and
  also for internal management use; please note the ~atom()~ type.
  This name must be unique.
  + The ~Name~ field is used for cross-reference purposes with the
    ~chain~ term.
  + There is no syntax yet for removing a FLU definition.
 + ~Hostname::string()~ The cross-reference name of the ~host~ that
  this FLU should run on.
 + ~Port::non_neg_integer()~ The TCP port used by this FLU server's
  Protocol Buffers network API listener service
 + ~props::proplist()~ is an Erlang-style property list for specifying
  additional configuration options, debugging information, sysadmin
  comments, etc.
 *** Term 'chain': define or reconfigure a chain
 A chain is defined relative to zero or more previously-defined ~flu~
 entities; an exception will be thrown if any ~flu~ cannot be
 cross-referenced.
 Two formats may be used to define/reconfigure a chain:
 #+BEGIN_SRC
 {chain, Name, FullList, Props}.
 {chain, Name, CMode, FullList, Witnesses, Props}.
 #+END_SRC
 The shorter tuple is shorthand notation for the latter.  If the
 shorthand form is used, then it will be converted automatically to the
 long form as:
 #+BEGIN_SRC
 {chain, Name, ap_mode, FullList, [], Props}.
 #+END_SRC
 Type information, description, and restrictions:
 + ~Name::atom()~ The name of the chain, as a human-friendly name and
  also for internal management use; please note the ~atom()~ type.
  This name must be unique.
  + There is no syntax yet for removing a chain definition.
 + ~CMode::'ap_mode'|'cp_mode'~ Defines the consistency mode of the
  chain, either eventual consistency or strong consistency,
  respectively.
  + A chain cannot change consistency mode, e.g., from
    strong~->~eventual consistency.
 + ~FullList::list(atom())~ Specifies the list of full-service FLU
  servers, i.e. servers that provide file data & metadata services as
  well as Humming Consensus.  Each atom in the list must
  cross-reference with a previously defined ~chain~; an exception will
  be thrown if any ~flu~ cannot be cross-referenced.
 + ~Witnesses::list(atom())~ Specifies the list of witness-only
  servers, i.e. servers that only participate in Humming Consensus.
  Each atom in the list must cross-reference with a previously defined
  ~chain~; an exception will be thrown if any ~flu~ cannot be
  cross-referenced.
  + This list must be empty for eventual consistency chains.
 + ~props::proplist()~ is an Erlang-style property list for specifying
  additional configuration options, debugging information, sysadmin
  comments, etc.
 + If this term specifies a new ~chain~ name, then all of the member
  FLU servers (full & witness types) will be bootstrapped to a
  starting configuration.
 + If this term specifies a previously-defined ~chain~ name, then all
  of the member FLU servers (full & witness types, respectively) will
  be adjusted to add or remove members, as appropriate.
  + Any FLU servers added to either list must not be assigned to any
    other chain, or they must be a member of this specific chain.
  + Any FLU servers removed from either list will be halted.
    (See the "FLU server administrative life cycle" section above.)
 ** Executing quick admin AST files via the 'machi-admin' utility
 Examples of quick admin AST files can be found in the
 ~priv/quick-admin/examples~ directory.  Below is an example that will
 define a new host ( ~"localhost"~ ), three new FLU servers ( ~f1~ & ~f2~
 and ~f3~ ), and an eventually consistent chain ( ~c1~ ) that uses the new
 FLU servers:
 #+BEGIN_SRC
 {host, "localhost", []}.
 {flu,f1,"localhost",20401,[]}.
 {flu,f2,"localhost",20402,[]}.
 {flu,f3,"localhost",20403,[]}.
 {chain,c1,[f1,f2,f3],[]}.
 #+END_SRC
 *** Checking the syntax of an AST file
 Given an AST config file, ~/path/to/ast/file~, its basic syntax and
 correctness can be checked without executing it.
 #+BEGIN_SRC
 ./rel/machi/bin/machi-admin quick-admin-check /path/to/ast/file
 #+END_SRC
 + The utility will exit with status zero and output ~ok~ if the syntax
  and proposed configuration appears to be correct.
 + If there is an error, the utility will exit with status one, and an
  error message will be printed.
 *** Executing an AST file
 Given an AST config file, ~/path/to/ast/file~, it can be executed
 using the command:
 #+BEGIN_SRC
 ./rel/machi/bin/machi-admin quick-admin-apply /path/to/ast/file RelativeHost
 #+END_SRC
 ... where the last argument, ~RelativeHost~, should be the exact
 spelling of one of the previously defined AST ~host~ entities,
 *and also* is the same host that the ~machi-admin~ utility is being
 executed on.
 Restrictions and warnings:
 + This is alpha quality software.
 + There is no "undo".
  + Of course there is, but you need to resort to doing things like
    using ~machi attach~ to attach to the server's CLI to then execute
    magic Erlang incantations to stop FLUs, unconfigure chains, etc.
    + Oh, and delete some files with magic paths, also.
 ** Using quick admin to manage multiple machines
 A quick sketch follows:
 1. Create the AST file to specify all of the changes that you wish to
   make to all hosts, FLUs, and/or chains, e.g., ~/tmp/ast.txt~.
 2. Check the basic syntax with the ~quick-admin-check~ argument to
   ~machi-admin~. 
 3. If the syntax is good, then copy ~/tmp/ast.txt~ to all hosts in the
   cluster, using the same path, ~/tmp/ast.txt~.
 4. For each machine in the cluster, run:
 #+BEGIN_SRC
 ./rel/machi/bin/machi-admin quick-admin-apply /tmp/ast.txt RelativeHost
 #+END_SRC
 ... where RelativeHost is the AST ~host~ name of the machine that you
 are executing the ~machi-admin~ command on.  The command should be
 successful, with exit status 0 and outputting the string ~ok~.
 Finally, for each machine in the cluster, a listing of all files in
 the directory ~rel/machi/etc/quick-admin-archive~ should show exactly
 the same files, one for each time that ~quick-admin-apply~ has been
 run successfully on that machine.
 * The "rc.d" style configuration file scheme
 This configuration scheme is inspired by BSD UNIX's ~init(8)~ process
 manager's configuration style, called "rc.d" after the name of the
 directory where these files are stored, ~/etc/rc.d~.  The ~init~
 process is responsible for (among other things) starting UNIX
 processes at machine boot time and stopping them when the machine is
 shut down.
 The original scheme used by ~init~ to start processes at boot time was
 a single Bourne shell script called ~/etc/rc~.  When a new software
 package was installed that required a daemon to be started at boot
 time, text was added to the ~/etc/rc~ file.  Uninstalling packages was
 much trickier, because it meant removing lines from a file that
 *is a computer program (run by the Bourne shell, a Turing-complete
 programming language)*.  Error-free editing of the ~/etc/rc~ script
 was impossible in all cases.
 Later, ~init~'s configuration was split into a few master Bourne shell
 scripts and a subdirectory, ~/etc/rc.d~.  The subdirectory contained
 shell scripts that were responsible for boot time starting of a single
 daemon or service, e.g. NFS or an HTTP server.  When a new software
 package was added, a new file was added to the ~rc.d~ subdirectory.
 When a package was removed, the corresponding file in ~rc.d~ was
 removed.  With this simple scheme, addition & removal of boot time
 scripts was vastly simplified.
 ** Riak had a similar configuration file editing problem (and its solution)
 Another software product from Basho Technologies, Riak, had a similar
 configuration file editing problem.  One file in particular,
 ~app.config~, had a syntax that made it difficult both for human
 systems administrators and also computer programs to edit the file in
 a syntactically correct manner.
 Later releases of Riak switched to an alternative configuration file
 format, one inspired by the BSD UNIX ~sysctl(8)~ utility and
 ~sysctl.conf(5)~ file syntax.  The ~sysctl.conf~ format is much easier
 to manage by computer programs to add items.  Removing items is not
 100% simple, however: the correct lines must be identified and then
 removed (e.g. with Perl or a text editor or combination of ~grep -v~
 and ~mv~), but removing any comment lines that "belong" to the removed
 config item(s) is not any easy for a 1-line shell script to do 100%
 correctly.
 Machi will use the ~sysctl.conf~ style configuration for some
 application configuration variables.  However, adding & removing FLUs
 and chains will be managed using the "rc.d" style because of the
 "rc.d" scheme's simplicity and tolerance of mistakes by administrators
 (human or computer).
 ** Machi's "rc.d" file scheme.
 Machi will use a single subdirectory that will contain configuration
 files for some life cycle management task, e.g. a single FLU or a
 single chain.
 The contents of the file should be a single Erlang term, serialized in
 ASCII form as Erlang source code statement, i.e. a single Erlang term
 ~T~ that is formatted by ~io:format("~w.",[T]).~.  This file must be
 parseable by the Erlang function ~file:consult()~.
 Later versions of Machi may change the file format to be more familiar
 to administrators who are unaccustomed to Erlang language syntax.
 ** FLU life cycle management using "rc.d" style files
 *** The key configuration components of a FLU
 1. The machine (or virtual machine) to run it on.
 2. The Machi software package's artifacts to execute.
 3. The disk device(s) used to store Machi file data & metadata, "rc.d"
   style config files, etc.
 4. The name, IP address and TCP port assigned to the FLU service.
 5. Its chain assignment.
 Notes:
 + Items 1-3 are currently outside of the scope of this life cycle
  document.  We assume that human administrators know how to do these
  things.
 + Item 4's properties are explicitly managed by a FLU-defining "rc.d"
  style config file.
 + Item 5 is managed by the chain life cycle management system.
 Here is an example of a properly formatted FLU config file:
 #+BEGIN_SRC
 {p_srvr,f1,machi_flu1_client,"192.168.72.23",20401,[]}.
 #+END_SRC
 ... which corresponds to the following Erlang record definition:
 #+BEGIN_SRC
 -record(p_srvr, {
          name            :: atom(),
          proto_mod = 'machi_flu1_client' :: atom(), % Module name
          address         :: term(), % Protocol-specific
          port            :: term(), % Protocol-specific
          props = []      :: list()  % proplist for other related info
         }).
 #+END_SRC
 + ~name~ is ~f1~.  This is name of the FLU.  This name should be
  unique over the lifetime of the administrative domain and thus
  managed by external policy.  This name must be the same as the name
  of the config file that defines the FLU.
 + ~proto_mod~ is used for internal management purposes and should be
  considered a mandatory constant.
 + ~address~ is "192.168.72.23".  The DNS hostname or IP address used
  by other servers to communicate with this FLU. This must be a valid
  IP address, previously assigned to this machine/VM using the
  appropriate operating system-specific procedure.
 + ~port~ is TCP port 20401.  The TCP port number that the FLU listens
  to for incoming Protocol Buffers-serialized communication.  This TCP
  port must not be in use (now or in the future) by another Machi FLU
  or any other process running on this machine/VM.
 + ~props~ is an Erlang-style property list for specifying additional
  configuration options, debugging information, sysadmin comments,
  etc.
 ** Chain life cycle management using "rc.d" style files
 Unlike FLUs, chains have a self-management aspect that makes a chain
 life cycle different from a single FLU server.  Machi's chains are
 self-managing, via Humming Consensus; see the
 https://github.com/basho/machi/tree/master/doc/ directory for much
 more detail about Humming Consensus.  After FLUs have received their
 initial chain configuration for Humming Consensus, the FLUs will
 manage the chain (and each other) by themselves.
 However, Humming Consensus does not handle three chain management
 problems:
 1. Specifying the very first chain configuration,
 2. Altering the membership of the chain (i.e. adding/removing FLUs
   from the chain),
 3. Stopping the chain permanently.
 A chain "rc.d" file will only be used to bootstrap a newly-defined FLU
 server.  It's like a piece of glue information to introduce the new
 FLU to the Humming Consensus group that is managing the chain's
 dynamic state (e.g. which members are up or down).  In all other
 respects, chain config files are ignored by life cycle management code.
 However, to mimic the life cycle of the FLU server's "rc.d" config
 files, a chain "rc.d" files is not deleted until the chain has been
 decommissioned (i.e. defined with length=0).
 *** The key configuration components of a chain
 1. The name of the chain.
 2. Consistency mode: eventually consistent or strongly consistent.
 3. The membership list of all FLU servers in the chain.
   + Remember, all servers in a single chain will manage full replicas
     of the same collection of Machi files.
 4. If the chain is defined to use strongly consistent mode, then a
   list of "witness servers" may also be defined.  See the
   [https://github.com/basho/machi/tree/master/doc/] documentation for
   more information on witness servers.
   + The witness list must be empty for all chains in eventual
     consistency mode. 
 Here is an example of a properly formatted chain config file:
 #+BEGIN_SRC
 {chain_def_v1,c1,ap_mode,
              [{p_srvr,f1,machi_flu1_client,"localhost",20401,[]},
               {p_srvr,f2,machi_flu1_client,"localhost",20402,[]},
               {p_srvr,f3,machi_flu1_client,"localhost",20403,[]}],
              [],[],[],
              [f1,f2,f3],
              [],[]}.
 #+END_SRC
 ... which corresponds to the following Erlang record definition:
 #+BEGIN_SRC
 -record(chain_def_v1, {
          name            :: atom(),         % chain name
          mode            :: 'ap_mode' | 'cp_mode',
          full = []       :: [p_srvr()],
          witnesses = []  :: [p_srvr()],
          old_full = []   :: [atom()], % guard against some races
          old_witnesses=[] :: [atom()], % guard against some races
          local_run = []  :: [atom()], % must be tailored to each machine!
          local_stop = [] :: [atom()], % must be tailored to each machine!
          props = []      :: list()   % proplist for other related info
         }).
 #+END_SRC
 + ~name~ is ~c1~, the name of the chain.  This name should be unique
  over the lifetime of the administrative domain and thus managed by
  external policy.  This name must be the same as the name of the
  config file that defines the chain.
 + ~mode~ is ~ap_mode~, an internal code symbol for eventual
  consistency mode.
 + ~full~ is a list of Erlang ~#p_srvr{}~ records for full-service
  members of the chain, i.e., providing Machi file data & metadata
  storage services.
 + ~witnesses~ is a list of Erlang ~#p_srvr{}~ records for witness-only
  FLU servers, i.e., providing only Humming Consensus service.
 + The next four fields are used for internal management only.
 + ~props~ is an Erlang-style property list for specifying additional
  configuration options, debugging information, sysadmin comments,
  etc.
--- a/doc/high-level-chain-mgr.pdf
+++ b/doc/high-level-chain-mgr.pdf
--- a/doc/high-level-machi.pdf
+++ b/doc/high-level-machi.pdf
--- a/doc/humming-consensus-demo.md
+++ b/doc/humming-consensus-demo.md
@ -1,372 +0,0 @@
 # Table of contents
 * [Hands-on experiments with Machi and Humming Consensus](#hands-on)
 * [Using the network partition simulator and convergence demo test code](#partition-simulator)
 <a name="hands-on">
 # Hands-on experiments with Machi and Humming Consensus
 ## Prerequisites
 Please refer to the
 [Machi development environment prerequisites doc](./dev-prerequisites.md)
 for Machi developer environment prerequisites.
 If you do not have an Erlang/OTP runtime system available, but you do
 have [the Vagrant virtual machine](https://www.vagrantup.com/) manager
 available, then please refer to the instructions in the prerequisites
 doc for using Vagrant.
 <a name="clone-compile">
 ## Clone and compile the code
 Please see the
 [Machi 'clone and compile' doc](./dev-clone-compile.md)
 for the short list of steps required to fetch the Machi source code
 from GitHub and to compile &amp; test Machi.
 ## Running three Machi instances on a single machine
 All of the commands that should be run at your login shell (e.g. Bash,
 c-shell) can be cut-and-pasted from this document directly to your
 login shell prompt.
 Run the following command:
    make stagedevrel
 This will create a directory structure like this:
          |-dev1-|... stand-alone Machi app + subdirectories
    |-dev-|-dev2-|... stand-alone Machi app + directories
          |-dev3-|... stand-alone Machi app + directories
 Each of the `dev/dev1`, `dev/dev2`, and `dev/dev3` are stand-alone
 application instances of Machi and can be run independently of each
 other on the same machine.  This demo will use all three.
 The lifecycle management utilities for Machi are a bit immature,
 currently.  They assume that each Machi server runs on a host with a
 unique hostname -- there is no flexibility built-in yet to easily run
 multiple Machi instances on the same machine.  To continue with the
 demo, we need to use `sudo` or `su` to obtain superuser privileges to
 edit the `/etc/hosts` file.
 Please add the following line to `/etc/hosts`, using this command:
    sudo sh -c 'echo "127.0.0.1 machi1 machi2 machi3" >> /etc/hosts'
 Next, we will use a shell script to finish setting up our cluster.  It
 will do the following for us:
 * Verify that the new line that was added to `/etc/hosts` is correct.
 * Modify the `etc/app.config` files to configure the Humming Consensus
  chain manager's actions logged to the `log/console.log` file.
 * Start the three application instances.
 * Verify that the three instances are running correctly.
 * Configure a single chain, with one FLU server per application
  instance.
 Please run this script using this command:
    ./priv/humming-consensus-demo.setup.sh
 If the output looks like this (and exits with status zero), then the
 script was successful.
    Step: Verify that the required entries in /etc/hosts are present
    Step: add a verbose logging option to app.config
    Step: start three three Machi application instances
    pong
    pong
    pong
    Step: configure one chain to start a Humming Consensus group with three members
    Result: ok
    Result: ok
    Result: ok
 We have now created a single replica chain, called `c1`, that has
 three file servers participating in the chain.  Thanks to the
 hostnames that we added to `/etc/hosts`, all are using the localhost
 network interface.
    | App instance | Pseudo   | FLU name | TCP port |
    | directory    | Hostname |          |   number |
    |--------------+----------+----------+----------|
    | dev1         | machi1   | flu1     |    20401 |
    | dev2         | machi2   | flu2     |    20402 |
    | dev3         | machi3   | flu3     |    20403 |
 The log files for each application instance can be found in the
 `./dev/devN/log/console.log` file, where the `N` is the instance
 number: 1, 2, or 3.
 ## Understanding the chain manager's log file output
 After running the `./priv/humming-consensus-demo.setup.sh` script,
 let's look at the last few lines of the `./dev/dev1/log/console.log`
 log file for Erlang VM process #1.
    2016-03-09 10:16:35.676 [info] <0.105.0>@machi_lifecycle_mgr:process_pending_flu:422 Started FLU f1 with supervisor pid <0.128.0>
    2016-03-09 10:16:35.676 [info] <0.105.0>@machi_lifecycle_mgr:move_to_flu_config:540 Creating FLU config file f1
    2016-03-09 10:16:35.790 [info] <0.105.0>@machi_lifecycle_mgr:bootstrap_chain2:312 Configured chain c1 via FLU f1 to mode=ap_mode all=[f1,f2,f3] witnesses=[]
    2016-03-09 10:16:35.790 [info] <0.105.0>@machi_lifecycle_mgr:move_to_chain_config:546 Creating chain config file c1
    2016-03-09 10:16:44.139 [info] <0.132.0> CONFIRM epoch 1141 <<155,42,7,221>> upi [] rep [] auth f1 by f1
    2016-03-09 10:16:44.271 [info] <0.132.0> CONFIRM epoch 1148 <<57,213,154,16>> upi [f1] rep [] auth f1 by f1
    2016-03-09 10:16:44.864 [info] <0.132.0> CONFIRM epoch 1151 <<239,29,39,70>> upi [f1] rep [f3] auth f1 by f1
    2016-03-09 10:16:45.235 [info] <0.132.0> CONFIRM epoch 1152 <<173,17,66,225>> upi [f2] rep [f1,f3] auth f2 by f1
    2016-03-09 10:16:47.343 [info] <0.132.0> CONFIRM epoch 1154 <<154,231,224,149>> upi [f2,f1,f3] rep [] auth f2 by f1
 Let's pick apart some of these lines.  We have started all three
 servers at about the same time.  We see some race conditions happen,
 and some jostling and readjustment happens pretty quickly in the first
 few seconds.
 * `Started FLU f1 with supervisor pid <0.128.0>`
  * This VM, #1,
  started a FLU (Machi data server) with the name `f1`.  In the Erlang
  process supervisor hierarchy, the process ID of the top supervisor
  is `<0.128.0>`.
 * `Configured chain c1 via FLU f1 to mode=ap_mode all=[f1,f2,f3] witnesses=[]`
  * A bootstrap configuration for a chain named `c1` has been created.
  * The FLUs/data servers that are eligible for participation in the
    chain have names `f1`, `f2`, and `f3`.
  * The chain will operate in eventual consistency mode (`ap_mode`)
  * The witness server list is empty.  Witness servers are never used
    in eventual consistency mode.
 * `CONFIRM epoch 1141 <<155,42,7,221>> upi [] rep [] auth f1 by f1`
  * All participants in epoch 1141 are unanimous in adopting epoch
    1141's projection.  All active membership lists are empty, so
    there is no functional chain replication yet, at least as far as
    server `f1` knows
  * The epoch's abbreviated checksum is `<<155,42,7,221>>`.
  * The UPI list, i.e. the replicas whose data is 100% in sync is
    `[]`, the empty list.  (UPI = Update Propagation Invariant)
  * The list of servers that are under data repair (`rep`) is also
    empty, `[]`.
  * This projection was authored by server `f1`.
  * The log message was generated by server `f1`.
 * `CONFIRM epoch 1148 <<57,213,154,16>> upi [f1] rep [] auth f1 by f1`
  * Now the server `f1` has created a chain of length 1, `[f1]`.
  * Chain repair/file re-sync is not required when the UPI server list
    changes from length 0 -> 1.
 * `CONFIRM epoch 1151 <<239,29,39,70>> upi [f1] rep [f3] auth f1 by f1`
  * Server `f1` has noticed that server `f3` is alive.  Apparently it
    has not yet noticed that server `f2` is also running.
  * Server `f3` is in the repair list.
 * `CONFIRM epoch 1152 <<173,17,66,225>> upi [f2] rep [f1,f3] auth f2 by f1`
  * Server `f2` is apparently now aware that all three servers are running.
  * The previous configuration used by `f2` was `upi [f2]`, i.e., `f2`
    was running in a chain of one.  `f2` noticed that `f1` and `f3`
    were now available and has started adding them to the chain.
  * All new servers are always added to the tail of the chain in the
    repair list.
  * In eventual consistency mode, a UPI change like this is OK.
    * When performing a read, a client must read from both tail of the
      UPI list and also from all repairing servers.
    * When performing a write, the client writes to both the UPI
      server list and also the repairing list, in that order.
      * I.e., the client concatenates both lists,
      `UPI ++ Repairing`, for its chain configuration for the write.
  * Server `f2` will trigger file repair/re-sync shortly.
    * The waiting time for starting repair has been configured to be
      extremely short, 1 second.  The default waiting time is 10
      seconds, in case Humming Consensus remains unstable.
 * `CONFIRM epoch 1154 <<154,231,224,149>> upi [f2,f1,f3] rep [] auth f2 by f1`
  * File repair/re-sync has finished.  All file data on all servers
    are now in sync.
  * The UPI/in-sync part of the chain is now `[f2,f1,f3]`, and there
    are no servers under repair.
 ## Let's create some failures
 Here are some suggestions for creating failures.
 * Use the `./dev/devN/bin/machi stop` and `./dev/devN/bin/machi start`
  commands to stop & start VM #`N`.
 * Stop a VM abnormally by using `kill`.  The OS process name to look
  for is `beam.smp`.
 * Suspend and resume a VM, using the `SIGSTOP` and `SIGCONT` signals.
  * E.g. `kill -STOP 9823` and `kill -CONT 9823`
 The network partition simulator is not (yet) available when running
 Machi in this mode.  Please see the next section for instructions on
 how to use partition simulator.
 <a name="partition-simulator">
 # Using the network partition simulator and convergence demo test code
 This is the demo code mentioned in the presentation that Scott Lystig
 Fritchie gave at the
 [RICON 2015 conference](http://ricon.io).
 * [slides (PDF format)](http://ricon.io/speakers/slides/Scott_Fritchie_Ricon_2015.pdf)
 * [video](https://www.youtube.com/watch?v=yR5kHL1bu1Q)
 ## A complete example of all input and output
 If you don't have an Erlang/OTP 17 runtime environment available,
 please see this file for full input and output of a strong consistency
 length=3 chain test:
 https://gist.github.com/slfritchie/8352efc88cc18e62c72c
 This file contains all commands input and all simulator output from a
 sample run of the simulator.
 To help interpret the output of the test, please skip ahead to the
 "The test output is very verbose" section.
 ## Prerequisites
 If you don't have `git` and/or the Erlang 17 runtime system available
 on your OS X, FreeBSD, Linux, or Solaris machine, please take a look
 at the [Prerequisites section](#prerequisites) first.  When you have
 installed the prerequisite software, please return back here.
 ## Clone and compile the code
 Please briefly visit the [Clone and compile the code](#clone-compile)
 section.  When finished, please return back here.
 ## Run an interactive Erlang CLI shell
 Run the following command at your login shell:
    erl -pz .eunit ebin deps/*/ebin
 If you are using Erlang/OTP version 17, you should see some CLI output
 that looks like this:
    Erlang/OTP 17 [erts-6.4] [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]
    Eshell V6.4  (abort with ^G)
    1>
 ## The test output is very verbose ... what are the important parts?
 The output of the Erlang command
 `machi_chain_manager1_converge_demo:help()` will display the following
 guide to the output of the tests.
    A visualization of the convergence behavior of the chain self-management
    algorithm for Machi.
      1. Set up some server and chain manager pairs.
      2. Create a number of different network partition scenarios, where
         (simulated) partitions may be symmetric or asymmetric.  Then stop changing
         the partitions and keep the simulated network stable (and perhaps broken).
      3. Run a number of iterations of the algorithm in parallel by poking each
         of the manager processes on a random'ish basis.
      4. Afterward, fetch the chain transition changes made by each FLU and
         verify that no transition was unsafe.
    During the iteration periods, the following is a cheatsheet for the output.
    See the internal source for interpreting the rest of the output.
        'SET partitions = '
            A pair-wise list of actors which cannot send messages.  The
            list is uni-directional.  If there are three servers (a,b,c),
            and if the partitions list is '[{a,b},{b,c}]' then all
            messages from a->b and b->c will be dropped, but any other
            sender->recipient messages will be delivered successfully.
        'x uses:'
            The FLU x has made an internal state transition and is using
            this epoch's projection as operating chain configuration.  The
            rest of the line is a summary of the projection.
        'CONFIRM epoch {N}'
            This message confirms that all of the servers listed in the
            UPI and repairing lists of the projection at epoch {N} have
            agreed to use this projection because they all have written
            this projection to their respective private projection stores.
            The chain is now usable by/available to all clients.
        'Sweet, private projections are stable'
            This report announces that this iteration of the test cycle
            has passed successfully.  The report that follows briefly
            summarizes the latest private projection used by each
            participating server.  For example, when in strong consistency
            mode with 'a' as a witness and 'b' and 'c' as real servers:
            %% Legend:
            %% server name, epoch ID, UPI list, repairing list, down list, ...
            %%                         ... witness list, 'false' (a constant value)
            [{a,{{1116,<<23,143,246,55>>},[a,b],[],[c],[a],false}},
             {b,{{1116,<<23,143,246,55>>},[a,b],[],[c],[a],false}}]
            Both servers 'a' and 'b' agree on epoch 1116 with epoch ID
            {1116,<<23,143,246,55>>} where UPI=[a,b], repairing=[],
            down=[c], and witnesses=[a].
            Server 'c' is not shown because 'c' has wedged itself OOS (out
            of service) by configuring a chain length of zero.
            If no servers are listed in the report (i.e. only '[]' is
            displayed), then all servers have wedged themselves OOS, and
            the chain is unavailable.
        'DoIt,' 
            This marks a group of tick events which trigger the manager
            processes to evaluate their environment and perhaps make a
            state transition.
    A long chain of 'DoIt,DoIt,DoIt,' means that the chain state has
    (probably) settled to a stable configuration, which is the goal of the
    algorithm.
    Press control-c to interrupt the test....".
 ## Run a test in eventual consistency mode
 Run the following command at the Erlang CLI prompt:
    machi_chain_manager1_converge_demo:t(3, [{private_write_verbose,true}]).
 The first argument, `3`, is the number of servers to participate in
 the chain.  Please note:
 * Chain lengths as short as 1 or 2 are valid, but the results are a
  bit boring.
 * Chain lengths as long as 7 or 9 can be used, but they may
  suffer from longer periods of churn/instability before all chain
  managers reach agreement via humming consensus.  (It is future work
  to shorten the worst of the unstable churn latencies.)
 * In eventual consistency mode, chain lengths may be even numbers,
  e.g. 2, 4, or 6.
 * The simulator will choose partition events from the permutations of
  all 1, 2, and 3 node partition pairs.  The total runtime will
  increase *dramatically* with chain length.
    * Chain length 2: about 3 partition cases
    * Chain length 3: about 35 partition cases
    * Chain length 4: about 230 partition cases
    * Chain length 5: about 1100 partition cases
 ## Run a test in strong consistency mode (with witnesses):
 *NOTE:* Due to a bug in the test code, please do not try to run the
 convergence test in strong consistency mode and also without the
 correct minority number of witness servers!  If in doubt, please run
 the commands shown below exactly.
 Run the following command at the Erlang CLI prompt:
    machi_chain_manager1_converge_demo:t(3, [{private_write_verbose,true}, {consistency_mode, cp_mode}, {witnesses, [a]}]).
 The first argument, `3`, is the number of servers to participate in
 the chain.  Chain lengths as long as 7 or 9 can be used, but they may
 suffer from longer periods of churn/instability before all chain
 managers reach agreement via humming consensus.
 Due to the bug mentioned above, please use the following
 commands when running with chain lengths of 5 or 7, respectively.
    machi_chain_manager1_converge_demo:t(5, [{private_write_verbose,true}, {consistency_mode, cp_mode}, {witnesses, [a,b]}]).
    machi_chain_manager1_converge_demo:t(7, [{private_write_verbose,true}, {consistency_mode, cp_mode}, {witnesses, [a,b,c]}]).
--- a/doc/process-protocol-module-overview.jpg
+++ b/doc/process-protocol-module-overview.jpg
--- a/doc/src.high-level/.gitignore
+++ b/doc/src.high-level/.gitignore
@ -1,4 +0,0 @@
 *.aux
 *.dvi
 *.log
 *.pdf
--- a/doc/src.high-level/Makefile
+++ b/doc/src.high-level/Makefile
@ -1,12 +0,0 @@
 all: machi chain-mgr
 machi:
 	latex high-level-machi.tex
 	dvipdfm high-level-machi.dvi
 chain-mgr:
 	latex high-level-chain-mgr.tex
 	dvipdfm high-level-chain-mgr.dvi
 clean:
 	rm -f *.aux *.dvi *.log
--- a/doc/src.high-level/append-flow.eps
+++ b/doc/src.high-level/append-flow.eps
@ -1,268 +0,0 @@
 %!PS-Adobe-3.0 EPSF-2.0
 %%BoundingBox: 0 0 416.500000 280.000000
 %%Creator: mscgen 0.18
 %%EndComments
 0.700000 0.700000 scale
 0 0 moveto
 0 400 lineto
 595 400 lineto
 595 0 lineto
 closepath
 clip
 %PageTrailer
 %Page: 1 1
 /Helvetica findfont
 10 scalefont
 setfont
 /Helvetica findfont
 12 scalefont
 setfont
 0 400 translate
 /mtrx matrix def
 /ellipse
  { /endangle exch def
    /startangle exch def
    /ydia exch def
    /xdia exch def
    /y exch def
    /x exch def
    /savematrix mtrx currentmatrix def
    x y translate
    xdia 2 div ydia 2 div scale
    1 -1 scale
    0 0 1 startangle endangle arc
    savematrix setmatrix
 } def
 (client) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup dup newpath 42 -17 moveto 2 div neg 0 rmoveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 0.000000 0.000000 0.000000 setrgbcolor
 42 -15 moveto dup stringwidth pop 2 div neg 0 rmoveto show
 (Projection) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup dup newpath 127 -17 moveto 2 div neg 0 rmoveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 0.000000 0.000000 0.000000 setrgbcolor
 127 -15 moveto dup stringwidth pop 2 div neg 0 rmoveto show
 (ProjStore_A) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup dup newpath 212 -17 moveto 2 div neg 0 rmoveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 0.000000 0.000000 0.000000 setrgbcolor
 212 -15 moveto dup stringwidth pop 2 div neg 0 rmoveto show
 (Sequencer_A) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup dup newpath 297 -17 moveto 2 div neg 0 rmoveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 0.000000 0.000000 0.000000 setrgbcolor
 297 -15 moveto dup stringwidth pop 2 div neg 0 rmoveto show
 (FLU_A) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup dup newpath 382 -17 moveto 2 div neg 0 rmoveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 0.000000 0.000000 0.000000 setrgbcolor
 382 -15 moveto dup stringwidth pop 2 div neg 0 rmoveto show
 (FLU_B) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup dup newpath 467 -17 moveto 2 div neg 0 rmoveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 0.000000 0.000000 0.000000 setrgbcolor
 467 -15 moveto dup stringwidth pop 2 div neg 0 rmoveto show
 (FLU_C) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup dup newpath 552 -17 moveto 2 div neg 0 rmoveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 0.000000 0.000000 0.000000 setrgbcolor
 552 -15 moveto dup stringwidth pop 2 div neg 0 rmoveto show
 newpath 42 -22 moveto 42 -49 lineto stroke
 newpath 127 -22 moveto 127 -49 lineto stroke
 newpath 212 -22 moveto 212 -49 lineto stroke
 newpath 297 -22 moveto 297 -49 lineto stroke
 newpath 382 -22 moveto 382 -49 lineto stroke
 newpath 467 -22 moveto 467 -49 lineto stroke
 newpath 552 -22 moveto 552 -49 lineto stroke
 newpath 42 -35 moveto 127 -35 lineto stroke
 newpath 127 -35 moveto 117 -41 lineto stroke
 (get current) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup newpath 57 -33 moveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 0.000000 0.000000 0.000000 setrgbcolor
 57 -33 moveto show
 newpath 42 -49 moveto 42 -76 lineto stroke
 newpath 127 -49 moveto 127 -76 lineto stroke
 newpath 212 -49 moveto 212 -76 lineto stroke
 newpath 297 -49 moveto 297 -76 lineto stroke
 newpath 382 -49 moveto 382 -76 lineto stroke
 newpath 467 -49 moveto 467 -76 lineto stroke
 newpath 552 -49 moveto 552 -76 lineto stroke
 newpath 127 -62 moveto 42 -62 lineto stroke
 newpath 42 -62 moveto 52 -68 lineto stroke
 (ok, #12...) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup newpath 61 -60 moveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 0.000000 0.000000 0.000000 setrgbcolor
 61 -60 moveto show
 newpath 42 -76 moveto 42 -103 lineto stroke
 newpath 127 -76 moveto 127 -103 lineto stroke
 newpath 212 -76 moveto 212 -103 lineto stroke
 newpath 297 -76 moveto 297 -103 lineto stroke
 newpath 382 -76 moveto 382 -103 lineto stroke
 newpath 467 -76 moveto 467 -103 lineto stroke
 newpath 552 -76 moveto 552 -103 lineto stroke
 newpath 42 -89 moveto 297 -89 lineto stroke
 newpath 297 -89 moveto 287 -95 lineto stroke
 (Req. 123 bytes, prefix="foo", epoch=12) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup newpath 66 -87 moveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 0.000000 0.000000 0.000000 setrgbcolor
 66 -87 moveto show
 newpath 42 -103 moveto 42 -130 lineto stroke
 newpath 127 -103 moveto 127 -130 lineto stroke
 newpath 212 -103 moveto 212 -130 lineto stroke
 newpath 297 -103 moveto 297 -130 lineto stroke
 newpath 382 -103 moveto 382 -130 lineto stroke
 newpath 467 -103 moveto 467 -130 lineto stroke
 newpath 552 -103 moveto 552 -130 lineto stroke
 newpath 297 -116 moveto 42 -116 lineto stroke
 newpath 42 -116 moveto 52 -122 lineto stroke
 1.000000 0.000000 0.000000 setrgbcolor
 (bad_epoch, 13) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup newpath 131 -114 moveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 1.000000 0.000000 0.000000 setrgbcolor
 131 -114 moveto show
 0.000000 0.000000 0.000000 setrgbcolor
 newpath 42 -130 moveto 42 -157 lineto stroke
 newpath 127 -130 moveto 127 -157 lineto stroke
 newpath 212 -130 moveto 212 -157 lineto stroke
 newpath 297 -130 moveto 297 -157 lineto stroke
 newpath 382 -130 moveto 382 -157 lineto stroke
 newpath 467 -130 moveto 467 -157 lineto stroke
 newpath 552 -130 moveto 552 -157 lineto stroke
 newpath 42 -143 moveto 212 -143 lineto stroke
 newpath 212 -143 moveto 202 -149 lineto stroke
 (get epoch #13) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup newpath 89 -141 moveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 0.000000 0.000000 0.000000 setrgbcolor
 89 -141 moveto show
 newpath 42 -157 moveto 42 -184 lineto stroke
 newpath 127 -157 moveto 127 -184 lineto stroke
 newpath 212 -157 moveto 212 -184 lineto stroke
 newpath 297 -157 moveto 297 -184 lineto stroke
 newpath 382 -157 moveto 382 -184 lineto stroke
 newpath 467 -157 moveto 467 -184 lineto stroke
 newpath 552 -157 moveto 552 -184 lineto stroke
 newpath 212 -170 moveto 42 -170 lineto stroke
 newpath 42 -170 moveto 52 -176 lineto stroke
 (ok, #13...) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup newpath 103 -168 moveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 0.000000 0.000000 0.000000 setrgbcolor
 103 -168 moveto show
 newpath 42 -184 moveto 42 -211 lineto stroke
 newpath 127 -184 moveto 127 -211 lineto stroke
 newpath 212 -184 moveto 212 -211 lineto stroke
 newpath 297 -184 moveto 297 -211 lineto stroke
 newpath 382 -184 moveto 382 -211 lineto stroke
 newpath 467 -184 moveto 467 -211 lineto stroke
 newpath 552 -184 moveto 552 -211 lineto stroke
 newpath 42 -197 moveto 297 -197 lineto stroke
 newpath 297 -197 moveto 287 -203 lineto stroke
 (Req. 123 bytes, prefix="foo", epoch=13) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup newpath 66 -195 moveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 0.000000 0.000000 0.000000 setrgbcolor
 66 -195 moveto show
 newpath 42 -211 moveto 42 -238 lineto stroke
 newpath 127 -211 moveto 127 -238 lineto stroke
 newpath 212 -211 moveto 212 -238 lineto stroke
 newpath 297 -211 moveto 297 -238 lineto stroke
 newpath 382 -211 moveto 382 -238 lineto stroke
 newpath 467 -211 moveto 467 -238 lineto stroke
 newpath 552 -211 moveto 552 -238 lineto stroke
 newpath 297 -224 moveto 42 -224 lineto stroke
 newpath 42 -224 moveto 52 -230 lineto stroke
 (ok, "foo.seq_a.009" offset=447) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup newpath 89 -222 moveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 0.000000 0.000000 0.000000 setrgbcolor
 89 -222 moveto show
 newpath 42 -238 moveto 42 -265 lineto stroke
 newpath 127 -238 moveto 127 -265 lineto stroke
 newpath 212 -238 moveto 212 -265 lineto stroke
 newpath 297 -238 moveto 297 -265 lineto stroke
 newpath 382 -238 moveto 382 -265 lineto stroke
 newpath 467 -238 moveto 467 -265 lineto stroke
 newpath 552 -238 moveto 552 -265 lineto stroke
 newpath 42 -251 moveto 382 -251 lineto stroke
 newpath 382 -251 moveto 372 -257 lineto stroke
 (write "foo.seq_a.009" offset=447 <<123 bytes...>> epoch=13) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup newpath 51 -249 moveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 0.000000 0.000000 0.000000 setrgbcolor
 51 -249 moveto show
 newpath 42 -265 moveto 42 -292 lineto stroke
 newpath 127 -265 moveto 127 -292 lineto stroke
 newpath 212 -265 moveto 212 -292 lineto stroke
 newpath 297 -265 moveto 297 -292 lineto stroke
 newpath 382 -265 moveto 382 -292 lineto stroke
 newpath 467 -265 moveto 467 -292 lineto stroke
 newpath 552 -265 moveto 552 -292 lineto stroke
 newpath 382 -278 moveto 42 -278 lineto stroke
 newpath 42 -278 moveto 52 -284 lineto stroke
 (ok) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup newpath 206 -276 moveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 0.000000 0.000000 0.000000 setrgbcolor
 206 -276 moveto show
 newpath 42 -292 moveto 42 -319 lineto stroke
 newpath 127 -292 moveto 127 -319 lineto stroke
 newpath 212 -292 moveto 212 -319 lineto stroke
 newpath 297 -292 moveto 297 -319 lineto stroke
 newpath 382 -292 moveto 382 -319 lineto stroke
 newpath 467 -292 moveto 467 -319 lineto stroke
 newpath 552 -292 moveto 552 -319 lineto stroke
 newpath 42 -305 moveto 467 -305 lineto stroke
 newpath 467 -305 moveto 457 -311 lineto stroke
 (write "foo.seq_a.009" offset=447 <<123 bytes...>> epoch=13) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup newpath 94 -303 moveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 0.000000 0.000000 0.000000 setrgbcolor
 94 -303 moveto show
 newpath 42 -319 moveto 42 -346 lineto stroke
 newpath 127 -319 moveto 127 -346 lineto stroke
 newpath 212 -319 moveto 212 -346 lineto stroke
 newpath 297 -319 moveto 297 -346 lineto stroke
 newpath 382 -319 moveto 382 -346 lineto stroke
 newpath 467 -319 moveto 467 -346 lineto stroke
 newpath 552 -319 moveto 552 -346 lineto stroke
 newpath 467 -332 moveto 42 -332 lineto stroke
 newpath 42 -332 moveto 52 -338 lineto stroke
 (ok) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup newpath 249 -330 moveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 0.000000 0.000000 0.000000 setrgbcolor
 249 -330 moveto show
 newpath 42 -346 moveto 42 -373 lineto stroke
 newpath 127 -346 moveto 127 -373 lineto stroke
 newpath 212 -346 moveto 212 -373 lineto stroke
 newpath 297 -346 moveto 297 -373 lineto stroke
 newpath 382 -346 moveto 382 -373 lineto stroke
 newpath 467 -346 moveto 467 -373 lineto stroke
 newpath 552 -346 moveto 552 -373 lineto stroke
 newpath 42 -359 moveto 552 -359 lineto stroke
 newpath 552 -359 moveto 542 -365 lineto stroke
 (write "foo.seq_a.009" offset=447 <<123 bytes...>> epoch=13) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup newpath 136 -357 moveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 0.000000 0.000000 0.000000 setrgbcolor
 136 -357 moveto show
 newpath 42 -373 moveto 42 -400 lineto stroke
 newpath 127 -373 moveto 127 -400 lineto stroke
 newpath 212 -373 moveto 212 -400 lineto stroke
 newpath 297 -373 moveto 297 -400 lineto stroke
 newpath 382 -373 moveto 382 -400 lineto stroke
 newpath 467 -373 moveto 467 -400 lineto stroke
 newpath 552 -373 moveto 552 -400 lineto stroke
 newpath 552 -386 moveto 42 -386 lineto stroke
 newpath 42 -386 moveto 52 -392 lineto stroke
 (ok) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup newpath 291 -384 moveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 0.000000 0.000000 0.000000 setrgbcolor
 291 -384 moveto show
--- a/doc/src.high-level/append-flow2.eps
+++ b/doc/src.high-level/append-flow2.eps
@ -1,345 +0,0 @@
 %!PS-Adobe-3.0 EPSF-2.0
 %%BoundingBox: 0 0 416.500000 355.600006
 %%Creator: mscgen 0.18
 %%EndComments
 0.700000 0.700000 scale
 0 0 moveto
 0 508 lineto
 595 508 lineto
 595 0 lineto
 closepath
 clip
 %PageTrailer
 %Page: 1 1
 /Helvetica findfont
 10 scalefont
 setfont
 /Helvetica findfont
 12 scalefont
 setfont
 0 508 translate
 /mtrx matrix def
 /ellipse
  { /endangle exch def
    /startangle exch def
    /ydia exch def
    /xdia exch def
    /y exch def
    /x exch def
    /savematrix mtrx currentmatrix def
    x y translate
    xdia 2 div ydia 2 div scale
    1 -1 scale
    0 0 1 startangle endangle arc
    savematrix setmatrix
 } def
 (client) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup dup newpath 42 -17 moveto 2 div neg 0 rmoveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 0.000000 0.000000 0.000000 setrgbcolor
 42 -15 moveto dup stringwidth pop 2 div neg 0 rmoveto show
 (Projection) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup dup newpath 127 -17 moveto 2 div neg 0 rmoveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 0.000000 0.000000 0.000000 setrgbcolor
 127 -15 moveto dup stringwidth pop 2 div neg 0 rmoveto show
 (ProjStore_A) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup dup newpath 212 -17 moveto 2 div neg 0 rmoveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 0.000000 0.000000 0.000000 setrgbcolor
 212 -15 moveto dup stringwidth pop 2 div neg 0 rmoveto show
 (Sequencer_A) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup dup newpath 297 -17 moveto 2 div neg 0 rmoveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 0.000000 0.000000 0.000000 setrgbcolor
 297 -15 moveto dup stringwidth pop 2 div neg 0 rmoveto show
 (FLU_A) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup dup newpath 382 -17 moveto 2 div neg 0 rmoveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 0.000000 0.000000 0.000000 setrgbcolor
 382 -15 moveto dup stringwidth pop 2 div neg 0 rmoveto show
 (FLU_B) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup dup newpath 467 -17 moveto 2 div neg 0 rmoveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 0.000000 0.000000 0.000000 setrgbcolor
 467 -15 moveto dup stringwidth pop 2 div neg 0 rmoveto show
 (FLU_C) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup dup newpath 552 -17 moveto 2 div neg 0 rmoveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 0.000000 0.000000 0.000000 setrgbcolor
 552 -15 moveto dup stringwidth pop 2 div neg 0 rmoveto show
 newpath 42 -22 moveto 42 -49 lineto stroke
 newpath 127 -22 moveto 127 -49 lineto stroke
 newpath 212 -22 moveto 212 -49 lineto stroke
 newpath 297 -22 moveto 297 -49 lineto stroke
 newpath 382 -22 moveto 382 -49 lineto stroke
 newpath 467 -22 moveto 467 -49 lineto stroke
 newpath 552 -22 moveto 552 -49 lineto stroke
 newpath 42 -35 moveto 127 -35 lineto stroke
 newpath 127 -35 moveto 117 -41 lineto stroke
 (get current) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup newpath 57 -33 moveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 0.000000 0.000000 0.000000 setrgbcolor
 57 -33 moveto show
 newpath 42 -49 moveto 42 -76 lineto stroke
 newpath 127 -49 moveto 127 -76 lineto stroke
 newpath 212 -49 moveto 212 -76 lineto stroke
 newpath 297 -49 moveto 297 -76 lineto stroke
 newpath 382 -49 moveto 382 -76 lineto stroke
 newpath 467 -49 moveto 467 -76 lineto stroke
 newpath 552 -49 moveto 552 -76 lineto stroke
 newpath 127 -62 moveto 42 -62 lineto stroke
 newpath 42 -62 moveto 52 -68 lineto stroke
 (ok, #12...) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup newpath 61 -60 moveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 0.000000 0.000000 0.000000 setrgbcolor
 61 -60 moveto show
 newpath 42 -76 moveto 42 -103 lineto stroke
 newpath 127 -76 moveto 127 -103 lineto stroke
 newpath 212 -76 moveto 212 -103 lineto stroke
 newpath 297 -76 moveto 297 -103 lineto stroke
 newpath 382 -76 moveto 382 -103 lineto stroke
 newpath 467 -76 moveto 467 -103 lineto stroke
 newpath 552 -76 moveto 552 -103 lineto stroke
 newpath 42 -89 moveto 382 -89 lineto stroke
 newpath 382 -89 moveto 372 -95 lineto stroke
 (append prefix="foo" <<123 bytes...>> epoch=12) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup newpath 85 -87 moveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 0.000000 0.000000 0.000000 setrgbcolor
 85 -87 moveto show
 newpath 42 -103 moveto 42 -130 lineto stroke
 newpath 127 -103 moveto 127 -130 lineto stroke
 newpath 212 -103 moveto 212 -130 lineto stroke
 newpath 297 -103 moveto 297 -130 lineto stroke
 newpath 382 -103 moveto 382 -130 lineto stroke
 newpath 467 -103 moveto 467 -130 lineto stroke
 newpath 552 -103 moveto 552 -130 lineto stroke
 newpath 382 -116 moveto 42 -116 lineto stroke
 newpath 42 -116 moveto 52 -122 lineto stroke
 1.000000 0.000000 0.000000 setrgbcolor
 (bad_epoch, 13) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup newpath 173 -114 moveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 1.000000 0.000000 0.000000 setrgbcolor
 173 -114 moveto show
 0.000000 0.000000 0.000000 setrgbcolor
 newpath 42 -130 moveto 42 -157 lineto stroke
 newpath 127 -130 moveto 127 -157 lineto stroke
 newpath 212 -130 moveto 212 -157 lineto stroke
 newpath 297 -130 moveto 297 -157 lineto stroke
 newpath 382 -130 moveto 382 -157 lineto stroke
 newpath 467 -130 moveto 467 -157 lineto stroke
 newpath 552 -130 moveto 552 -157 lineto stroke
 newpath 42 -143 moveto 212 -143 lineto stroke
 newpath 212 -143 moveto 202 -149 lineto stroke
 (get epoch #13) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup newpath 89 -141 moveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 0.000000 0.000000 0.000000 setrgbcolor
 89 -141 moveto show
 newpath 42 -157 moveto 42 -184 lineto stroke
 newpath 127 -157 moveto 127 -184 lineto stroke
 newpath 212 -157 moveto 212 -184 lineto stroke
 newpath 297 -157 moveto 297 -184 lineto stroke
 newpath 382 -157 moveto 382 -184 lineto stroke
 newpath 467 -157 moveto 467 -184 lineto stroke
 newpath 552 -157 moveto 552 -184 lineto stroke
 newpath 212 -170 moveto 42 -170 lineto stroke
 newpath 42 -170 moveto 52 -176 lineto stroke
 (ok, #13...) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup newpath 103 -168 moveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 0.000000 0.000000 0.000000 setrgbcolor
 103 -168 moveto show
 newpath 42 -184 moveto 42 -211 lineto stroke
 newpath 127 -184 moveto 127 -211 lineto stroke
 newpath 212 -184 moveto 212 -211 lineto stroke
 newpath 297 -184 moveto 297 -211 lineto stroke
 newpath 382 -184 moveto 382 -211 lineto stroke
 newpath 467 -184 moveto 467 -211 lineto stroke
 newpath 552 -184 moveto 552 -211 lineto stroke
 newpath 42 -197 moveto 382 -197 lineto stroke
 newpath 382 -197 moveto 372 -203 lineto stroke
 (append prefix="foo" <<123 bytes...>> epoch=13) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup newpath 85 -195 moveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 0.000000 0.000000 0.000000 setrgbcolor
 85 -195 moveto show
 newpath 42 -211 moveto 42 -238 lineto stroke
 newpath 127 -211 moveto 127 -238 lineto stroke
 newpath 212 -211 moveto 212 -238 lineto stroke
 newpath 297 -211 moveto 297 -238 lineto stroke
 newpath 382 -211 moveto 382 -238 lineto stroke
 newpath 467 -211 moveto 467 -238 lineto stroke
 newpath 552 -211 moveto 552 -238 lineto stroke
 1.000000 1.000000 1.000000 setrgbcolor
 newpath 263 -211 moveto 417 -211 lineto 417 -236 lineto 263 -236 lineto closepath fill
 0.000000 0.000000 0.000000 setrgbcolor
 newpath 269 -211 moveto 411 -211 lineto stroke
 newpath 269 -236 moveto 411 -236 lineto stroke
 newpath 269 -211 moveto 263 -223 lineto stroke
 newpath 263 -223 moveto 269 -236 lineto stroke
 newpath 411 -211 moveto 417 -223 lineto stroke
 newpath 417 -223 moveto 411 -236 lineto stroke
 (Co-located on same box) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup newpath 275 -227 moveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 0.000000 0.000000 0.000000 setrgbcolor
 275 -227 moveto show
 newpath 42 -238 moveto 42 -265 lineto stroke
 newpath 127 -238 moveto 127 -265 lineto stroke
 newpath 212 -238 moveto 212 -265 lineto stroke
 newpath 297 -238 moveto 297 -265 lineto stroke
 newpath 382 -238 moveto 382 -265 lineto stroke
 newpath 467 -238 moveto 467 -265 lineto stroke
 newpath 552 -238 moveto 552 -265 lineto stroke
 newpath 382 -251 moveto 297 -251 lineto stroke
 newpath 297 -251 moveto 307 -257 lineto stroke
 (Req. 123 bytes, prefix="foo", epoch=13) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup newpath 236 -249 moveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 0.000000 0.000000 0.000000 setrgbcolor
 236 -249 moveto show
 newpath 42 -265 moveto 42 -292 lineto stroke
 newpath 127 -265 moveto 127 -292 lineto stroke
 newpath 212 -265 moveto 212 -292 lineto stroke
 newpath 297 -265 moveto 297 -292 lineto stroke
 newpath 382 -265 moveto 382 -292 lineto stroke
 newpath 467 -265 moveto 467 -292 lineto stroke
 newpath 552 -265 moveto 552 -292 lineto stroke
 newpath 297 -278 moveto 382 -278 lineto stroke
 newpath 382 -278 moveto 372 -284 lineto stroke
 (ok, "foo.seq_a.009" offset=447) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup newpath 259 -276 moveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 0.000000 0.000000 0.000000 setrgbcolor
 259 -276 moveto show
 newpath 42 -292 moveto 42 -319 lineto stroke
 newpath 127 -292 moveto 127 -319 lineto stroke
 newpath 212 -292 moveto 212 -319 lineto stroke
 newpath 297 -292 moveto 297 -319 lineto stroke
 newpath 382 -292 moveto 382 -319 lineto stroke
 newpath 467 -292 moveto 467 -319 lineto stroke
 newpath 552 -292 moveto 552 -319 lineto stroke
 newpath 382 -305 85 13 270 90 ellipse stroke
 newpath 382 -311 moveto 392 -317 lineto stroke
 (write "foo.seq_a.009" offset=447 <<123 bytes...>> epoch=13) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup newpath 58 -303 moveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 0.000000 0.000000 0.000000 setrgbcolor
 58 -303 moveto show
 newpath 42 -319 moveto 42 -346 lineto stroke
 newpath 127 -319 moveto 127 -346 lineto stroke
 newpath 212 -319 moveto 212 -346 lineto stroke
 newpath 297 -319 moveto 297 -346 lineto stroke
 newpath 382 -319 moveto 382 -346 lineto stroke
 newpath 467 -319 moveto 467 -346 lineto stroke
 newpath 552 -319 moveto 552 -346 lineto stroke
 newpath 382 -332 moveto 467 -332 lineto stroke
 newpath 467 -332 moveto 457 -338 lineto stroke
 (write "foo.seq_a.009" offset=447 <<123 bytes...>> epoch=13) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup newpath 264 -330 moveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 0.000000 0.000000 0.000000 setrgbcolor
 264 -330 moveto show
 newpath 42 -346 moveto 42 -373 lineto stroke
 newpath 127 -346 moveto 127 -373 lineto stroke
 newpath 212 -346 moveto 212 -373 lineto stroke
 newpath 297 -346 moveto 297 -373 lineto stroke
 newpath 382 -346 moveto 382 -373 lineto stroke
 newpath 467 -346 moveto 467 -373 lineto stroke
 newpath 552 -346 moveto 552 -373 lineto stroke
 newpath 467 -359 moveto 552 -359 lineto stroke
 newpath 552 -359 moveto 542 -365 lineto stroke
 (write "foo.seq_a.009" offset=447 <<123 bytes...>> epoch=13) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup newpath 273 -357 moveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 0.000000 0.000000 0.000000 setrgbcolor
 273 -357 moveto show
 newpath 42 -373 moveto 42 -400 lineto stroke
 newpath 127 -373 moveto 127 -400 lineto stroke
 newpath 212 -373 moveto 212 -400 lineto stroke
 newpath 297 -373 moveto 297 -400 lineto stroke
 newpath 382 -373 moveto 382 -400 lineto stroke
 newpath 467 -373 moveto 467 -400 lineto stroke
 newpath 552 -373 moveto 552 -400 lineto stroke
 newpath 552 -386 moveto 42 -386 lineto stroke
 newpath 42 -386 moveto 52 -392 lineto stroke
 (ok, "foo.seq_a.009" offset=447) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup newpath 216 -384 moveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 0.000000 0.000000 0.000000 setrgbcolor
 216 -384 moveto show
 newpath 42 -400 moveto 42 -427 lineto stroke
 newpath 127 -400 moveto 127 -427 lineto stroke
 newpath 212 -400 moveto 212 -427 lineto stroke
 newpath 297 -400 moveto 297 -427 lineto stroke
 newpath 382 -400 moveto 382 -427 lineto stroke
 newpath 467 -400 moveto 467 -427 lineto stroke
 newpath 552 -400 moveto 552 -427 lineto stroke
 (The above is "fast path" for FLU->FLU forwarding.) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup newpath 164 -416 moveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 0.000000 0.000000 0.000000 setrgbcolor
 164 -416 moveto show
 [2] 0 setdash
 newpath 21 -413 moveto 162 -413 lineto stroke
 [] 0 setdash
 [2] 0 setdash
 newpath 432 -413 moveto 574 -413 lineto stroke
 [] 0 setdash
 newpath 42 -427 moveto 42 -454 lineto stroke
 newpath 127 -427 moveto 127 -454 lineto stroke
 newpath 212 -427 moveto 212 -454 lineto stroke
 newpath 297 -427 moveto 297 -454 lineto stroke
 newpath 382 -427 moveto 382 -454 lineto stroke
 newpath 467 -427 moveto 467 -454 lineto stroke
 newpath 552 -427 moveto 552 -454 lineto stroke
 (If, in an alternate scenario, FLU_C has an error...) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup newpath 167 -443 moveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 0.000000 0.000000 0.000000 setrgbcolor
 167 -443 moveto show
 [2] 0 setdash
 newpath 21 -440 moveto 165 -440 lineto stroke
 [] 0 setdash
 [2] 0 setdash
 newpath 429 -440 moveto 574 -440 lineto stroke
 [] 0 setdash
 newpath 42 -454 moveto 42 -481 lineto stroke
 newpath 127 -454 moveto 127 -481 lineto stroke
 newpath 212 -454 moveto 212 -481 lineto stroke
 newpath 297 -454 moveto 297 -481 lineto stroke
 newpath 382 -454 moveto 382 -481 lineto stroke
 newpath 467 -454 moveto 467 -481 lineto stroke
 newpath 552 -454 moveto 552 -481 lineto stroke
 newpath 552 -467 moveto 42 -467 lineto stroke
 newpath 42 -467 moveto 52 -473 lineto stroke
 1.000000 0.000000 0.000000 setrgbcolor
 (bad_epoch, 15) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup newpath 258 -465 moveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 1.000000 0.000000 0.000000 setrgbcolor
 258 -465 moveto show
 0.000000 0.000000 0.000000 setrgbcolor
 newpath 42 -481 moveto 42 -508 lineto stroke
 newpath 127 -481 moveto 127 -508 lineto stroke
 newpath 212 -481 moveto 212 -508 lineto stroke
 newpath 297 -481 moveto 297 -508 lineto stroke
 newpath 382 -481 moveto 382 -508 lineto stroke
 newpath 467 -481 moveto 467 -508 lineto stroke
 newpath 552 -481 moveto 552 -508 lineto stroke
 (... then repair becomes the client's responsibility \("slow path"\).) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup newpath 133 -497 moveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 0.000000 0.000000 0.000000 setrgbcolor
 133 -497 moveto show
 [2] 0 setdash
 newpath 21 -494 moveto 131 -494 lineto stroke
 [] 0 setdash
 [2] 0 setdash
 newpath 464 -494 moveto 574 -494 lineto stroke
 [] 0 setdash
--- a/doc/src.high-level/chain-self-management-sketch.Diagram1.eps
+++ b/doc/src.high-level/chain-self-management-sketch.Diagram1.eps
--- a/doc/src.high-level/figure6.eps
+++ b/doc/src.high-level/figure6.eps
@ -1,557 +0,0 @@
 %!PS-Adobe-3.0 EPSF-3.0
 %%Title: figure6.fig
 %%Creator: fig2dev Version 3.2 Patchlevel 5d
 %%CreationDate: Mon Oct 20 21:56:33 2014
 %%For: fritchie@sbb3.local (Scott Lystig Fritchie)
 %%BoundingBox: 0 0 633 332
 %Magnification: 1.0000
 %%EndComments
 %%BeginProlog
 /$F2psDict 200 dict def
 $F2psDict begin
 $F2psDict /mtrx matrix put
 /col-1 {0 setgray} bind def
 /col0 {0.000 0.000 0.000 srgb} bind def
 /col1 {0.000 0.000 1.000 srgb} bind def
 /col2 {0.000 1.000 0.000 srgb} bind def
 /col3 {0.000 1.000 1.000 srgb} bind def
 /col4 {1.000 0.000 0.000 srgb} bind def
 /col5 {1.000 0.000 1.000 srgb} bind def
 /col6 {1.000 1.000 0.000 srgb} bind def
 /col7 {1.000 1.000 1.000 srgb} bind def
 /col8 {0.000 0.000 0.560 srgb} bind def
 /col9 {0.000 0.000 0.690 srgb} bind def
 /col10 {0.000 0.000 0.820 srgb} bind def
 /col11 {0.530 0.810 1.000 srgb} bind def
 /col12 {0.000 0.560 0.000 srgb} bind def
 /col13 {0.000 0.690 0.000 srgb} bind def
 /col14 {0.000 0.820 0.000 srgb} bind def
 /col15 {0.000 0.560 0.560 srgb} bind def
 /col16 {0.000 0.690 0.690 srgb} bind def
 /col17 {0.000 0.820 0.820 srgb} bind def
 /col18 {0.560 0.000 0.000 srgb} bind def
 /col19 {0.690 0.000 0.000 srgb} bind def
 /col20 {0.820 0.000 0.000 srgb} bind def
 /col21 {0.560 0.000 0.560 srgb} bind def
 /col22 {0.690 0.000 0.690 srgb} bind def
 /col23 {0.820 0.000 0.820 srgb} bind def
 /col24 {0.500 0.190 0.000 srgb} bind def
 /col25 {0.630 0.250 0.000 srgb} bind def
 /col26 {0.750 0.380 0.000 srgb} bind def
 /col27 {1.000 0.500 0.500 srgb} bind def
 /col28 {1.000 0.630 0.630 srgb} bind def
 /col29 {1.000 0.750 0.750 srgb} bind def
 /col30 {1.000 0.880 0.880 srgb} bind def
 /col31 {1.000 0.840 0.000 srgb} bind def
 end
 /cp {closepath} bind def
 /ef {eofill} bind def
 /gr {grestore} bind def
 /gs {gsave} bind def
 /sa {save} bind def
 /rs {restore} bind def
 /l {lineto} bind def
 /m {moveto} bind def
 /rm {rmoveto} bind def
 /n {newpath} bind def
 /s {stroke} bind def
 /sh {show} bind def
 /slc {setlinecap} bind def
 /slj {setlinejoin} bind def
 /slw {setlinewidth} bind def
 /srgb {setrgbcolor} bind def
 /rot {rotate} bind def
 /sc {scale} bind def
 /sd {setdash} bind def
 /ff {findfont} bind def
 /sf {setfont} bind def
 /scf {scalefont} bind def
 /sw {stringwidth} bind def
 /tr {translate} bind def
 /tnt {dup dup currentrgbcolor
  4 -2 roll dup 1 exch sub 3 -1 roll mul add
  4 -2 roll dup 1 exch sub 3 -1 roll mul add
  4 -2 roll dup 1 exch sub 3 -1 roll mul add srgb}
  bind def
 /shd {dup dup currentrgbcolor 4 -2 roll mul 4 -2 roll mul
  4 -2 roll mul srgb} bind def
 /reencdict 12 dict def /ReEncode { reencdict begin
 /newcodesandnames exch def /newfontname exch def /basefontname exch def
 /basefontdict basefontname findfont def /newfont basefontdict maxlength dict def
 basefontdict { exch dup /FID ne { dup /Encoding eq
 { exch dup length array copy newfont 3 1 roll put }
 { exch newfont 3 1 roll put } ifelse } { pop pop } ifelse } forall
 newfont /FontName newfontname put newcodesandnames aload pop
 128 1 255 { newfont /Encoding get exch /.notdef put } for
 newcodesandnames length 2 idiv { newfont /Encoding get 3 1 roll put } repeat
 newfontname newfont definefont pop end } def
 /isovec [
 8#055 /minus 8#200 /grave 8#201 /acute 8#202 /circumflex 8#203 /tilde
 8#204 /macron 8#205 /breve 8#206 /dotaccent 8#207 /dieresis
 8#210 /ring 8#211 /cedilla 8#212 /hungarumlaut 8#213 /ogonek 8#214 /caron
 8#220 /dotlessi 8#230 /oe 8#231 /OE
 8#240 /space 8#241 /exclamdown 8#242 /cent 8#243 /sterling
 8#244 /currency 8#245 /yen 8#246 /brokenbar 8#247 /section 8#250 /dieresis
 8#251 /copyright 8#252 /ordfeminine 8#253 /guillemotleft 8#254 /logicalnot
 8#255 /hyphen 8#256 /registered 8#257 /macron 8#260 /degree 8#261 /plusminus
 8#262 /twosuperior 8#263 /threesuperior 8#264 /acute 8#265 /mu 8#266 /paragraph
 8#267 /periodcentered 8#270 /cedilla 8#271 /onesuperior 8#272 /ordmasculine
 8#273 /guillemotright 8#274 /onequarter 8#275 /onehalf
 8#276 /threequarters 8#277 /questiondown 8#300 /Agrave 8#301 /Aacute
 8#302 /Acircumflex 8#303 /Atilde 8#304 /Adieresis 8#305 /Aring
 8#306 /AE 8#307 /Ccedilla 8#310 /Egrave 8#311 /Eacute
 8#312 /Ecircumflex 8#313 /Edieresis 8#314 /Igrave 8#315 /Iacute
 8#316 /Icircumflex 8#317 /Idieresis 8#320 /Eth 8#321 /Ntilde 8#322 /Ograve
 8#323 /Oacute 8#324 /Ocircumflex 8#325 /Otilde 8#326 /Odieresis 8#327 /multiply
 8#330 /Oslash 8#331 /Ugrave 8#332 /Uacute 8#333 /Ucircumflex
 8#334 /Udieresis 8#335 /Yacute 8#336 /Thorn 8#337 /germandbls 8#340 /agrave
 8#341 /aacute 8#342 /acircumflex 8#343 /atilde 8#344 /adieresis 8#345 /aring
 8#346 /ae 8#347 /ccedilla 8#350 /egrave 8#351 /eacute
 8#352 /ecircumflex 8#353 /edieresis 8#354 /igrave 8#355 /iacute
 8#356 /icircumflex 8#357 /idieresis 8#360 /eth 8#361 /ntilde 8#362 /ograve
 8#363 /oacute 8#364 /ocircumflex 8#365 /otilde 8#366 /odieresis 8#367 /divide
 8#370 /oslash 8#371 /ugrave 8#372 /uacute 8#373 /ucircumflex
 8#374 /udieresis 8#375 /yacute 8#376 /thorn 8#377 /ydieresis] def
 /Times-Bold /Times-Bold-iso isovec ReEncode
 /Times-Roman /Times-Roman-iso isovec ReEncode
 /$F2psBegin {$F2psDict begin /$F2psEnteredState save def} def
 /$F2psEnd {$F2psEnteredState restore end} def
 /pageheader {
 save
 newpath 0 332 moveto 0 0 lineto 633 0 lineto 633 332 lineto closepath clip newpath
 -331.7 473.8 translate
 1 -1 scale
 $F2psBegin
 10 setmiterlimit
 0 slj 0 slc
 0.06000 0.06000 sc
 } bind def
 /pagefooter {
 $F2psEnd
 restore
 } bind def
 %%EndProlog
 pageheader
 %
 % Fig objects follow
 %
 % 
 % here starts figure with depth 50
 /Times-Roman-iso ff 166.67 scf sf
 5925 7275 m
 gs 1 -1 sc (Step 6: Client now knows that projection 12 is invalid. Fetch projection 13, then retry at step #8.) col16 sh gr
 /Times-Bold-iso ff 200.00 scf sf
 8550 3225 m
 gs 1 -1 sc (Get epoch 13) col0 sh gr
 /Times-Bold-iso ff 200.00 scf sf
 5925 6900 m
 gs 1 -1 sc (Active=[a,b,c]) col0 sh gr
 /Times-Bold-iso ff 200.00 scf sf
 5925 6675 m
 gs 1 -1 sc (Members=[a,b,c]) col0 sh gr
 /Times-Bold-iso ff 200.00 scf sf
 5925 6450 m
 gs 1 -1 sc (Epoch=13) col0 sh gr
 /Times-Bold-iso ff 200.00 scf sf
 5925 5580 m
 gs 1 -1 sc (Epoch=12) col0 sh gr
 /Times-Bold-iso ff 200.00 scf sf
 5925 5835 m
 gs 1 -1 sc (Members=[a,b,c]) col0 sh gr
 /Times-Bold-iso ff 200.00 scf sf
 5925 6090 m
 gs 1 -1 sc (Active=[a,b]) col0 sh gr
 /Times-Bold-iso ff 200.00 scf sf
 5925 5175 m
 gs 1 -1 sc (Projection \(data structure\)) col0 sh gr
 % Polyline
 0 slj
 0 slc
 15.000 slw
 n 8400 4950 m 5625 4950 l 5625 7050 l 8400 7050 l
 cp gs col0 s gr 
 /Times-Bold-iso ff 200.00 scf sf
 12825 6405 m
 gs 1 -1 sc (- write once) col0 sh gr
 /Times-Bold-iso ff 200.00 scf sf
 12825 6660 m
 gs 1 -1 sc (- key=integer) col0 sh gr
 /Times-Bold-iso ff 200.00 scf sf
 12825 6915 m
 gs 1 -1 sc (- value=projection data structure) col0 sh gr
 /Times-Bold-iso ff 200.00 scf sf
 12825 7170 m
 gs 1 -1 sc (k=11, v=...) col0 sh gr
 /Times-Bold-iso ff 200.00 scf sf
 12825 7425 m
 gs 1 -1 sc (k=12, v=...) col0 sh gr
 /Times-Bold-iso ff 200.00 scf sf
 12825 7680 m
 gs 1 -1 sc (k=13, v=...) col0 sh gr
 /Times-Bold-iso ff 200.00 scf sf
 12825 6150 m
 gs 1 -1 sc (FLU projection store \(proc\)) col0 sh gr
 % Polyline
 n 12750 5925 m 15900 5925 l 15900 7725 l 12750 7725 l
 cp gs col0 s gr 
 % Polyline
 gs  clippath
 14788 5055 m 14940 5055 l 14940 4995 l 14788 4995 l 14788 4995 l 14908 5025 l 14788 5055 l cp
 14612 4995 m 14460 4995 l 14460 5055 l 14612 5055 l 14612 5055 l 14492 5025 l 14612 4995 l cp
 eoclip
 n 14475 5025 m
 14925 5025 l gs col0 s gr gr
 % arrowhead
 7.500 slw
 n 14612 4995 m 14492 5025 l 14612 5055 l  col0 s
 % arrowhead
 n 14788 5055 m 14908 5025 l 14788 4995 l  col0 s
 % Polyline
 15.000 slw
 gs  clippath
 15688 5055 m 15840 5055 l 15840 4995 l 15688 4995 l 15688 4995 l 15808 5025 l 15688 5055 l cp
 15137 4995 m 14985 4995 l 14985 5055 l 15137 5055 l 15137 5055 l 15017 5025 l 15137 4995 l cp
 eoclip
 n 15000 5025 m
 15825 5025 l gs col0 s gr gr
 % arrowhead
 7.500 slw
 n 15137 4995 m 15017 5025 l 15137 5055 l  col0 s
 % arrowhead
 n 15688 5055 m 15808 5025 l 15688 4995 l  col0 s
 % Polyline
 15.000 slw
 gs  clippath
 14638 5355 m 14790 5355 l 14790 5295 l 14638 5295 l 14638 5295 l 14758 5325 l 14638 5355 l cp
 14612 5295 m 14460 5295 l 14460 5355 l 14612 5355 l 14612 5355 l 14492 5325 l 14612 5295 l cp
 eoclip
 n 14475 5325 m 14550 5325 l 14625 5325 l 14700 5325 l 14775 5325 l 14700 5325 l
 14775 5325 l gs col0 s gr gr
 % arrowhead
 7.500 slw
 n 14612 5295 m 14492 5325 l 14612 5355 l  col0 s
 % arrowhead
 n 14638 5355 m 14758 5325 l 14638 5295 l  col0 s
 % Polyline
 15.000 slw
 gs  clippath
 15163 5355 m 15315 5355 l 15315 5295 l 15163 5295 l 15163 5295 l 15283 5325 l 15163 5355 l cp
 15137 5295 m 14985 5295 l 14985 5355 l 15137 5355 l 15137 5355 l 15017 5325 l 15137 5295 l cp
 eoclip
 n 15000 5325 m 15075 5325 l 15150 5325 l 15225 5325 l
 15300 5325 l gs col0 s gr gr
 % arrowhead
 7.500 slw
 n 15137 5295 m 15017 5325 l 15137 5355 l  col0 s
 % arrowhead
 n 15163 5355 m 15283 5325 l 15163 5295 l  col0 s
 % Polyline
 15.000 slw
 gs  clippath
 15688 5355 m 15840 5355 l 15840 5295 l 15688 5295 l 15688 5295 l 15808 5325 l 15688 5355 l cp
 15587 5295 m 15435 5295 l 15435 5355 l 15587 5355 l 15587 5355 l 15467 5325 l 15587 5295 l cp
 eoclip
 n 15450 5325 m 15525 5325 l 15600 5325 l 15675 5325 l 15750 5325 l
 15825 5325 l gs col0 s gr gr
 % arrowhead
 7.500 slw
 n 15587 5295 m 15467 5325 l 15587 5355 l  col0 s
 % arrowhead
 n 15688 5355 m 15808 5325 l 15688 5295 l  col0 s
 % Polyline
 [60] 0 sd
 n 14475 5025 m
 15825 5025 l gs col0 s gr  [] 0 sd
 % Polyline
 [60] 0 sd
 n 14475 5325 m
 15825 5325 l gs col0 s gr  [] 0 sd
 % Polyline
 [60] 0 sd
 n 14475 5550 m
 15825 5550 l gs col0 s gr  [] 0 sd
 /Times-Bold-iso ff 200.00 scf sf
 12825 4575 m
 gs 1 -1 sc (epoch=13) col0 sh gr
 /Times-Bold-iso ff 200.00 scf sf
 12825 4830 m
 gs 1 -1 sc (files:) col0 sh gr
 /Times-Bold-iso ff 200.00 scf sf
 12825 5085 m
 gs 1 -1 sc (  "foo.seq_a.006") col0 sh gr
 /Times-Bold-iso ff 200.00 scf sf
 12825 5340 m
 gs 1 -1 sc (  "foo.seq_b.007") col0 sh gr
 /Times-Bold-iso ff 200.00 scf sf
 12825 5595 m
 gs 1 -1 sc (  "foo.seq_b.008") col0 sh gr
 /Times-Bold-iso ff 200.00 scf sf
 12825 4275 m
 gs 1 -1 sc (FLU \(proc\)) col0 sh gr
 % Polyline
 15.000 slw
 n 12750 4050 m 15975 4050 l 15975 5775 l 12750 5775 l
 cp gs col0 s gr 
 % Polyline
 n 12750 2775 m 15150 2775 l 15150 3900 l 12750 3900 l
 cp gs col0 s gr 
 /Times-Bold-iso ff 200.00 scf sf
 12825 3000 m
 gs 1 -1 sc (Sequencer \(proc\)) col0 sh gr
 /Times-Bold-iso ff 200.00 scf sf
 12825 3300 m
 gs 1 -1 sc (epoch=13) col0 sh gr
 /Times-Bold-iso ff 200.00 scf sf
 12825 3555 m
 gs 1 -1 sc (map=[{"foo", next_file=8,) col0 sh gr
 /Times-Bold-iso ff 200.00 scf sf
 13500 3750 m
 gs 1 -1 sc (next_offset=0}...]) col0 sh gr
 % Polyline
 n 5700 3975 m 5700 4275 l 8250 4275 l 8250 3075 l 7950 3075 l 7950 3975 l
 5700 3975 l  cp gs col0 s gr 
 /Times-Bold-iso ff 200.00 scf sf
 5775 4200 m
 gs 1 -1 sc (server logic) col0 sh gr
 /Times-Bold-iso ff 200.00 scf sf
 5775 3060 m
 gs 1 -1 sc (Append <<123 bytes>>) col0 sh gr
 /Times-Bold-iso ff 200.00 scf sf
 5775 3315 m
 gs 1 -1 sc (to a file with prefix) col0 sh gr
 /Times-Bold-iso ff 200.00 scf sf
 5775 3570 m
 gs 1 -1 sc ("foo".) col0 sh gr
 /Times-Bold-iso ff 200.00 scf sf
 5775 2700 m
 gs 1 -1 sc (CLIENT \(proc\)) col0 sh gr
 % Polyline
 gs  clippath
 5970 3763 m 5970 3915 l 6030 3915 l 6030 3763 l 6030 3763 l 6000 3883 l 5970 3763 l cp
 eoclip
 n 6000 3600 m
 6000 3900 l gs col0 s gr gr
 % arrowhead
 7.500 slw
 n 5970 3763 m 6000 3883 l 6030 3763 l  col0 s
 % Polyline
 15.000 slw
 gs  clippath
 6630 3737 m 6630 3585 l 6570 3585 l 6570 3737 l 6570 3737 l 6600 3617 l 6630 3737 l cp
 eoclip
 n 6600 3900 m 6600 3825 l 6600 3750 l 6600 3675 l
 6600 3600 l gs col0 s gr gr
 % arrowhead
 7.500 slw
 n 6630 3737 m 6600 3617 l 6570 3737 l  col0 s
 /Times-Bold-iso ff 200.00 scf sf
 6675 3900 m
 gs 1 -1 sc (ok) col0 sh gr
 % Polyline
 15.000 slw
 n 5550 4350 m 8325 4350 l 8325 2475 l 5550 2475 l
 cp gs col0 s gr 
 % Polyline
 gs  clippath
 8143 4500 m 8035 4393 l 7993 4435 l 8100 4543 l 8100 4543 l 8037 4437 l 8143 4500 l cp
 eoclip
 n 12525 5175 m 8775 5175 l
 8025 4425 l gs col0 s gr gr
 % arrowhead
 7.500 slw
 n 8143 4500 m 8037 4437 l 8100 4543 l  col0 s
 /Times-Bold-iso ff 200.00 scf sf
 11625 5100 m
 gs 1 -1 sc (ok) col0 sh gr
 % Polyline
 15.000 slw
 gs  clippath
 5970 4663 m 5970 4815 l 6030 4815 l 6030 4663 l 6030 4663 l 6000 4783 l 5970 4663 l cp
 eoclip
 n 6000 4425 m
 6000 4800 l gs col0 s gr gr
 % arrowhead
 7.500 slw
 n 5970 4663 m 6000 4783 l 6030 4663 l  col0 s
 % Polyline
 15.000 slw
 gs  clippath
 6630 4562 m 6630 4410 l 6570 4410 l 6570 4562 l 6570 4562 l 6600 4442 l 6630 4562 l cp
 eoclip
 n 6600 4800 m 6600 4650 l 6600 4575 l 6600 4500 l
 6600 4425 l gs col0 s gr gr
 % arrowhead
 7.500 slw
 n 6630 4562 m 6600 4442 l 6570 4562 l  col0 s
 % Polyline
 15.000 slw
 gs  clippath
 12388 2730 m 12540 2730 l 12540 2670 l 12388 2670 l 12388 2670 l 12508 2700 l 12388 2730 l cp
 eoclip
 n 8475 2700 m
 12525 2700 l gs col0 s gr gr
 % arrowhead
 7.500 slw
 n 12388 2730 m 12508 2700 l 12388 2670 l  col0 s
 % Polyline
 15.000 slw
 gs  clippath
 8612 2970 m 8460 2970 l 8460 3030 l 8612 3030 l 8612 3030 l 8492 3000 l 8612 2970 l cp
 eoclip
 n 12525 3000 m
 8475 3000 l gs col0 s gr gr
 % arrowhead
 7.500 slw
 n 8612 2970 m 8492 3000 l 8612 3030 l  col0 s
 % Polyline
 15.000 slw
 gs  clippath
 8612 3645 m 8460 3645 l 8460 3705 l 8612 3705 l 8612 3705 l 8492 3675 l 8612 3645 l cp
 eoclip
 n 12525 6900 m 12000 6900 l 12000 3675 l
 8475 3675 l gs col0 s gr gr
 % arrowhead
 7.500 slw
 n 8612 3645 m 8492 3675 l 8612 3705 l  col0 s
 % Polyline
 15.000 slw
 gs  clippath
 12388 3330 m 12540 3330 l 12540 3270 l 12388 3270 l 12388 3270 l 12508 3300 l 12388 3330 l cp
 eoclip
 n 8475 3975 m 12300 3975 l 12300 3300 l
 12525 3300 l gs col0 s gr gr
 % arrowhead
 7.500 slw
 n 12388 3330 m 12508 3300 l 12388 3270 l  col0 s
 % Polyline
 15.000 slw
 gs  clippath
 12388 4905 m 12540 4905 l 12540 4845 l 12388 4845 l 12388 4845 l 12508 4875 l 12388 4905 l cp
 eoclip
 n 8250 4425 m 8700 4875 l
 12525 4875 l gs col0 s gr gr
 % arrowhead
 7.500 slw
 n 12388 4905 m 12508 4875 l 12388 4845 l  col0 s
 % Polyline
 15.000 slw
 n 12675 2400 m 16050 2400 l 16050 7875 l 12675 7875 l
 cp gs col0 s gr 
 % Polyline
 gs  clippath
 8612 4245 m 8460 4245 l 8460 4305 l 8612 4305 l 8612 4305 l 8492 4275 l 8612 4245 l cp
 eoclip
 n 12525 3600 m 12375 3600 l 12375 4275 l
 8475 4275 l gs col0 s gr gr
 % arrowhead
 7.500 slw
 n 8612 4245 m 8492 4275 l 8612 4305 l  col0 s
 /Times-Bold-iso ff 200.00 scf sf
 8850 5625 m
 gs 1 -1 sc (Write to FLU B -> ok) col0 sh gr
 /Times-Bold-iso ff 200.00 scf sf
 8850 6135 m
 gs 1 -1 sc (Write to FLU C -> ok) col0 sh gr
 /Times-Bold-iso ff 200.00 scf sf
 8550 2625 m
 gs 1 -1 sc (Request 123 bytes, prefix="foo", epoch=12) col0 sh gr
 /Times-Bold-iso ff 200.00 scf sf
 11100 2925 m
 gs 1 -1 sc ({bad_epoch,13}) col4 sh gr
 /Times-Bold-iso ff 200.00 scf sf
 10875 3600 m
 gs 1 -1 sc ({ok, proj=...}) col0 sh gr
 /Times-Bold-iso ff 200.00 scf sf
 8550 4500 m
 gs 1 -1 sc (Write <<123 bytes>> to) col0 sh gr
 /Times-Bold-iso ff 200.00 scf sf
 8550 4755 m
 gs 1 -1 sc (file="foo.seq_a.008", offset=0) col0 sh gr
 /Times-Bold-iso ff 200.00 scf sf
 13575 2625 m
 gs 1 -1 sc (Server A) col0 sh gr
 /Times-Bold-iso ff 200.00 scf sf
 8550 3900 m
 gs 1 -1 sc (Req. 123 bytes, prefix="foo", epoch=13) col0 sh gr
 /Times-Roman-iso ff 166.67 scf sf
 6075 3825 m
 gs 1 -1 sc (1) col16 sh gr
 /Times-Roman-iso ff 166.67 scf sf
 6075 4650 m
 gs 1 -1 sc (2) col16 sh gr
 /Times-Roman-iso ff 166.67 scf sf
 6675 4650 m
 gs 1 -1 sc (3) col16 sh gr
 /Times-Roman-iso ff 166.67 scf sf
 10950 2850 m
 gs 1 -1 sc (5) col16 sh gr
 /Times-Roman-iso ff 166.67 scf sf
 10725 3525 m
 gs 1 -1 sc (7) col16 sh gr
 /Times-Bold-iso ff 200.00 scf sf
 9375 4200 m
 gs 1 -1 sc (file="foo.seq_a.008", offset=0) col0 sh gr
 /Times-Roman-iso ff 166.67 scf sf
 8400 3225 m
 gs 1 -1 sc (6) col16 sh gr
 /Times-Roman-iso ff 166.67 scf sf
 8400 2625 m
 gs 1 -1 sc (4) col16 sh gr
 /Times-Roman-iso ff 166.67 scf sf
 6675 3750 m
 gs 1 -1 sc (16) col16 sh gr
 /Times-Roman-iso ff 166.67 scf sf
 8400 3900 m
 gs 1 -1 sc (8) col16 sh gr
 /Times-Roman-iso ff 166.67 scf sf
 9225 4200 m
 gs 1 -1 sc (9) col16 sh gr
 /Times-Roman-iso ff 166.67 scf sf
 8400 4500 m
 gs 1 -1 sc (10) col16 sh gr
 /Times-Roman-iso ff 166.67 scf sf
 8475 5625 m
 gs 1 -1 sc (12,13) col16 sh gr
 /Times-Roman-iso ff 166.67 scf sf
 8475 6075 m
 gs 1 -1 sc (14,15) col16 sh gr
 /Times-Roman-iso ff 166.67 scf sf
 11400 5100 m
 gs 1 -1 sc (11) col16 sh gr
 % Polyline
 15.000 slw
 gs  clippath
 12388 6630 m 12540 6630 l 12540 6570 l 12388 6570 l 12388 6570 l 12508 6600 l 12388 6630 l cp
 eoclip
 n 8475 3300 m 12150 3300 l 12150 6600 l
 12525 6600 l gs col0 s gr gr
 % arrowhead
 7.500 slw
 n 12388 6630 m 12508 6600 l 12388 6570 l  col0 s
 % here ends figure;
 pagefooter
 showpage
 %%Trailer
 %EOF
--- a/doc/src.high-level/high-level-chain-mgr.tex
+++ b/doc/src.high-level/high-level-chain-mgr.tex
--- a/doc/src.high-level/high-level-machi.tex
+++ b/doc/src.high-level/high-level-machi.tex
--- a/doc/src.high-level/read-flow.eps
+++ b/doc/src.high-level/read-flow.eps
@ -1,145 +0,0 @@
 %!PS-Adobe-3.0 EPSF-2.0
 %%BoundingBox: 0 0 420.000000 166.599991
 %%Creator: mscgen 0.18
 %%EndComments
 0.700000 0.700000 scale
 0 0 moveto
 0 238 lineto
 600 238 lineto
 600 0 lineto
 closepath
 clip
 %PageTrailer
 %Page: 1 1
 /Helvetica findfont
 10 scalefont
 setfont
 /Helvetica findfont
 12 scalefont
 setfont
 0 238 translate
 /mtrx matrix def
 /ellipse
  { /endangle exch def
    /startangle exch def
    /ydia exch def
    /xdia exch def
    /y exch def
    /x exch def
    /savematrix mtrx currentmatrix def
    x y translate
    xdia 2 div ydia 2 div scale
    1 -1 scale
    0 0 1 startangle endangle arc
    savematrix setmatrix
 } def
 (client) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup dup newpath 75 -17 moveto 2 div neg 0 rmoveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 0.000000 0.000000 0.000000 setrgbcolor
 75 -15 moveto dup stringwidth pop 2 div neg 0 rmoveto show
 (Projection) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup dup newpath 225 -17 moveto 2 div neg 0 rmoveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 0.000000 0.000000 0.000000 setrgbcolor
 225 -15 moveto dup stringwidth pop 2 div neg 0 rmoveto show
 (ProjStore_C) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup dup newpath 375 -17 moveto 2 div neg 0 rmoveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 0.000000 0.000000 0.000000 setrgbcolor
 375 -15 moveto dup stringwidth pop 2 div neg 0 rmoveto show
 (FLU_C) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup dup newpath 525 -17 moveto 2 div neg 0 rmoveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 0.000000 0.000000 0.000000 setrgbcolor
 525 -15 moveto dup stringwidth pop 2 div neg 0 rmoveto show
 newpath 75 -22 moveto 75 -49 lineto stroke
 newpath 225 -22 moveto 225 -49 lineto stroke
 newpath 375 -22 moveto 375 -49 lineto stroke
 newpath 525 -22 moveto 525 -49 lineto stroke
 newpath 75 -35 moveto 225 -35 lineto stroke
 newpath 225 -35 moveto 215 -41 lineto stroke
 (get current) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup newpath 122 -33 moveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 0.000000 0.000000 0.000000 setrgbcolor
 122 -33 moveto show
 newpath 75 -49 moveto 75 -76 lineto stroke
 newpath 225 -49 moveto 225 -76 lineto stroke
 newpath 375 -49 moveto 375 -76 lineto stroke
 newpath 525 -49 moveto 525 -76 lineto stroke
 newpath 225 -62 moveto 75 -62 lineto stroke
 newpath 75 -62 moveto 85 -68 lineto stroke
 (ok, #12...) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup newpath 126 -60 moveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 0.000000 0.000000 0.000000 setrgbcolor
 126 -60 moveto show
 newpath 75 -76 moveto 75 -103 lineto stroke
 newpath 225 -76 moveto 225 -103 lineto stroke
 newpath 375 -76 moveto 375 -103 lineto stroke
 newpath 525 -76 moveto 525 -103 lineto stroke
 newpath 75 -89 moveto 525 -89 lineto stroke
 newpath 525 -89 moveto 515 -95 lineto stroke
 (read "foo.seq_a.009" offset=447 bytes=123 epoch=12) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup newpath 157 -87 moveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 0.000000 0.000000 0.000000 setrgbcolor
 157 -87 moveto show
 newpath 75 -103 moveto 75 -130 lineto stroke
 newpath 225 -103 moveto 225 -130 lineto stroke
 newpath 375 -103 moveto 375 -130 lineto stroke
 newpath 525 -103 moveto 525 -130 lineto stroke
 newpath 525 -116 moveto 75 -116 lineto stroke
 newpath 75 -116 moveto 85 -122 lineto stroke
 1.000000 0.000000 0.000000 setrgbcolor
 (bad_epoch, 13) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup newpath 261 -114 moveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 1.000000 0.000000 0.000000 setrgbcolor
 261 -114 moveto show
 0.000000 0.000000 0.000000 setrgbcolor
 newpath 75 -130 moveto 75 -157 lineto stroke
 newpath 225 -130 moveto 225 -157 lineto stroke
 newpath 375 -130 moveto 375 -157 lineto stroke
 newpath 525 -130 moveto 525 -157 lineto stroke
 newpath 75 -143 moveto 375 -143 lineto stroke
 newpath 375 -143 moveto 365 -149 lineto stroke
 (get epoch #13) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup newpath 187 -141 moveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 0.000000 0.000000 0.000000 setrgbcolor
 187 -141 moveto show
 newpath 75 -157 moveto 75 -184 lineto stroke
 newpath 225 -157 moveto 225 -184 lineto stroke
 newpath 375 -157 moveto 375 -184 lineto stroke
 newpath 525 -157 moveto 525 -184 lineto stroke
 newpath 375 -170 moveto 75 -170 lineto stroke
 newpath 75 -170 moveto 85 -176 lineto stroke
 (ok, #13...) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup newpath 201 -168 moveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 0.000000 0.000000 0.000000 setrgbcolor
 201 -168 moveto show
 newpath 75 -184 moveto 75 -211 lineto stroke
 newpath 225 -184 moveto 225 -211 lineto stroke
 newpath 375 -184 moveto 375 -211 lineto stroke
 newpath 525 -184 moveto 525 -211 lineto stroke
 newpath 75 -197 moveto 525 -197 lineto stroke
 newpath 525 -197 moveto 515 -203 lineto stroke
 (read "foo.seq_a.009" offset=447 bytes=123 epoch=13) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup newpath 157 -195 moveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 0.000000 0.000000 0.000000 setrgbcolor
 157 -195 moveto show
 newpath 75 -211 moveto 75 -238 lineto stroke
 newpath 225 -211 moveto 225 -238 lineto stroke
 newpath 375 -211 moveto 375 -238 lineto stroke
 newpath 525 -211 moveto 525 -238 lineto stroke
 newpath 525 -224 moveto 75 -224 lineto stroke
 newpath 75 -224 moveto 85 -230 lineto stroke
 (ok, <<...123...>>) dup stringwidth
 1.000000 1.000000 1.000000 setrgbcolor
 pop dup newpath 257 -222 moveto 0 rlineto 0 11 rlineto neg 0 rlineto closepath fill
 0.000000 0.000000 0.000000 setrgbcolor
 257 -222 moveto show
--- a/doc/src.high-level/sigplanconf.cls
+++ b/doc/src.high-level/sigplanconf.cls
--- a/ebin/.gitignore
+++ b/ebin/.gitignore
@ -1,2 +0,0 @@
 *.beam
 *.app
--- a/edoc/.gitignore
+++ b/edoc/.gitignore
@ -0,0 +1 @@
 tmp.*
--- a/edoc/edoc-info
+++ b/edoc/edoc-info
@ -0,0 +1,13 @@
 %% encoding: UTF-8
 {application,machi}.
 {packages,[]}.
 {modules,[machi_admin_util,machi_app,machi_basho_bench_driver,
          machi_chain_manager1,machi_chain_repair,machi_chash,machi_cinfo,
          machi_config,machi_cr_client,machi_csum_table,machi_dt,
          machi_file_proxy,machi_file_proxy_sup,machi_fitness,machi_flu1,
          machi_flu1_client,machi_flu_filename_mgr,machi_flu_metadata_mgr,
          machi_flu_metadata_mgr_sup,machi_flu_psup,machi_flu_sup,
          machi_lifecycle_mgr,machi_merkle_tree,machi_pb_high_client,
          machi_pb_translate,machi_plist,machi_projection,
          machi_projection_store,machi_proxy_flu1_client,machi_sup,machi_util,
          machi_yessir_client]}.
--- a/edoc/erlang.png
+++ b/edoc/erlang.png
--- a/edoc/index.html
+++ b/edoc/index.html
@ -0,0 +1,17 @@
 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
 <html>
 <head>
 <title>The machi application</title>
 </head>
 <frameset cols="20%,80%">
 <frame src="modules-frame.html" name="modulesFrame" title="">
 <frame src="overview-summary.html" name="overviewFrame" title="">
 <noframes>
 <h2>This page uses frames</h2>
 <p>Your browser does not accept frames.
 <br>You should go to the <a href="overview-summary.html">non-frame version</a> instead.
 </p>
 </noframes>
 </frameset>
 </html>
--- a/edoc/machi_admin_util.html
+++ b/edoc/machi_admin_util.html
@ -0,0 +1,50 @@
 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
 <html>
 <head>
 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
 <title>Module machi_admin_util</title>
 <link rel="stylesheet" type="text/css" href="stylesheet.css" title="EDoc">
 </head>
 <body bgcolor="white">
 <div class="navbar"><a name="#navbar_top"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <hr>
 <h1>Module machi_admin_util</h1>
 <ul class="index"><li><a href="#description">Description</a></li><li><a href="#index">Function Index</a></li><li><a href="#functions">Function Details</a></li></ul>Machi chain replication administration utilities.
 <h2><a name="description">Description</a></h2>Machi chain replication administration utilities.
 <h2><a name="index">Function Index</a></h2>
 <table width="100%" border="1" cellspacing="0" cellpadding="2" summary="function index"><tr><td valign="top"><a href="#verify_file_checksums_local-3">verify_file_checksums_local/3</a></td><td></td></tr>
 <tr><td valign="top"><a href="#verify_file_checksums_local-4">verify_file_checksums_local/4</a></td><td></td></tr>
 <tr><td valign="top"><a href="#verify_file_checksums_remote-3">verify_file_checksums_remote/3</a></td><td></td></tr>
 <tr><td valign="top"><a href="#verify_file_checksums_remote-4">verify_file_checksums_remote/4</a></td><td></td></tr>
 </table>
 <h2><a name="functions">Function Details</a></h2>
 <h3 class="function"><a name="verify_file_checksums_local-3">verify_file_checksums_local/3</a></h3>
 <div class="spec">
 <p><tt>verify_file_checksums_local(Sock1::port(), EpochID::<a href="machi_dt.html#type-epoch_id">machi_dt:epoch_id()</a>, Path::binary() | list()) -&gt; {ok, [tuple()]} | {error, term()}</tt><br></p>
 </div>
 <h3 class="function"><a name="verify_file_checksums_local-4">verify_file_checksums_local/4</a></h3>
 <div class="spec">
 <p><tt>verify_file_checksums_local(Host::<a href="machi_dt.html#type-inet_host">machi_dt:inet_host()</a>, TcpPort::<a href="machi_dt.html#type-inet_port">machi_dt:inet_port()</a>, EpochID::<a href="machi_dt.html#type-epoch_id">machi_dt:epoch_id()</a>, Path::binary() | list()) -&gt; {ok, [tuple()]} | {error, term()}</tt><br></p>
 </div>
 <h3 class="function"><a name="verify_file_checksums_remote-3">verify_file_checksums_remote/3</a></h3>
 <div class="spec">
 <p><tt>verify_file_checksums_remote(Sock1::port(), EpochID::<a href="machi_dt.html#type-epoch_id">machi_dt:epoch_id()</a>, File::binary() | list()) -&gt; {ok, [tuple()]} | {error, term()}</tt><br></p>
 </div>
 <h3 class="function"><a name="verify_file_checksums_remote-4">verify_file_checksums_remote/4</a></h3>
 <div class="spec">
 <p><tt>verify_file_checksums_remote(Host::<a href="machi_dt.html#type-inet_host">machi_dt:inet_host()</a>, TcpPort::<a href="machi_dt.html#type-inet_port">machi_dt:inet_port()</a>, EpochID::<a href="machi_dt.html#type-epoch_id">machi_dt:epoch_id()</a>, File::binary() | list()) -&gt; {ok, [tuple()]} | {error, term()}</tt><br></p>
 </div>
 <hr>
 <div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <p><i>Generated by EDoc, Dec 8 2015, 21:37:38.</i></p>
 </body>
 </html>
--- a/edoc/machi_app.html
+++ b/edoc/machi_app.html
@ -0,0 +1,42 @@
 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
 <html>
 <head>
 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
 <title>Module machi_app</title>
 <link rel="stylesheet" type="text/css" href="stylesheet.css" title="EDoc">
 </head>
 <body bgcolor="white">
 <div class="navbar"><a name="#navbar_top"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <hr>
 <h1>Module machi_app</h1>
 <ul class="index"><li><a href="#description">Description</a></li><li><a href="#index">Function Index</a></li><li><a href="#functions">Function Details</a></li></ul>Start the top-level supervisor for the Machi application.
 <p><b>Behaviours:</b> <a href="application.html"><tt>application</tt></a>.</p>
 <h2><a name="description">Description</a></h2><p>Start the top-level supervisor for the Machi application.</p>
  See <a href="machi_flu_psup.html"><code>machi_flu_psup</code></a> for an illustration of the entire Machi
  application process structure.
 <h2><a name="index">Function Index</a></h2>
 <table width="100%" border="1" cellspacing="0" cellpadding="2" summary="function index"><tr><td valign="top"><a href="#start-2">start/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#stop-1">stop/1</a></td><td></td></tr>
 </table>
 <h2><a name="functions">Function Details</a></h2>
 <h3 class="function"><a name="start-2">start/2</a></h3>
 <div class="spec">
 <p><tt>start(StartType, StartArgs) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="stop-1">stop/1</a></h3>
 <div class="spec">
 <p><tt>stop(State) -&gt; any()</tt></p>
 </div>
 <hr>
 <div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <p><i>Generated by EDoc, Dec 8 2015, 21:37:38.</i></p>
 </body>
 </html>
--- a/edoc/machi_basho_bench_driver.html
+++ b/edoc/machi_basho_bench_driver.html
@ -0,0 +1,64 @@
 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
 <html>
 <head>
 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
 <title>Module machi_basho_bench_driver</title>
 <link rel="stylesheet" type="text/css" href="stylesheet.css" title="EDoc">
 </head>
 <body bgcolor="white">
 <div class="navbar"><a name="#navbar_top"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <hr>
 <h1>Module machi_basho_bench_driver</h1>
 <ul class="index"><li><a href="#description">Description</a></li><li><a href="#index">Function Index</a></li><li><a href="#functions">Function Details</a></li></ul>A simple basho_bench driver for Machi.
 <h2><a name="description">Description</a></h2><p>A simple basho_bench driver for Machi</p>
  <p>Basho_bench was originally developed to stress test key-value
  stores (as was YCSB and several other mechmarking tools).  A person
  can consider the UNIX file system to be a key-value store and thus
  use basho_bench to measure its performance under a certain
  workload.  Machi is a bit different than most KV stores in that the
  client has no direct control over the keys -- Machi servers always
  assign the keys.  The schemes typically used by basho_bench &amp; YCSB  
 to use/mimic key naming conventions used internally ... are  
 difficult to adapt to Machi.</p>
  <p>So, we'll try to manage key reading by using a common ETS table  
 that is populated with:</p>
  <p>1. Key: <code>non_neg_integer()`
  2. Value: The `{File,Offset,Size}` for a chunk previously written.
  At startup time, basho_bench can use the `list_files</code> and
  <code>checksum_list</code> API operations to fetch all of the
  <code>{File,Offset,Size}` tuples that currently reside in the cluster.
  Also, optionally (?), each new `append</code> operation by the b_b driver  
 could add new entries to this ETS table.</p>
  Now we can use various integer-centric key generators that are
  already bundled with basho_bench.
 <h2><a name="index">Function Index</a></h2>
 <table width="100%" border="1" cellspacing="0" cellpadding="2" summary="function index"><tr><td valign="top"><a href="#new-1">new/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#run-4">run/4</a></td><td></td></tr>
 </table>
 <h2><a name="functions">Function Details</a></h2>
 <h3 class="function"><a name="new-1">new/1</a></h3>
 <div class="spec">
 <p><tt>new(Id) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="run-4">run/4</a></h3>
 <div class="spec">
 <p><tt>run(X1, KeyGen, ValueGen, M) -&gt; any()</tt></p>
 </div>
 <hr>
 <div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <p><i>Generated by EDoc, Dec 8 2015, 21:37:38.</i></p>
 </body>
 </html>
--- a/edoc/machi_chain_manager1.html
+++ b/edoc/machi_chain_manager1.html
@ -0,0 +1,289 @@
 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
 <html>
 <head>
 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
 <title>Module machi_chain_manager1</title>
 <link rel="stylesheet" type="text/css" href="stylesheet.css" title="EDoc">
 </head>
 <body bgcolor="white">
 <div class="navbar"><a name="#navbar_top"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <hr>
 <h1>Module machi_chain_manager1</h1>
 <ul class="index"><li><a href="#description">Description</a></li><li><a href="#index">Function Index</a></li><li><a href="#functions">Function Details</a></li></ul>The Machi chain manager, Guardian of all things related to  
 Chain Replication state, status, and data replica safety.
 <p><b>Behaviours:</b> <a href="gen_server.html"><tt>gen_server</tt></a>.</p>
 <h2><a name="description">Description</a></h2><p>The Machi chain manager, Guardian of all things related to  
 Chain Replication state, status, and data replica safety.</p>
  <p>The Chain Manager is responsible for managing the state of Machi's  
 "Chain Replication" state.  This role is roughly analogous to the  
 "Riak Core" application inside of Riak, which takes care of  
 coordinating replica placement and replica repair.</p>
  <p>For each primitive data server in the cluster, a Machi FLU, there  
 is a Chain Manager process that manages its FLU's role within the  
 Machi cluster's Chain Replication scheme.  Each Chain Manager  
 process executes locally and independently to manage the  
 distributed state of a single Machi Chain Replication chain.</p>
  Machi's Chain Manager process performs similar tasks as Riak Core's
  claimant.  However, Machi has several active Chain Manager
  processes, one per FLU server, instead of a single active process
  like Core's claimant.  Each Chain Manager process acts
  independently; each is constrained so that it will reach consensus
  via independent computation &amp; action.
 <h2><a name="index">Function Index</a></h2>
 <table width="100%" border="1" cellspacing="0" cellpadding="2" summary="function index"><tr><td valign="top"><a href="#chain_state_transition_is_sane-6">chain_state_transition_is_sane/6</a></td><td>Check if a projection transition is sane &amp; safe: we assume  
 that the caller has checked basic projection data structure contents.</td></tr>
 <tr><td valign="top"><a href="#code_change-3">code_change/3</a></td><td></td></tr>
 <tr><td valign="top"><a href="#format_status-2">format_status/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#get_ps-2">get_ps/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#get_remember_down_list-0">get_remember_down_list/0</a></td><td></td></tr>
 <tr><td valign="top"><a href="#handle_call-3">handle_call/3</a></td><td></td></tr>
 <tr><td valign="top"><a href="#handle_cast-2">handle_cast/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#handle_info-2">handle_info/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#has_not_sane-2">has_not_sane/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#init-1">init/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#init_remember_down_list-0">init_remember_down_list/0</a></td><td></td></tr>
 <tr><td valign="top"><a href="#make_chmgr_regname-1">make_chmgr_regname/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#mk-3">mk/3</a></td><td>Create a 2-tuple that describes how <code>UPI1</code> + <code>Repair1</code> are
       transformed into <code>UPI2</code> in a chain state change.</td></tr>
 <tr><td valign="top"><a href="#perhaps_call-5">perhaps_call/5</a></td><td></td></tr>
 <tr><td valign="top"><a href="#ping-1">ping/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#projection_transitions_are_sane-2">projection_transitions_are_sane/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#scan_dir-4">scan_dir/4</a></td><td></td></tr>
 <tr><td valign="top"><a href="#set_active-2">set_active/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#set_chain_members-2">set_chain_members/2</a></td><td>Set chain members list.</td></tr>
 <tr><td valign="top"><a href="#set_chain_members-6">set_chain_members/6</a></td><td></td></tr>
 <tr><td valign="top"><a href="#simple_chain_state_transition_is_sane-3">simple_chain_state_transition_is_sane/3</a></td><td>A simple technique for checking chain state transition safety.</td></tr>
 <tr><td valign="top"><a href="#simple_chain_state_transition_is_sane-5">simple_chain_state_transition_is_sane/5</a></td><td>Simple check if a projection transition is sane &amp; safe: we assume  
 that the caller has checked basic projection data structure contents.</td></tr>
 <tr><td valign="top"><a href="#start_link-2">start_link/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#start_link-3">start_link/3</a></td><td></td></tr>
 <tr><td valign="top"><a href="#stop-1">stop/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#strip_dbg2-1">strip_dbg2/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#terminate-2">terminate/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#test_calc_projection-2">test_calc_projection/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#test_read_latest_public_projection-2">test_read_latest_public_projection/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#test_write_public_projection-2">test_write_public_projection/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#trigger_react_to_env-1">trigger_react_to_env/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#update_remember_down_list-1">update_remember_down_list/1</a></td><td></td></tr>
 </table>
 <h2><a name="functions">Function Details</a></h2>
 <h3 class="function"><a name="chain_state_transition_is_sane-6">chain_state_transition_is_sane/6</a></h3>
 <div class="spec">
 <p><tt>chain_state_transition_is_sane(Author1, UPI1, Repair1, Author2, UPI2, RelativeToServer) -&gt; any()</tt></p>
 </div><p><p>Check if a projection transition is sane &amp; safe: we assume  
 that the caller has checked basic projection data structure contents.</p>
  NOTE: The return value convention is <code>true</code> for sane/safe and <code>term() /=
  true</code> for any unsafe/insane value.  This function (and its callee
  functions) are the only functions (throughout all of the chain state
  transition sanity checking functions) that is allowed to return <code>false</code>.</p>
 <h3 class="function"><a name="code_change-3">code_change/3</a></h3>
 <div class="spec">
 <p><tt>code_change(OldVsn, S, Extra) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="format_status-2">format_status/2</a></h3>
 <div class="spec">
 <p><tt>format_status(Opt, X2) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="get_ps-2">get_ps/2</a></h3>
 <div class="spec">
 <p><tt>get_ps(Projection_v1, Acc) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="get_remember_down_list-0">get_remember_down_list/0</a></h3>
 <div class="spec">
 <p><tt>get_remember_down_list() -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="handle_call-3">handle_call/3</a></h3>
 <div class="spec">
 <p><tt>handle_call(Call, From, S) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="handle_cast-2">handle_cast/2</a></h3>
 <div class="spec">
 <p><tt>handle_cast(Cast, S) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="handle_info-2">handle_info/2</a></h3>
 <div class="spec">
 <p><tt>handle_info(Msg, Ch_mgr) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="has_not_sane-2">has_not_sane/2</a></h3>
 <div class="spec">
 <p><tt>has_not_sane(Projection_v1, Acc) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="init-1">init/1</a></h3>
 <div class="spec">
 <p><tt>init(X1) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="init_remember_down_list-0">init_remember_down_list/0</a></h3>
 <div class="spec">
 <p><tt>init_remember_down_list() -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="make_chmgr_regname-1">make_chmgr_regname/1</a></h3>
 <div class="spec">
 <p><tt>make_chmgr_regname(A) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="mk-3">mk/3</a></h3>
 <div class="spec">
 <p><tt>mk(UPI1, Repair1, UPI2) -&gt; any()</tt></p>
 </div><p><p>Create a 2-tuple that describes how <code>UPI1</code> + <code>Repair1</code> are
       transformed into <code>UPI2</code> in a chain state change.</p>
  <p>The 1st part of the 2-tuple is a list of <code>keep</code> and <code>del</code> instructions,
  relative to the items in UPI1 and whether they are present (<code>keep</code>) or
  absent (<code>del</code>) in <code>UPI2</code>.</p>
  The 2nd part of the 2-tuple is <code>list(non_neg_integer()|error)</code> that
  describes the relative order of items in <code>Repair1</code> that appear in
  <code>UPI2</code>.  The <code>error</code> atom is used to denote items not present in
  <code>Repair1</code>.</p>
 <h3 class="function"><a name="perhaps_call-5">perhaps_call/5</a></h3>
 <div class="spec">
 <p><tt>perhaps_call(ProxyPid, MyName, Partitions, FLU, DoIt) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="ping-1">ping/1</a></h3>
 <div class="spec">
 <p><tt>ping(Pid) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="projection_transitions_are_sane-2">projection_transitions_are_sane/2</a></h3>
 <div class="spec">
 <p><tt>projection_transitions_are_sane(Ps, RelativeToServer) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="scan_dir-4">scan_dir/4</a></h3>
 <div class="spec">
 <p><tt>scan_dir(Dir, FileFilterFun, FoldEachFun, FoldEachAcc) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="set_active-2">set_active/2</a></h3>
 <div class="spec">
 <p><tt>set_active(Pid, Boolean) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="set_chain_members-2">set_chain_members/2</a></h3>
 <div class="spec">
 <p><tt>set_chain_members(Pid, MembersDict) -&gt; any()</tt></p>
 </div><p><p>Set chain members list.</p>
  NOTE: This implementation is a bit brittle, in that an author with
  higher rank may try to re-suggest the old membership list if it
  races with an author of lower rank.  For now, we suggest calling
  set_chain_members() first on the author of highest rank and finish
  with lowest rank, i.e. name z* first, name a* last.</p>
 <h3 class="function"><a name="set_chain_members-6">set_chain_members/6</a></h3>
 <div class="spec">
 <p><tt>set_chain_members(Pid, ChainName, OldEpoch, CMode, MembersDict, Witness_list) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="simple_chain_state_transition_is_sane-3">simple_chain_state_transition_is_sane/3</a></h3>
 <div class="spec">
 <p><tt>simple_chain_state_transition_is_sane(UPI1, Repair1, UPI2) -&gt; any()</tt></p>
 </div><p><p>A simple technique for checking chain state transition safety.</p>
  <p>Math tells us that any change state <code>UPI1</code> plus <code>Repair1</code> to state
  <code>UPI2</code> is OK as long as <code>UPI2</code> is a concatenation of some
  order-preserving combination from <code>UPI1</code> with some order-preserving
  combination from <code>Repair1</code>.</p>
  <pre>    Good_UPI2s = [ X ++ Y || X &lt;- machi_util:ordered_combinations(UPI1),
                             Y &lt;- machi_util:ordered_combinations(Repair1)]</pre>
  <p>Rather than creating that list and then checking if <code>UPI2</code> is in
  it, we try a <code>diff</code>-like technique to check for basic state
  transition safety.  See docs for <a href="#mk-3"><code>mk/3</code></a> for more detail.</p>
  <pre>  2&gt; machi_chain_manager1:mk([a,b], [], [a]).
  {[keep,del],[]}        %% good transition
  3&gt; machi_chain_manager1:mk([a,b], [], [b,a]).
  {[del,keep],[]}        %% bad transition: too few 'keep' for UPI2's length 2
  4&gt; machi_chain_manager1:mk([a,b], [c,d,e], [a,d]).
  {[keep,del],[2]}       %% good transition
  5&gt; machi_chain_manager1:mk([a,b], [c,d,e], [a,bogus]).
  {[keep,del],[error]}   %% bad transition: 'bogus' not in Repair1</pre></p>
 <h3 class="function"><a name="simple_chain_state_transition_is_sane-5">simple_chain_state_transition_is_sane/5</a></h3>
 <div class="spec">
 <p><tt>simple_chain_state_transition_is_sane(Author1, UPI1, Repair1, Author2, UPI2) -&gt; any()</tt></p>
 </div><p><p>Simple check if a projection transition is sane &amp; safe: we assume  
 that the caller has checked basic projection data structure contents.</p>
  NOTE: The return value convention is <code>true</code> for sane/safe and
  <code>term() /= true</code> for any unsafe/insane value.</p>
 <h3 class="function"><a name="start_link-2">start_link/2</a></h3>
 <div class="spec">
 <p><tt>start_link(MyName, MembersDict) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="start_link-3">start_link/3</a></h3>
 <div class="spec">
 <p><tt>start_link(MyName, MembersDict, MgrOpts) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="stop-1">stop/1</a></h3>
 <div class="spec">
 <p><tt>stop(Pid) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="strip_dbg2-1">strip_dbg2/1</a></h3>
 <div class="spec">
 <p><tt>strip_dbg2(P) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="terminate-2">terminate/2</a></h3>
 <div class="spec">
 <p><tt>terminate(Reason, S) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="test_calc_projection-2">test_calc_projection/2</a></h3>
 <div class="spec">
 <p><tt>test_calc_projection(Pid, KeepRunenvP) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="test_read_latest_public_projection-2">test_read_latest_public_projection/2</a></h3>
 <div class="spec">
 <p><tt>test_read_latest_public_projection(Pid, ReadRepairP) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="test_write_public_projection-2">test_write_public_projection/2</a></h3>
 <div class="spec">
 <p><tt>test_write_public_projection(Pid, Proj) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="trigger_react_to_env-1">trigger_react_to_env/1</a></h3>
 <div class="spec">
 <p><tt>trigger_react_to_env(Pid) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="update_remember_down_list-1">update_remember_down_list/1</a></h3>
 <div class="spec">
 <p><tt>update_remember_down_list(FLU) -&gt; any()</tt></p>
 </div>
 <hr>
 <div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <p><i>Generated by EDoc, Dec 8 2015, 21:37:38.</i></p>
 </body>
 </html>
--- a/edoc/machi_chain_repair.html
+++ b/edoc/machi_chain_repair.html
@ -0,0 +1,91 @@
 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
 <html>
 <head>
 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
 <title>Module machi_chain_repair</title>
 <link rel="stylesheet" type="text/css" href="stylesheet.css" title="EDoc">
 </head>
 <body bgcolor="white">
 <div class="navbar"><a name="#navbar_top"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <hr>
 <h1>Module machi_chain_repair</h1>
 <ul class="index"><li><a href="#description">Description</a></li><li><a href="#index">Function Index</a></li><li><a href="#functions">Function Details</a></li></ul>Perform "chain repair", i.e., resynchronization of Machi file  
 contents and metadata as servers are (re-)added to the chain.
 <h2><a name="description">Description</a></h2><p>Perform "chain repair", i.e., resynchronization of Machi file  
 contents and metadata as servers are (re-)added to the chain.</p>
  <p>The implementation here is a very basic one, and is probably a bit
  slower than the original "demo day" implementation at
  <a href="https://github.com/basho/machi/blob/master/prototype/demo-day-hack/file0_repair_server.escript" target="_top"><tt>https://github.com/basho/machi/blob/master/prototype/demo-day-hack/file0_repair_server.escript</tt></a></p>
  <p>It's so easy to bikeshed this into a 1 year programming exercise.</p>
  <p>General TODO note: There are a lot of areas for exploiting parallelism here.
  I've set the bikeshed aside for now, but "make repair faster" has a
  lot of room for exploiting concurrency, overlapping reads &amp; writes,  
 etc etc.  There are also lots of different trade-offs to make with  
 regard to RAM use vs. disk use.</p>
  <p>There's no reason why repair can't be done:</p>
  <ol>
  <li> Repair in parallel across multiple repairees ... Optimization.
  </li>
  <li> Repair multiple byte ranges concurrently ... Optimization.
  </li>
  <li><p> Use bigger chunks than the client originally used to write the file     
 ... Optimization ... but it would be the easiest to implement, e.g. use     
 constant-sized 4MB chunks.  Unfortuntely, it would also destroy     
 the ability to verify here that the chunk checksums are correct     
 *and* also propagate the correct checksum metadata to the     
 destination FLU.</p>
     As an additional optimization, add a bit of #2 to start the next
     read while the current write is still in progress.
  </li>
  <li> The current method centralizes the "smarts" required to compare
     checksum differences ... move some computation to each FLU, then use
     a Merkle- or other-compression-style scheme to reduce the amount of
     data sent across a network.
  </li>
  </ol>
  <p>Most/all of this could be executed in parallel on each FLU relative to  
 its own files.  Then, in another TODO option, perhaps build a Merkle tree  
 or other summary of the local files and send that data structure to the  
 repair coordinator.</p>
  <p>Also, as another TODO note, repair_both_present() in the  
 prototype/demo-day code uses an optimization of calculating the MD5  
 checksum of the chunk checksum data as it arrives, and if the two MD5s  
 match, then we consider the two files in sync.  If there isn't a match,  
 then we sort the lines and try another MD5, and if they match, then we're  
 in sync.  In theory, that's lower overhead than the procedure used here.</p>
  NOTE that one reason I chose the "directives list" method is to have an
  option, later, of choosing to repair a subset of repairee FLUs if there
  is a big discrepency between out of sync files: e.g., if FLU x has N
  bytes out of sync but FLU y has 50N bytes out of sync, then it's likely
  better to repair x only so that x can return to the UPI list quickly.
  Also, in the event that all repairees are roughly comparably out of sync,
  then the repair network traffic can be minimized by reading each chunk
  only once.
 <h2><a name="index">Function Index</a></h2>
 <table width="100%" border="1" cellspacing="0" cellpadding="2" summary="function index"><tr><td valign="top"><a href="#repair-7">repair/7</a></td><td></td></tr>
 </table>
 <h2><a name="functions">Function Details</a></h2>
 <h3 class="function"><a name="repair-7">repair/7</a></h3>
 <div class="spec">
 <p><tt>repair(ConsistencyMode, Src, Repairing, UPI, MembersDict, ETS, Opts) -&gt; any()</tt></p>
 </div>
 <hr>
 <div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <p><i>Generated by EDoc, Dec 8 2015, 21:37:38.</i></p>
 </body>
 </html>
--- a/edoc/machi_chash.html
+++ b/edoc/machi_chash.html
@ -0,0 +1,171 @@
 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
 <html>
 <head>
 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
 <title>Module machi_chash</title>
 <link rel="stylesheet" type="text/css" href="stylesheet.css" title="EDoc">
 </head>
 <body bgcolor="white">
 <div class="navbar"><a name="#navbar_top"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <hr>
 <h1>Module machi_chash</h1>
 <ul class="index"><li><a href="#description">Description</a></li><li><a href="#types">Data Types</a></li><li><a href="#index">Function Index</a></li><li><a href="#functions">Function Details</a></li></ul>Consistent hashing library.
 <h2><a name="description">Description</a></h2><p>Consistent hashing library.  Also known as "random slicing".</p>
  This code was originally from the Hibari DB source code at
  <a href="https://github.com/hibari" target="_top"><tt>https://github.com/hibari</tt></a>
 <h2><a name="types">Data Types</a></h2>
 <h3 class="typedecl"><a name="type-float_map">float_map()</a></h3>
 <p><tt>float_map() = [{<a href="#type-owner_name">owner_name()</a>, float()}]</tt></p>
 <p>  A float map subdivides the unit interval, starting at 0.0, to
  partitions that are assigned to various owners.  The sum of all
  floats must be exactly 1.0 (or close enough for floating point
  purposes).</p>
 <h3 class="typedecl"><a name="type-float_tree">float_tree()</a></h3>
 <p><b>abstract datatype</b>: <tt>float_tree()</tt></p>
 <p>  We can't use gb_trees:tree() because 'nil' (the empty tree) is
  never valid in our case.  But teaching Dialyzer that is difficult.</p>
 <h3 class="typedecl"><a name="type-nextfloat_list">nextfloat_list()</a></h3>
 <p><tt>nextfloat_list() = [{float(), <a href="#type-brick">brick()</a>}]</tt></p>
 <p>A nextfloat_list
  differs from a float_map in two respects: 1) nextfloat_list contains
  tuples with the brick name in 2nd position, 2) the float() at each
  position I_n &gt; I_m, for all n, m such that n &gt; m.
  For example, a nextfloat_list of the float_map example above,
  [{0.25, {br1, nd1}}, {0.75, {br2, nd1}}, {1.0, {br3, nd1}].</p>
 <h3 class="typedecl"><a name="type-owner_int_range">owner_int_range()</a></h3>
 <p><tt>owner_int_range() = {<a href="#type-owner_name">owner_name()</a>, non_neg_integer(), non_neg_integer()}</tt></p>
 <p>  Used when "prettying" a float map.</p>
 <h3 class="typedecl"><a name="type-owner_name">owner_name()</a></h3>
 <p><tt>owner_name() = term()</tt></p>
 <p>  Owner for a range on the unit interval.  We are agnostic about its
  type.</p>
 <h3 class="typedecl"><a name="type-owner_weight">owner_weight()</a></h3>
 <p><tt>owner_weight() = {<a href="#type-owner_name">owner_name()</a>, <a href="#type-weight">weight()</a>}</tt></p>
 <h3 class="typedecl"><a name="type-owner_weight_list">owner_weight_list()</a></h3>
 <p><tt>owner_weight_list() = [<a href="#type-owner_weight">owner_weight()</a>]</tt></p>
 <p>  A owner_weight_list is a definition of brick assignments over the
  unit interval [0.0, 1.0].  The sum of all floats must be 1.0.  For
  example, [{{br1,nd1}, 0.25}, {{br2,nd1}, 0.5}, {{br3,nd1}, 0.25}].</p>
 <h3 class="typedecl"><a name="type-weight">weight()</a></h3>
 <p><tt>weight() = non_neg_integer()</tt></p>
 <p>  For this library, a weight is an integer which specifies the
  capacity of a "owner" relative to other owners.  For example, if
  owner A with a weight of 10, and if owner B has a weight of 20,
  then B will be assigned twice as much of the unit interval as A.</p>
 <h2><a name="index">Function Index</a></h2>
 <table width="100%" border="1" cellspacing="0" cellpadding="2" summary="function index"><tr><td valign="top"><a href="#hash_binary_via_float_map-2">hash_binary_via_float_map/2</a></td><td>Query a float map with a binary (inefficient).</td></tr>
 <tr><td valign="top"><a href="#hash_binary_via_float_tree-2">hash_binary_via_float_tree/2</a></td><td>Query a float tree with a binary.</td></tr>
 <tr><td valign="top"><a href="#make_demo_map1-0">make_demo_map1/0</a></td><td>Create a sample float map.</td></tr>
 <tr><td valign="top"><a href="#make_demo_map2-0">make_demo_map2/0</a></td><td>Create a sample float map.</td></tr>
 <tr><td valign="top"><a href="#make_float_map-1">make_float_map/1</a></td><td>Create a float map, based on a basic owner weight list.</td></tr>
 <tr><td valign="top"><a href="#make_float_map-2">make_float_map/2</a></td><td>Create a float map, based on an older float map and a new weight  
 list.</td></tr>
 <tr><td valign="top"><a href="#make_tree-1">make_tree/1</a></td><td>Create a float tree, which is the rapid lookup data structure
  for consistent hash queries.</td></tr>
 <tr><td valign="top"><a href="#pretty_with_integers-2">pretty_with_integers/2</a></td><td>Make a pretty/human-friendly version of a float map that describes
  integer ranges between 1 and <code>Scale</code>.</td></tr>
 <tr><td valign="top"><a href="#pretty_with_integers-3">pretty_with_integers/3</a></td><td>Make a pretty/human-friendly version of a float map (based
  upon a float map created from <code>OldWeights</code> and <code>NewWeights</code>) that
  describes integer ranges between 1 and <code>Scale</code>.</td></tr>
 <tr><td valign="top"><a href="#query_tree-2">query_tree/2</a></td><td>Low-level function for querying a float tree: the (floating
  point) point within the unit interval.</td></tr>
 <tr><td valign="top"><a href="#sum_map_weights-1">sum_map_weights/1</a></td><td>Create a human-friendly summary of a float map.</td></tr>
 <tr><td valign="top"><a href="#zzz_usage_details-0">zzz_usage_details/0</a></td><td>Various usage examples, see source code below this function
     for full details.</td></tr>
 </table>
 <h2><a name="functions">Function Details</a></h2>
 <h3 class="function"><a name="hash_binary_via_float_map-2">hash_binary_via_float_map/2</a></h3>
 <div class="spec">
 <p><tt>hash_binary_via_float_map(Key::binary(), Map::<a href="#type-float_map">float_map()</a>) -&gt; {float(), <a href="#type-owner_name">owner_name()</a>}</tt><br></p>
 </div><p>Query a float map with a binary (inefficient).</p>
 <h3 class="function"><a name="hash_binary_via_float_tree-2">hash_binary_via_float_tree/2</a></h3>
 <div class="spec">
 <p><tt>hash_binary_via_float_tree(Key::binary(), Tree::<a href="#type-float_tree">float_tree()</a>) -&gt; {float(), <a href="#type-owner_name">owner_name()</a>}</tt><br></p>
 </div><p>Query a float tree with a binary.</p>
 <h3 class="function"><a name="make_demo_map1-0">make_demo_map1/0</a></h3>
 <div class="spec">
 <p><tt>make_demo_map1() -&gt; <a href="#type-float_map">float_map()</a></tt><br></p>
 </div><p>Create a sample float map.</p>
 <h3 class="function"><a name="make_demo_map2-0">make_demo_map2/0</a></h3>
 <div class="spec">
 <p><tt>make_demo_map2() -&gt; <a href="#type-float_map">float_map()</a></tt><br></p>
 </div><p>Create a sample float map.</p>
 <h3 class="function"><a name="make_float_map-1">make_float_map/1</a></h3>
 <div class="spec">
 <p><tt>make_float_map(NewOwnerWeights::<a href="#type-owner_weight_list">owner_weight_list()</a>) -&gt; <a href="#type-float_map">float_map()</a></tt><br></p>
 </div><p>Create a float map, based on a basic owner weight list.</p>
 <h3 class="function"><a name="make_float_map-2">make_float_map/2</a></h3>
 <div class="spec">
 <p><tt>make_float_map(OldFloatMap::<a href="#type-float_map">float_map()</a>, NewOwnerWeights::<a href="#type-owner_weight_list">owner_weight_list()</a>) -&gt; <a href="#type-float_map">float_map()</a></tt><br></p>
 </div><p><p>Create a float map, based on an older float map and a new weight  
 list.</p>
  The weights in the new weight list may be different than (or the
  same as) whatever weights were used to make the older float map.</p>
 <h3 class="function"><a name="make_tree-1">make_tree/1</a></h3>
 <div class="spec">
 <p><tt>make_tree(Map::<a href="#type-float_map">float_map()</a>) -&gt; <a href="#type-float_tree">float_tree()</a></tt><br></p>
 </div><p>Create a float tree, which is the rapid lookup data structure
  for consistent hash queries.</p>
 <h3 class="function"><a name="pretty_with_integers-2">pretty_with_integers/2</a></h3>
 <div class="spec">
 <p><tt>pretty_with_integers(Map::<a href="#type-float_map">float_map()</a>, Scale::integer()) -&gt; [<a href="#type-owner_int_range">owner_int_range()</a>]</tt><br></p>
 </div><p>Make a pretty/human-friendly version of a float map that describes
  integer ranges between 1 and <code>Scale</code>.</p>
 <h3 class="function"><a name="pretty_with_integers-3">pretty_with_integers/3</a></h3>
 <div class="spec">
 <p><tt>pretty_with_integers(OldWeights::<a href="#type-owner_weight_list">owner_weight_list()</a>, NewWeights::<a href="#type-owner_weight_list">owner_weight_list()</a>, Scale::integer()) -&gt; [<a href="#type-owner_int_range">owner_int_range()</a>]</tt><br></p>
 </div><p>Make a pretty/human-friendly version of a float map (based
  upon a float map created from <code>OldWeights</code> and <code>NewWeights</code>) that
  describes integer ranges between 1 and <code>Scale</code>.</p>
 <h3 class="function"><a name="query_tree-2">query_tree/2</a></h3>
 <div class="spec">
 <p><tt>query_tree(Val::float(), Tree::<a href="#type-float_tree">float_tree()</a>) -&gt; {float(), <a href="#type-owner_name">owner_name()</a>}</tt><br></p>
 </div><p>Low-level function for querying a float tree: the (floating
  point) point within the unit interval.</p>
 <h3 class="function"><a name="sum_map_weights-1">sum_map_weights/1</a></h3>
 <div class="spec">
 <p><tt>sum_map_weights(Map::<a href="#type-float_map">float_map()</a>) -&gt; {{per_owner, <a href="#type-float_map">float_map()</a>}, {weight_sum, float()}}</tt><br></p>
 </div><p><p>Create a human-friendly summary of a float map.</p>
  The two parts of the summary are: a per-owner total of the unit
  interval range(s) owned by each owner, and a total sum of all
  per-owner ranges (which should be 1.0 but is not enforced).</p>
 <h3 class="function"><a name="zzz_usage_details-0">zzz_usage_details/0</a></h3>
 <div class="spec">
 <p><tt>zzz_usage_details() -&gt; any()</tt></p>
 </div><p>Various usage examples, see source code below this function
     for full details.</p>
 <hr>
 <div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <p><i>Generated by EDoc, Dec 8 2015, 21:37:38.</i></p>
 </body>
 </html>
--- a/edoc/machi_cinfo.html
+++ b/edoc/machi_cinfo.html
@ -0,0 +1,82 @@
 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
 <html>
 <head>
 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
 <title>Module machi_cinfo</title>
 <link rel="stylesheet" type="text/css" href="stylesheet.css" title="EDoc">
 </head>
 <body bgcolor="white">
 <div class="navbar"><a name="#navbar_top"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <hr>
 <h1>Module machi_cinfo</h1>
 <ul class="index"><li><a href="#description">Description</a></li><li><a href="#index">Function Index</a></li><li><a href="#functions">Function Details</a></li></ul>cluster_info callback module for machi specific information
  gathering.
 <h2><a name="description">Description</a></h2>cluster_info callback module for machi specific information
  gathering.
 <h2><a name="index">Function Index</a></h2>
 <table width="100%" border="1" cellspacing="0" cellpadding="2" summary="function index"><tr><td valign="top"><a href="#chain_manager-1">chain_manager/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#cluster_info_generator_funs-0">cluster_info_generator_funs/0</a></td><td></td></tr>
 <tr><td valign="top"><a href="#cluster_info_init-0">cluster_info_init/0</a></td><td></td></tr>
 <tr><td valign="top"><a href="#dump-0">dump/0</a></td><td></td></tr>
 <tr><td valign="top"><a href="#fitness-1">fitness/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#flu1-1">flu1/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#private_projection-1">private_projection/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#public_projection-1">public_projection/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#register-0">register/0</a></td><td></td></tr>
 </table>
 <h2><a name="functions">Function Details</a></h2>
 <h3 class="function"><a name="chain_manager-1">chain_manager/1</a></h3>
 <div class="spec">
 <p><tt>chain_manager(FluName::atom()) -&gt; term()</tt><br></p>
 </div>
 <h3 class="function"><a name="cluster_info_generator_funs-0">cluster_info_generator_funs/0</a></h3>
 <div class="spec">
 <p><tt>cluster_info_generator_funs() -&gt; [{string(), fun((pid()) -&gt; ok)}]</tt><br></p>
 </div>
 <h3 class="function"><a name="cluster_info_init-0">cluster_info_init/0</a></h3>
 <div class="spec">
 <p><tt>cluster_info_init() -&gt; ok</tt><br></p>
 </div>
 <h3 class="function"><a name="dump-0">dump/0</a></h3>
 <div class="spec">
 <p><tt>dump() -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="fitness-1">fitness/1</a></h3>
 <div class="spec">
 <p><tt>fitness(FluName::atom()) -&gt; term()</tt><br></p>
 </div>
 <h3 class="function"><a name="flu1-1">flu1/1</a></h3>
 <div class="spec">
 <p><tt>flu1(FluName::atom()) -&gt; [{atom(), term()}]</tt><br></p>
 </div>
 <h3 class="function"><a name="private_projection-1">private_projection/1</a></h3>
 <div class="spec">
 <p><tt>private_projection(FluName::atom()) -&gt; [{atom(), term()}]</tt><br></p>
 </div>
 <h3 class="function"><a name="public_projection-1">public_projection/1</a></h3>
 <div class="spec">
 <p><tt>public_projection(FluName::atom()) -&gt; [{atom(), term()}]</tt><br></p>
 </div>
 <h3 class="function"><a name="register-0">register/0</a></h3>
 <div class="spec">
 <p><tt>register() -&gt; ok</tt><br></p>
 </div>
 <hr>
 <div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <p><i>Generated by EDoc, Dec 8 2015, 21:37:38.</i></p>
 </body>
 </html>
--- a/edoc/machi_config.html
+++ b/edoc/machi_config.html
@ -0,0 +1,44 @@
 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
 <html>
 <head>
 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
 <title>Module machi_config</title>
 <link rel="stylesheet" type="text/css" href="stylesheet.css" title="EDoc">
 </head>
 <body bgcolor="white">
 <div class="navbar"><a name="#navbar_top"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <hr>
 <h1>Module machi_config</h1>
 <ul class="index"><li><a href="#description">Description</a></li><li><a href="#index">Function Index</a></li><li><a href="#functions">Function Details</a></li></ul>Configuration consulting utilities.
 <h2><a name="description">Description</a></h2>Configuration consulting utilities. Some conventions:
  - The function name should match with exact configuration
    name in <code>app.config</code> or <code>advanced.config</code> of <code>machi</code> section.
  - The default value of that configuration is expected to be in
    cuttlefish schema file. Otherwise some macro in headers may
    be chosen.
  - Documentation of the configuration is supposed to be written
    in cuttlefish schema file, rather than @doc section of the function.
  - spec of the function should be written.
  - Returning <code>undefined</code> is strongly discouraged. Return some default
    value instead.
  - <code>application:get_env/3</code> is recommended. See <code>max_file_size/0</code> for
    example.
 <h2><a name="index">Function Index</a></h2>
 <table width="100%" border="1" cellspacing="0" cellpadding="2" summary="function index"><tr><td valign="top"><a href="#max_file_size-0">max_file_size/0</a></td><td></td></tr>
 </table>
 <h2><a name="functions">Function Details</a></h2>
 <h3 class="function"><a name="max_file_size-0">max_file_size/0</a></h3>
 <div class="spec">
 <p><tt>max_file_size() -&gt; pos_integer()</tt><br></p>
 </div>
 <hr>
 <div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <p><i>Generated by EDoc, Dec 8 2015, 21:37:38.</i></p>
 </body>
 </html>
--- a/edoc/machi_cr_client.html
+++ b/edoc/machi_cr_client.html
@ -0,0 +1,263 @@
 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
 <html>
 <head>
 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
 <title>Module machi_cr_client</title>
 <link rel="stylesheet" type="text/css" href="stylesheet.css" title="EDoc">
 </head>
 <body bgcolor="white">
 <div class="navbar"><a name="#navbar_top"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <hr>
 <h1>Module machi_cr_client</h1>
 <ul class="index"><li><a href="#description">Description</a></li><li><a href="#index">Function Index</a></li><li><a href="#functions">Function Details</a></li></ul>Erlang API for the Machi client-implemented Chain Replication  
 (CORFU-style) protocol.
 <p><b>Behaviours:</b> <a href="gen_server.html"><tt>gen_server</tt></a>.</p>
 <h2><a name="description">Description</a></h2><p>Erlang API for the Machi client-implemented Chain Replication  
 (CORFU-style) protocol.</p>
  <p>See also the docs for <a href="machi_flu1_client.html"><code>machi_flu1_client</code></a> for additional  
 details on data types and operation descriptions.</p>
  <p>The API here is much simpler than the <a href="machi_flu1_client.html"><code>machi_flu1_client</code></a> or
  <a href="machi_proxy_flu1_client.html"><code>machi_proxy_flu1_client</code></a> APIs.  This module's API is a
  proposed simple-but-complete form for clients who are not
  interested in being an active participant in a Machi cluster and to
  have the responsibility for Machi internals, i.e., client-side
  Chain Replication, client-side read repair, client-side tracking of
  internal Machi epoch &amp; projection changes, etc.</p>
  <p>This client is implemented as a long-lived Erlang process using
  <code>gen_server</code>-style OTP code practice.  A naive client can expect
  that this process will manage all transient TCP session
  disconnections and Machi chain reconfigurations.  This client's
  efforts are best-effort and can require some time to retry
  operations in certain failure cases, i.e., up to several seconds
  during a Machi projection &amp; epoch change when a new server is  
 added to the chain.</p>
  <p>Doc TODO: Once this API stabilizes, add all relevant data type details  
 to the EDoc here.</p>
  <h4><a name="Missing_API_features">Missing API features</a></h4>
  <p>So far, there is one missing client API feature that ought to be  
 added to Machi in the near future: more flexible checksum  
 management.</p>
  <p>Add a <code>source</code> annotation to all checksums to indicate where the  
 checksum was calculated.  For example,</p>
  <ul>
  <li> Calculated by client that performed the original chunk append,
  </li>
  <li> Calculated by the 1st Machi server to receive an
       un-checksummed append request
  </li>
  <li> Re-calculated by Machi to manage fewer checksums of blocks of
       data larger than the original client-specified chunks.
  </li>
  </ul>
  <p>Client-side checksums would be the "strongest" type of  
 checksum, meaning that any data corruption (of the original  
 data and/or of the checksum itself) can be detected after the  
 client-side calculation.  There are too many horror stories on  
 The Net about IP PDUs that are corrupted but unnoticed due to  
 weak TCP checksums, buggy hardware, buggy OS drivers, etc.  
 Checksum versioning is also desirable if/when the current checksum  
 implementation changes from SHA-1 to something else.</p>
  <h4><a name="Implementation_notes">Implementation notes</a></h4>
  <p>The major operation processing is implemented in a state machine-like
  manner.  Before attempting an operation <code>X</code>, there's an initial
  operation <code>pre-X</code> that takes care of updating the epoch id,
  restarting client protocol proxies, and if there's any server
  instability (e.g. some server is wedged), then insert some sleep
  time.  When the chain appears to have stabilized, then we try the <code>X</code>  
 operation again.</p>
  <p>Function name for the <code>pre-X</code> stuff is usually <code>X()</code>, and the
  function name for the <code>X</code> stuff is usually <code>X2()</code>.  (I.e., the <code>X</code>
  stuff follows after <code>pre-X</code> and therefore has a <code>2</code> suffix on the  
 function name.)</p>
  In the case of read repair, there are two stages: find the value to
  perform the repair, then perform the repair writes.  In the case of
  the repair writes, the <code>pre-X</code> function is named <code>read_repair3()</code>,
  and the <code>X</code> function is named <code>read_repair4()</code>.
 <h2><a name="index">Function Index</a></h2>
 <table width="100%" border="1" cellspacing="0" cellpadding="2" summary="function index"><tr><td valign="top"><a href="#append_chunk-3">append_chunk/3</a></td><td>Append a chunk (binary- or iolist-style) of data to a file
  with <code>Prefix</code>.</td></tr>
 <tr><td valign="top"><a href="#append_chunk-4">append_chunk/4</a></td><td>Append a chunk (binary- or iolist-style) of data to a file
  with <code>Prefix</code>.</td></tr>
 <tr><td valign="top"><a href="#append_chunk_extra-4">append_chunk_extra/4</a></td><td>Append a chunk (binary- or iolist-style) of data to a file
  with <code>Prefix</code>.</td></tr>
 <tr><td valign="top"><a href="#append_chunk_extra-5">append_chunk_extra/5</a></td><td>Append a chunk (binary- or iolist-style) of data to a file
  with <code>Prefix</code>.</td></tr>
 <tr><td valign="top"><a href="#checksum_list-2">checksum_list/2</a></td><td>Fetch the list of chunk checksums for <code>File</code>.</td></tr>
 <tr><td valign="top"><a href="#checksum_list-3">checksum_list/3</a></td><td>Fetch the list of chunk checksums for <code>File</code>.</td></tr>
 <tr><td valign="top"><a href="#code_change-3">code_change/3</a></td><td></td></tr>
 <tr><td valign="top"><a href="#handle_call-3">handle_call/3</a></td><td></td></tr>
 <tr><td valign="top"><a href="#handle_cast-2">handle_cast/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#handle_info-2">handle_info/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#init-1">init/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#list_files-1">list_files/1</a></td><td>Fetch the list of all files on the remote FLU.</td></tr>
 <tr><td valign="top"><a href="#list_files-2">list_files/2</a></td><td>Fetch the list of all files on the remote FLU.</td></tr>
 <tr><td valign="top"><a href="#quit-1">quit/1</a></td><td>Quit &amp; close the connection to remote FLU and stop our
  proxy process.</td></tr>
 <tr><td valign="top"><a href="#read_chunk-5">read_chunk/5</a></td><td>Read a chunk of data of size <code>Size</code> from <code>File</code> at <code>Offset</code>.</td></tr>
 <tr><td valign="top"><a href="#read_chunk-6">read_chunk/6</a></td><td>Read a chunk of data of size <code>Size</code> from <code>File</code> at <code>Offset</code>.</td></tr>
 <tr><td valign="top"><a href="#start_link-1">start_link/1</a></td><td>Start a local, long-lived process that will be our steady
  &amp; reliable communication proxy with the fickle &amp; flaky
  remote Machi server.</td></tr>
 <tr><td valign="top"><a href="#start_link-2">start_link/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#terminate-2">terminate/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#trim_chunk-4">trim_chunk/4</a></td><td>Trim a chunk of data of size <code>Size</code> from <code>File</code> at <code>Offset</code>.</td></tr>
 <tr><td valign="top"><a href="#trim_chunk-5">trim_chunk/5</a></td><td>Trim a chunk of data of size <code>Size</code> from <code>File</code> at <code>Offset</code>.</td></tr>
 <tr><td valign="top"><a href="#write_chunk-4">write_chunk/4</a></td><td>Write a chunk of data (that has already been
  allocated/sequenced by an earlier append_chunk_extra() call) to
  <code>File</code> at <code>Offset</code>.</td></tr>
 <tr><td valign="top"><a href="#write_chunk-5">write_chunk/5</a></td><td>Read a chunk of data of size <code>Size</code> from <code>File</code> at <code>Offset</code>.</td></tr>
 </table>
 <h2><a name="functions">Function Details</a></h2>
 <h3 class="function"><a name="append_chunk-3">append_chunk/3</a></h3>
 <div class="spec">
 <p><tt>append_chunk(PidSpec, Prefix, Chunk) -&gt; any()</tt></p>
 </div><p>Append a chunk (binary- or iolist-style) of data to a file
  with <code>Prefix</code>.</p>
 <h3 class="function"><a name="append_chunk-4">append_chunk/4</a></h3>
 <div class="spec">
 <p><tt>append_chunk(PidSpec, Prefix, Chunk, Timeout) -&gt; any()</tt></p>
 </div><p>Append a chunk (binary- or iolist-style) of data to a file
  with <code>Prefix</code>.</p>
 <h3 class="function"><a name="append_chunk_extra-4">append_chunk_extra/4</a></h3>
 <div class="spec">
 <p><tt>append_chunk_extra(PidSpec, Prefix, Chunk, ChunkExtra) -&gt; any()</tt></p>
 </div><p>Append a chunk (binary- or iolist-style) of data to a file
  with <code>Prefix</code>.</p>
 <h3 class="function"><a name="append_chunk_extra-5">append_chunk_extra/5</a></h3>
 <div class="spec">
 <p><tt>append_chunk_extra(PidSpec, Prefix, Chunk, ChunkExtra, Timeout0) -&gt; any()</tt></p>
 </div><p>Append a chunk (binary- or iolist-style) of data to a file
  with <code>Prefix</code>.</p>
 <h3 class="function"><a name="checksum_list-2">checksum_list/2</a></h3>
 <div class="spec">
 <p><tt>checksum_list(PidSpec, File) -&gt; any()</tt></p>
 </div><p>Fetch the list of chunk checksums for <code>File</code>.</p>
 <h3 class="function"><a name="checksum_list-3">checksum_list/3</a></h3>
 <div class="spec">
 <p><tt>checksum_list(PidSpec, File, Timeout0) -&gt; any()</tt></p>
 </div><p>Fetch the list of chunk checksums for <code>File</code>.</p>
 <h3 class="function"><a name="code_change-3">code_change/3</a></h3>
 <div class="spec">
 <p><tt>code_change(OldVsn, S, Extra) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="handle_call-3">handle_call/3</a></h3>
 <div class="spec">
 <p><tt>handle_call(Request, From, S) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="handle_cast-2">handle_cast/2</a></h3>
 <div class="spec">
 <p><tt>handle_cast(Msg, S) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="handle_info-2">handle_info/2</a></h3>
 <div class="spec">
 <p><tt>handle_info(Info, S) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="init-1">init/1</a></h3>
 <div class="spec">
 <p><tt>init(X1) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="list_files-1">list_files/1</a></h3>
 <div class="spec">
 <p><tt>list_files(PidSpec) -&gt; any()</tt></p>
 </div><p>Fetch the list of all files on the remote FLU.</p>
 <h3 class="function"><a name="list_files-2">list_files/2</a></h3>
 <div class="spec">
 <p><tt>list_files(PidSpec, Timeout0) -&gt; any()</tt></p>
 </div><p>Fetch the list of all files on the remote FLU.</p>
 <h3 class="function"><a name="quit-1">quit/1</a></h3>
 <div class="spec">
 <p><tt>quit(PidSpec) -&gt; any()</tt></p>
 </div><p>Quit &amp; close the connection to remote FLU and stop our
  proxy process.</p>
 <h3 class="function"><a name="read_chunk-5">read_chunk/5</a></h3>
 <div class="spec">
 <p><tt>read_chunk(PidSpec, File, Offset, Size, Opts) -&gt; any()</tt></p>
 </div><p>Read a chunk of data of size <code>Size</code> from <code>File</code> at <code>Offset</code>.</p>
 <h3 class="function"><a name="read_chunk-6">read_chunk/6</a></h3>
 <div class="spec">
 <p><tt>read_chunk(PidSpec, File, Offset, Size, Opts, Timeout0) -&gt; any()</tt></p>
 </div><p>Read a chunk of data of size <code>Size</code> from <code>File</code> at <code>Offset</code>.</p>
 <h3 class="function"><a name="start_link-1">start_link/1</a></h3>
 <div class="spec">
 <p><tt>start_link(P_srvr_list) -&gt; any()</tt></p>
 </div><p>Start a local, long-lived process that will be our steady
  &amp; reliable communication proxy with the fickle &amp; flaky
  remote Machi server.</p>
 <h3 class="function"><a name="start_link-2">start_link/2</a></h3>
 <div class="spec">
 <p><tt>start_link(P_srvr_list, Opts) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="terminate-2">terminate/2</a></h3>
 <div class="spec">
 <p><tt>terminate(Reason, State) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="trim_chunk-4">trim_chunk/4</a></h3>
 <div class="spec">
 <p><tt>trim_chunk(PidSpec, File, Offset, Size) -&gt; any()</tt></p>
 </div><p>Trim a chunk of data of size <code>Size</code> from <code>File</code> at <code>Offset</code>.</p>
 <h3 class="function"><a name="trim_chunk-5">trim_chunk/5</a></h3>
 <div class="spec">
 <p><tt>trim_chunk(PidSpec, File, Offset, Size, Timeout0) -&gt; any()</tt></p>
 </div><p>Trim a chunk of data of size <code>Size</code> from <code>File</code> at <code>Offset</code>.</p>
 <h3 class="function"><a name="write_chunk-4">write_chunk/4</a></h3>
 <div class="spec">
 <p><tt>write_chunk(PidSpec, File, Offset, Chunk) -&gt; any()</tt></p>
 </div><p>Write a chunk of data (that has already been
  allocated/sequenced by an earlier append_chunk_extra() call) to
  <code>File</code> at <code>Offset</code>.</p>
 <h3 class="function"><a name="write_chunk-5">write_chunk/5</a></h3>
 <div class="spec">
 <p><tt>write_chunk(PidSpec, File, Offset, Chunk, Timeout0) -&gt; any()</tt></p>
 </div><p>Read a chunk of data of size <code>Size</code> from <code>File</code> at <code>Offset</code>.</p>
 <hr>
 <div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <p><i>Generated by EDoc, Dec 8 2015, 21:37:38.</i></p>
 </body>
 </html>
--- a/edoc/machi_csum_table.html
+++ b/edoc/machi_csum_table.html
@ -0,0 +1,186 @@
 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
 <html>
 <head>
 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
 <title>Module machi_csum_table</title>
 <link rel="stylesheet" type="text/css" href="stylesheet.css" title="EDoc">
 </head>
 <body bgcolor="white">
 <div class="navbar"><a name="#navbar_top"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <hr>
 <h1>Module machi_csum_table</h1>
 <ul class="index"><li><a href="#types">Data Types</a></li><li><a href="#index">Function Index</a></li><li><a href="#functions">Function Details</a></li></ul>
 <h2><a name="types">Data Types</a></h2>
 <h3 class="typedecl"><a name="type-byte_sequence">byte_sequence()</a></h3>
 <p><tt>byte_sequence() = {Offset::non_neg_integer(), Size::pos_integer() | infinity}</tt></p>
 <h3 class="typedecl"><a name="type-table">table()</a></h3>
 <p><tt>table() = #machi_csum_table{}</tt></p>
 <h2><a name="index">Function Index</a></h2>
 <table width="100%" border="1" cellspacing="0" cellpadding="2" summary="function index"><tr><td valign="top"><a href="#all-1">all/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#all_trimmed-2">all_trimmed/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#all_trimmed-3">all_trimmed/3</a></td><td></td></tr>
 <tr><td valign="top"><a href="#any_trimmed-3">any_trimmed/3</a></td><td></td></tr>
 <tr><td valign="top"><a href="#calc_unwritten_bytes-1">calc_unwritten_bytes/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#close-1">close/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#decode_csum_file_entry-1">decode_csum_file_entry/1</a></td><td>Decode a single <code>binary()</code> blob into an
       <code>{Offset,Size,TaggedCSum}</code> tuple.</td></tr>
 <tr><td valign="top"><a href="#delete-1">delete/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#encode_csum_file_entry-3">encode_csum_file_entry/3</a></td><td>Encode <code>Offset + Size + TaggedCSum</code> into an <code>iolist()</code> type for
  internal storage by the FLU.</td></tr>
 <tr><td valign="top"><a href="#encode_csum_file_entry_bin-3">encode_csum_file_entry_bin/3</a></td><td>Encode <code>Offset + Size + TaggedCSum</code> into an <code>binary()</code> type for
  internal storage by the FLU.</td></tr>
 <tr><td valign="top"><a href="#find-3">find/3</a></td><td></td></tr>
 <tr><td valign="top"><a href="#find_leftneighbor-2">find_leftneighbor/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#find_rightneighbor-2">find_rightneighbor/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#foldl_chunks-3">foldl_chunks/3</a></td><td></td></tr>
 <tr><td valign="top"><a href="#open-2">open/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#split_checksum_list_blob_decode-1">split_checksum_list_blob_decode/1</a></td><td>Split a <code>binary()</code> blob of <code>checksum_list</code> data into a list of
  <code>{Offset,Size,TaggedCSum}</code> tuples.</td></tr>
 <tr><td valign="top"><a href="#sync-1">sync/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#trim-3">trim/3</a></td><td></td></tr>
 <tr><td valign="top"><a href="#trim-5">trim/5</a></td><td></td></tr>
 <tr><td valign="top"><a href="#write-4">write/4</a></td><td></td></tr>
 <tr><td valign="top"><a href="#write-6">write/6</a></td><td></td></tr>
 </table>
 <h2><a name="functions">Function Details</a></h2>
 <h3 class="function"><a name="all-1">all/1</a></h3>
 <div class="spec">
 <p><tt>all(Machi_csum_table) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="all_trimmed-2">all_trimmed/2</a></h3>
 <div class="spec">
 <p><tt>all_trimmed(Machi_csum_table::<a href="#type-table">table()</a>, Pos::non_neg_integer()) -&gt; boolean()</tt><br></p>
 </div>
 <h3 class="function"><a name="all_trimmed-3">all_trimmed/3</a></h3>
 <div class="spec">
 <p><tt>all_trimmed(Machi_csum_table::<a href="#type-table">table()</a>, Left::non_neg_integer(), Right::non_neg_integer()) -&gt; boolean()</tt><br></p>
 </div>
 <h3 class="function"><a name="any_trimmed-3">any_trimmed/3</a></h3>
 <div class="spec">
 <p><tt>any_trimmed(CsumT::<a href="#type-table">table()</a>, Offset::pos_integer(), Size::<a href="machi_dt.html#type-chunk_size">machi_dt:chunk_size()</a>) -&gt; boolean()</tt><br></p>
 </div>
 <h3 class="function"><a name="calc_unwritten_bytes-1">calc_unwritten_bytes/1</a></h3>
 <div class="spec">
 <p><tt>calc_unwritten_bytes(Machi_csum_table::<a href="#type-table">table()</a>) -&gt; [<a href="#type-byte_sequence">byte_sequence()</a>]</tt><br></p>
 </div>
 <h3 class="function"><a name="close-1">close/1</a></h3>
 <div class="spec">
 <p><tt>close(Machi_csum_table::<a href="#type-table">table()</a>) -&gt; ok</tt><br></p>
 </div>
 <h3 class="function"><a name="decode_csum_file_entry-1">decode_csum_file_entry/1</a></h3>
 <div class="spec">
 <p><tt>decode_csum_file_entry(Else::binary()) -&gt; error | {<a href="machi_dt.html#type-file_offset">machi_dt:file_offset()</a>, <a href="machi_dt.html#type-chunk_size">machi_dt:chunk_size()</a>, <a href="machi_dt.html#type-chunk_s">machi_dt:chunk_s()</a>}</tt><br></p>
 </div><p><p>Decode a single <code>binary()</code> blob into an
       <code>{Offset,Size,TaggedCSum}</code> tuple.</p>
  <p>The internal encoding (which is currently exposed to the outside world  
 via this function and related ones) is:</p>
  <ul>
  <li> 1 byte: record length
  </li>
  <li> 8 bytes (unsigned big-endian): byte offset
  </li>
  <li> 4 bytes (unsigned big-endian): chunk size
  </li>
  <li> all remaining bytes: tagged checksum (1st byte = type tag)
  </li>
  </ul>
  See <code>machi.hrl</code> for the tagged checksum types, e.g.,
  <code>?CSUM_TAG_NONE</code>.</p>
 <h3 class="function"><a name="delete-1">delete/1</a></h3>
 <div class="spec">
 <p><tt>delete(Machi_csum_table::<a href="#type-table">table()</a>) -&gt; ok</tt><br></p>
 </div>
 <h3 class="function"><a name="encode_csum_file_entry-3">encode_csum_file_entry/3</a></h3>
 <div class="spec">
 <p><tt>encode_csum_file_entry(Offset::<a href="machi_dt.html#type-file_offset">machi_dt:file_offset()</a>, Size::<a href="machi_dt.html#type-chunk_size">machi_dt:chunk_size()</a>, TaggedCSum::<a href="machi_dt.html#type-chunk_s">machi_dt:chunk_s()</a>) -&gt; iolist()</tt><br></p>
 </div><p>Encode <code>Offset + Size + TaggedCSum</code> into an <code>iolist()</code> type for
  internal storage by the FLU.</p>
 <h3 class="function"><a name="encode_csum_file_entry_bin-3">encode_csum_file_entry_bin/3</a></h3>
 <div class="spec">
 <p><tt>encode_csum_file_entry_bin(Offset::<a href="machi_dt.html#type-file_offset">machi_dt:file_offset()</a>, Size::<a href="machi_dt.html#type-chunk_size">machi_dt:chunk_size()</a>, TaggedCSum::<a href="machi_dt.html#type-chunk_s">machi_dt:chunk_s()</a>) -&gt; binary()</tt><br></p>
 </div><p>Encode <code>Offset + Size + TaggedCSum</code> into an <code>binary()</code> type for
  internal storage by the FLU.</p>
 <h3 class="function"><a name="find-3">find/3</a></h3>
 <div class="spec">
 <p><tt>find(Machi_csum_table::<a href="#type-table">table()</a>, Offset::<a href="machi_dt.html#type-file_offset">machi_dt:file_offset()</a>, Size::<a href="machi_dt.html#type-file_size">machi_dt:file_size()</a>) -&gt; [{<a href="machi_dt.html#type-file_offset">machi_dt:file_offset()</a>, <a href="machi_dt.html#type-file_size">machi_dt:file_size()</a>, <a href="machi_dt.html#type-chunk_csum">machi_dt:chunk_csum()</a> | trimmed}]</tt><br></p>
 </div>
 <h3 class="function"><a name="find_leftneighbor-2">find_leftneighbor/2</a></h3>
 <div class="spec">
 <p><tt>find_leftneighbor(CsumT::<a href="#type-table">table()</a>, Offset::non_neg_integer()) -&gt; undefined | {non_neg_integer(), <a href="machi_dt.html#type-chunk_size">machi_dt:chunk_size()</a>, trimmed | <a href="machi_dt.html#type-chunk_csum">machi_dt:chunk_csum()</a>}</tt><br></p>
 </div>
 <h3 class="function"><a name="find_rightneighbor-2">find_rightneighbor/2</a></h3>
 <div class="spec">
 <p><tt>find_rightneighbor(CsumT::<a href="#type-table">table()</a>, Offset::non_neg_integer()) -&gt; undefined | {non_neg_integer(), <a href="machi_dt.html#type-chunk_size">machi_dt:chunk_size()</a>, trimmed | <a href="machi_dt.html#type-chunk_csum">machi_dt:chunk_csum()</a>}</tt><br></p>
 </div>
 <h3 class="function"><a name="foldl_chunks-3">foldl_chunks/3</a></h3>
 <div class="spec">
 <p><tt>foldl_chunks(Fun::fun(({non_neg_integer(), non_neg_integer(), term()}, Acc0::term()) -&gt; Acc::term()), Acc0::term(), Machi_csum_table::<a href="#type-table">table()</a>) -&gt; Acc::term()</tt><br></p>
 </div>
 <h3 class="function"><a name="open-2">open/2</a></h3>
 <div class="spec">
 <p><tt>open(CSumFilename::string(), Opts::<a href="proplists.html#type-proplist">proplists:proplist()</a>) -&gt; {ok, <a href="#type-table">table()</a>} | {error, <a href="file.html#type-posix">file:posix()</a>}</tt><br></p>
 </div>
 <h3 class="function"><a name="split_checksum_list_blob_decode-1">split_checksum_list_blob_decode/1</a></h3>
 <div class="spec">
 <p><tt>split_checksum_list_blob_decode(Bin::binary()) -&gt; {[{<a href="machi_dt.html#type-file_offset">machi_dt:file_offset()</a>, <a href="machi_dt.html#type-chunk_size">machi_dt:chunk_size()</a>, <a href="machi_dt.html#type-chunk_s">machi_dt:chunk_s()</a>}], TrailingJunk::binary()}</tt><br></p>
 </div><p>Split a <code>binary()</code> blob of <code>checksum_list</code> data into a list of
  <code>{Offset,Size,TaggedCSum}</code> tuples.</p>
 <h3 class="function"><a name="sync-1">sync/1</a></h3>
 <div class="spec">
 <p><tt>sync(Machi_csum_table::<a href="#type-table">table()</a>) -&gt; ok | {error, <a href="file.html#type-posix">file:posix()</a>}</tt><br></p>
 </div>
 <h3 class="function"><a name="trim-3">trim/3</a></h3>
 <div class="spec">
 <p><tt>trim(Machi_csum_table::<a href="#type-table">table()</a>, Offset::<a href="machi_dt.html#type-file_offset">machi_dt:file_offset()</a>, Size::<a href="machi_dt.html#type-file_size">machi_dt:file_size()</a>) -&gt; ok | {error, <a href="file.html#type-posix">file:posix()</a>}</tt><br></p>
 </div>
 <h3 class="function"><a name="trim-5">trim/5</a></h3>
 <div class="spec">
 <p><tt>trim(CsumT, Offset, Size, LeftUpdate, RightUpdate) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="write-4">write/4</a></h3>
 <div class="spec">
 <p><tt>write(CsumT::<a href="#type-table">table()</a>, Offset::<a href="machi_dt.html#type-file_offset">machi_dt:file_offset()</a>, Size::<a href="machi_dt.html#type-file_size">machi_dt:file_size()</a>, CSum::<a href="machi_dt.html#type-chunk_csum">machi_dt:chunk_csum()</a> | trimmed) -&gt; ok | {error, trimmed | <a href="file.html#type-posix">file:posix()</a>}</tt><br></p>
 </div>
 <h3 class="function"><a name="write-6">write/6</a></h3>
 <div class="spec">
 <p><tt>write(Machi_csum_table, Offset, Size, CSum, LeftUpdate, RightUpdate) -&gt; any()</tt></p>
 </div>
 <hr>
 <div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <p><i>Generated by EDoc, Dec 8 2015, 21:37:38.</i></p>
 </body>
 </html>
--- a/edoc/machi_dt.html
+++ b/edoc/machi_dt.html
@ -0,0 +1,110 @@
 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
 <html>
 <head>
 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
 <title>Module machi_dt</title>
 <link rel="stylesheet" type="text/css" href="stylesheet.css" title="EDoc">
 </head>
 <body bgcolor="white">
 <div class="navbar"><a name="#navbar_top"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <hr>
 <h1>Module machi_dt</h1>
 <ul class="index"><li><a href="#types">Data Types</a></li></ul>
 <h2><a name="types">Data Types</a></h2>
 <h3 class="typedecl"><a name="type-chunk">chunk()</a></h3>
 <p><tt>chunk() = <a href="#type-chunk_bin">chunk_bin()</a> | {<a href="#type-chunk_csum">chunk_csum()</a>, <a href="#type-chunk_bin">chunk_bin()</a>}</tt></p>
 <h3 class="typedecl"><a name="type-chunk_bin">chunk_bin()</a></h3>
 <p><tt>chunk_bin() = binary() | iolist()</tt></p>
 <p> client can use either</p>
 <h3 class="typedecl"><a name="type-chunk_csum">chunk_csum()</a></h3>
 <p><tt>chunk_csum() = binary()</tt></p>
 <p> 1 byte tag, N-1 bytes checksum</p>
 <h3 class="typedecl"><a name="type-chunk_pos">chunk_pos()</a></h3>
 <p><tt>chunk_pos() = {<a href="#type-file_offset">file_offset()</a>, <a href="#type-chunk_size">chunk_size()</a>, <a href="#type-file_name_s">file_name_s()</a>}</tt></p>
 <h3 class="typedecl"><a name="type-chunk_s">chunk_s()</a></h3>
 <p><tt>chunk_s() = trimmed | binary()</tt></p>
 <h3 class="typedecl"><a name="type-chunk_size">chunk_size()</a></h3>
 <p><tt>chunk_size() = non_neg_integer()</tt></p>
 <h3 class="typedecl"><a name="type-chunk_summary">chunk_summary()</a></h3>
 <p><tt>chunk_summary() = {<a href="#type-file_offset">file_offset()</a>, <a href="#type-chunk_size">chunk_size()</a>, binary()}</tt></p>
 <h3 class="typedecl"><a name="type-csum_tag">csum_tag()</a></h3>
 <p><tt>csum_tag() = none | client_sha | server_sha | server_regen_sha</tt></p>
 <h3 class="typedecl"><a name="type-epoch_csum">epoch_csum()</a></h3>
 <p><tt>epoch_csum() = binary()</tt></p>
 <h3 class="typedecl"><a name="type-epoch_id">epoch_id()</a></h3>
 <p><tt>epoch_id() = {<a href="#type-epoch_num">epoch_num()</a>, <a href="#type-epoch_csum">epoch_csum()</a>}</tt></p>
 <h3 class="typedecl"><a name="type-epoch_num">epoch_num()</a></h3>
 <p><tt>epoch_num() = -1 | non_neg_integer()</tt></p>
 <h3 class="typedecl"><a name="type-error_general">error_general()</a></h3>
 <p><tt>error_general() = bad_arg | wedged | bad_checksum</tt></p>
 <h3 class="typedecl"><a name="type-file_info">file_info()</a></h3>
 <p><tt>file_info() = {<a href="#type-file_size">file_size()</a>, <a href="#type-file_name_s">file_name_s()</a>}</tt></p>
 <h3 class="typedecl"><a name="type-file_name">file_name()</a></h3>
 <p><tt>file_name() = binary() | list()</tt></p>
 <h3 class="typedecl"><a name="type-file_name_s">file_name_s()</a></h3>
 <p><tt>file_name_s() = binary()</tt></p>
 <p> server reply</p>
 <h3 class="typedecl"><a name="type-file_offset">file_offset()</a></h3>
 <p><tt>file_offset() = non_neg_integer()</tt></p>
 <h3 class="typedecl"><a name="type-file_prefix">file_prefix()</a></h3>
 <p><tt>file_prefix() = binary() | list()</tt></p>
 <h3 class="typedecl"><a name="type-file_size">file_size()</a></h3>
 <p><tt>file_size() = non_neg_integer()</tt></p>
 <h3 class="typedecl"><a name="type-inet_host">inet_host()</a></h3>
 <p><tt>inet_host() = <a href="inet.html#type-ip_address">inet:ip_address()</a> | <a href="inet.html#type-hostname">inet:hostname()</a></tt></p>
 <h3 class="typedecl"><a name="type-inet_port">inet_port()</a></h3>
 <p><tt>inet_port() = <a href="inet.html#type-port_number">inet:port_number()</a></tt></p>
 <h3 class="typedecl"><a name="type-projection">projection()</a></h3>
 <p><tt>projection() = #projection_v1{}</tt></p>
 <h3 class="typedecl"><a name="type-projection_type">projection_type()</a></h3>
 <p><tt>projection_type() = public | private</tt></p>
 <hr>
 <div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <p><i>Generated by EDoc, Dec 8 2015, 21:37:38.</i></p>
 </body>
 </html>
--- a/edoc/machi_file_proxy.html
+++ b/edoc/machi_file_proxy.html
@ -0,0 +1,118 @@
 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
 <html>
 <head>
 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
 <title>Module machi_file_proxy</title>
 <link rel="stylesheet" type="text/css" href="stylesheet.css" title="EDoc">
 </head>
 <body bgcolor="white">
 <div class="navbar"><a name="#navbar_top"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <hr>
 <h1>Module machi_file_proxy</h1>
 <ul class="index"><li><a href="#description">Description</a></li><li><a href="#index">Function Index</a></li><li><a href="#functions">Function Details</a></li></ul>This is a proxy process which mediates access to Machi FLU  
 controlled files.
 <p><b>Behaviours:</b> <a href="gen_server.html"><tt>gen_server</tt></a>.</p>
 <h2><a name="description">Description</a></h2><p>This is a proxy process which mediates access to Machi FLU  
 controlled files.  In particular, it manages the "write-once register"  
 conceit at the heart of Machi's design.</p>
  <p>Read, write and append requests for a single file will be managed  
 through this proxy.  Clients can also request syncs for specific  
 types of filehandles.</p>
  As operations are requested, the proxy keeps track of how many
  operations it has performed (and how many errors were generated.)
  After a sufficient number of inactivity, the server terminates
  itself.
 <h2><a name="index">Function Index</a></h2>
 <table width="100%" border="1" cellspacing="0" cellpadding="2" summary="function index"><tr><td valign="top"><a href="#append-2">append/2</a></td><td>Append data.</td></tr>
 <tr><td valign="top"><a href="#append-4">append/4</a></td><td>Append data to file, supplying client metadata and (if desired) a
 reservation for additional space.</td></tr>
 <tr><td valign="top"><a href="#read-3">read/3</a></td><td>Read file at offset for length.</td></tr>
 <tr><td valign="top"><a href="#read-4">read/4</a></td><td></td></tr>
 <tr><td valign="top"><a href="#start_link-3">start_link/3</a></td><td>Start a new instance of the file proxy service.</td></tr>
 <tr><td valign="top"><a href="#stop-1">stop/1</a></td><td>Request to stop an instance of the file proxy service.</td></tr>
 <tr><td valign="top"><a href="#sync-1">sync/1</a></td><td>Force a sync of all filehandles.</td></tr>
 <tr><td valign="top"><a href="#sync-2">sync/2</a></td><td>Force a sync of a specific filehandle type.</td></tr>
 <tr><td valign="top"><a href="#trim-4">trim/4</a></td><td></td></tr>
 <tr><td valign="top"><a href="#write-3">write/3</a></td><td>Write data at offset.</td></tr>
 <tr><td valign="top"><a href="#write-4">write/4</a></td><td>Write data at offset, including the client metadata.</td></tr>
 </table>
 <h2><a name="functions">Function Details</a></h2>
 <h3 class="function"><a name="append-2">append/2</a></h3>
 <div class="spec">
 <p><tt>append(Pid::pid(), Data::binary()) -&gt; {ok, File::string(), Offset::non_neg_integer()} | {error, term()}</tt><br></p>
 </div><p>Append data</p>
 <h3 class="function"><a name="append-4">append/4</a></h3>
 <div class="spec">
 <p><tt>append(Pid::pid(), ClientMeta::<a href="proplists.html#type-proplist">proplists:proplist()</a>, Extra::non_neg_integer(), Data::binary()) -&gt; {ok, File::string(), Offset::non_neg_integer()} | {error, term()}</tt><br></p>
 </div><p>Append data to file, supplying client metadata and (if desired) a
 reservation for additional space.  ClientMeta is a proplist and expects the
 same keys as write/4.</p>
 <h3 class="function"><a name="read-3">read/3</a></h3>
 <div class="spec">
 <p><tt>read(Pid::pid(), Offset::non_neg_integer(), Length::non_neg_integer()) -&gt; {ok, [{Filename::string(), Offset::non_neg_integer(), Data::binary(), Checksum::binary()}]} | {error, Reason::term()}</tt><br></p>
 </div><p>Read file at offset for length. This returns a sequence of all
 written and trimmed (optional) bytes that overlaps with requested
 offset and length. Borders are not aligned.</p>
 <h3 class="function"><a name="read-4">read/4</a></h3>
 <div class="spec">
 <p><tt>read(Pid::pid(), Offset::non_neg_integer(), Length::non_neg_integer(), Opts::[{no_checksum | no_chunk | needs_trimmed, boolean()}]) -&gt; {ok, [{Filename::string(), Offset::non_neg_integer(), Data::binary(), Checksum::binary()}]} | {error, Reason::term()}</tt><br></p>
 </div>
 <h3 class="function"><a name="start_link-3">start_link/3</a></h3>
 <div class="spec">
 <p><tt>start_link(FluName::atom(), Filename::string(), DataDir::string()) -&gt; any()</tt><br></p>
 </div><p>Start a new instance of the file proxy service. Takes the filename
 and data directory as arguments. This function is typically called by the
 <code>machi_file_proxy_sup:start_proxy/2</code> function.</p>
 <h3 class="function"><a name="stop-1">stop/1</a></h3>
 <div class="spec">
 <p><tt>stop(Pid::pid()) -&gt; ok</tt><br></p>
 </div><p>Request to stop an instance of the file proxy service.</p>
 <h3 class="function"><a name="sync-1">sync/1</a></h3>
 <div class="spec">
 <p><tt>sync(Pid::pid()) -&gt; ok | {error, term()}</tt><br></p>
 </div><p>Force a sync of all filehandles</p>
 <h3 class="function"><a name="sync-2">sync/2</a></h3>
 <div class="spec">
 <p><tt>sync(Pid::pid(), Type::all | data | csum) -&gt; ok | {error, term()}</tt><br></p>
 </div><p>Force a sync of a specific filehandle type. Valid types are <code>all</code>, <code>csum</code> and <code>data</code>.</p>
 <h3 class="function"><a name="trim-4">trim/4</a></h3>
 <div class="spec">
 <p><tt>trim(Pid, Offset, Size, TriggerGC) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="write-3">write/3</a></h3>
 <div class="spec">
 <p><tt>write(Pid::pid(), Offset::non_neg_integer(), Data::binary()) -&gt; ok | {error, term()}</tt><br></p>
 </div><p>Write data at offset</p>
 <h3 class="function"><a name="write-4">write/4</a></h3>
 <div class="spec">
 <p><tt>write(Pid::pid(), Offset::non_neg_integer(), ClientMeta::<a href="proplists.html#type-proplist">proplists:proplist()</a>, Data::binary()) -&gt; ok | {error, term()}</tt><br></p>
 </div><p>Write data at offset, including the client metadata. ClientMeta is a proplist
 that expects the following keys and values:
 <ul>
       <li><code>client_csum_tag</code> - the type of checksum from the client as defined in the machi.hrl file</li>
       <li><code>client_csum</code> - the checksum value from the client</li>
 </ul></p>
 <hr>
 <div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <p><i>Generated by EDoc, Dec 8 2015, 21:37:38.</i></p>
 </body>
 </html>
--- a/edoc/machi_file_proxy_sup.html
+++ b/edoc/machi_file_proxy_sup.html
@ -0,0 +1,51 @@
 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
 <html>
 <head>
 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
 <title>Module machi_file_proxy_sup</title>
 <link rel="stylesheet" type="text/css" href="stylesheet.css" title="EDoc">
 </head>
 <body bgcolor="white">
 <div class="navbar"><a name="#navbar_top"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <hr>
 <h1>Module machi_file_proxy_sup</h1>
 <ul class="index"><li><a href="#description">Description</a></li><li><a href="#index">Function Index</a></li><li><a href="#functions">Function Details</a></li></ul>This is the main supervisor for the file proxies.
 <p><b>Behaviours:</b> <a href="supervisor.html"><tt>supervisor</tt></a>.</p>
 <h2><a name="description">Description</a></h2>This is the main supervisor for the file proxies.
 <h2><a name="index">Function Index</a></h2>
 <table width="100%" border="1" cellspacing="0" cellpadding="2" summary="function index"><tr><td valign="top"><a href="#child_spec-1">child_spec/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#init-1">init/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#start_link-1">start_link/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#start_proxy-3">start_proxy/3</a></td><td></td></tr>
 </table>
 <h2><a name="functions">Function Details</a></h2>
 <h3 class="function"><a name="child_spec-1">child_spec/1</a></h3>
 <div class="spec">
 <p><tt>child_spec(FluName) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="init-1">init/1</a></h3>
 <div class="spec">
 <p><tt>init(X1) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="start_link-1">start_link/1</a></h3>
 <div class="spec">
 <p><tt>start_link(FluName) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="start_proxy-3">start_proxy/3</a></h3>
 <div class="spec">
 <p><tt>start_proxy(FluName, DataDir, Filename) -&gt; any()</tt></p>
 </div>
 <hr>
 <div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <p><i>Generated by EDoc, Dec 8 2015, 21:37:38.</i></p>
 </body>
 </html>
--- a/edoc/machi_fitness.html
+++ b/edoc/machi_fitness.html
@ -0,0 +1,120 @@
 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
 <html>
 <head>
 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
 <title>Module machi_fitness</title>
 <link rel="stylesheet" type="text/css" href="stylesheet.css" title="EDoc">
 </head>
 <body bgcolor="white">
 <div class="navbar"><a name="#navbar_top"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <hr>
 <h1>Module machi_fitness</h1>
 <ul class="index"><li><a href="#index">Function Index</a></li><li><a href="#functions">Function Details</a></li></ul>
 <p><b>Behaviours:</b> <a href="gen_server.html"><tt>gen_server</tt></a>.</p>
 <h2><a name="index">Function Index</a></h2>
 <table width="100%" border="1" cellspacing="0" cellpadding="2" summary="function index"><tr><td valign="top"><a href="#add_admin_down-3">add_admin_down/3</a></td><td></td></tr>
 <tr><td valign="top"><a href="#code_change-3">code_change/3</a></td><td></td></tr>
 <tr><td valign="top"><a href="#delete_admin_down-2">delete_admin_down/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#format_status-2">format_status/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#get_unfit_list-1">get_unfit_list/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#handle_call-3">handle_call/3</a></td><td></td></tr>
 <tr><td valign="top"><a href="#handle_cast-2">handle_cast/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#handle_info-2">handle_info/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#init-1">init/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#send_fitness_update_spam-3">send_fitness_update_spam/3</a></td><td></td></tr>
 <tr><td valign="top"><a href="#send_spam_to_everyone-1">send_spam_to_everyone/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#start_link-1">start_link/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#terminate-2">terminate/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#trigger_early_adjustment-2">trigger_early_adjustment/2</a></td><td>For testing purposes, we don't want a test to wait for
       wall-clock time to elapse before the fitness server makes a
       down-&gt;up status decision.</td></tr>
 <tr><td valign="top"><a href="#update_local_down_list-3">update_local_down_list/3</a></td><td></td></tr>
 </table>
 <h2><a name="functions">Function Details</a></h2>
 <h3 class="function"><a name="add_admin_down-3">add_admin_down/3</a></h3>
 <div class="spec">
 <p><tt>add_admin_down(PidSpec, DownFLU, DownProps) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="code_change-3">code_change/3</a></h3>
 <div class="spec">
 <p><tt>code_change(OldVsn, S, Extra) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="delete_admin_down-2">delete_admin_down/2</a></h3>
 <div class="spec">
 <p><tt>delete_admin_down(PidSpec, DownFLU) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="format_status-2">format_status/2</a></h3>
 <div class="spec">
 <p><tt>format_status(Opt, X2) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="get_unfit_list-1">get_unfit_list/1</a></h3>
 <div class="spec">
 <p><tt>get_unfit_list(PidSpec) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="handle_call-3">handle_call/3</a></h3>
 <div class="spec">
 <p><tt>handle_call(Request, From, State) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="handle_cast-2">handle_cast/2</a></h3>
 <div class="spec">
 <p><tt>handle_cast(Msg, S) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="handle_info-2">handle_info/2</a></h3>
 <div class="spec">
 <p><tt>handle_info(Info, State) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="init-1">init/1</a></h3>
 <div class="spec">
 <p><tt>init(Args) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="send_fitness_update_spam-3">send_fitness_update_spam/3</a></h3>
 <div class="spec">
 <p><tt>send_fitness_update_spam(Pid, FromName, Dict) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="send_spam_to_everyone-1">send_spam_to_everyone/1</a></h3>
 <div class="spec">
 <p><tt>send_spam_to_everyone(Pid) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="start_link-1">start_link/1</a></h3>
 <div class="spec">
 <p><tt>start_link(Args) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="terminate-2">terminate/2</a></h3>
 <div class="spec">
 <p><tt>terminate(Reason, S) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="trigger_early_adjustment-2">trigger_early_adjustment/2</a></h3>
 <div class="spec">
 <p><tt>trigger_early_adjustment(Pid, FLU) -&gt; any()</tt></p>
 </div><p>For testing purposes, we don't want a test to wait for
       wall-clock time to elapse before the fitness server makes a
       down-&gt;up status decision.</p>
 <h3 class="function"><a name="update_local_down_list-3">update_local_down_list/3</a></h3>
 <div class="spec">
 <p><tt>update_local_down_list(PidSpec, Down, MembersDict) -&gt; any()</tt></p>
 </div>
 <hr>
 <div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <p><i>Generated by EDoc, Dec 8 2015, 21:37:38.</i></p>
 </body>
 </html>
--- a/edoc/machi_flu1.html
+++ b/edoc/machi_flu1.html
@ -0,0 +1,114 @@
 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
 <html>
 <head>
 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
 <title>Module machi_flu1</title>
 <link rel="stylesheet" type="text/css" href="stylesheet.css" title="EDoc">
 </head>
 <body bgcolor="white">
 <div class="navbar"><a name="#navbar_top"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <hr>
 <h1>Module machi_flu1</h1>
 <ul class="index"><li><a href="#description">Description</a></li><li><a href="#index">Function Index</a></li><li><a href="#functions">Function Details</a></li></ul>The Machi FLU file server + file location sequencer.
 <h2><a name="description">Description</a></h2><p>The Machi FLU file server + file location sequencer.</p>
  <p>This module implements only the Machi FLU file server and its  
 implicit sequencer.  
 Please see the EDoc "Overview" for details about the FLU as a  
 primitive file server process vs. the larger Machi design of a FLU  
 as a sequencer + file server + chain manager group of processes.</p>
  <p>The FLU is named after the CORFU server "FLU" or "FLash Unit" server.</p>
  <h4><a name="Protocol_origins">Protocol origins</a></h4>
  <p>Today's FLU fully supports a protocol that is based on Protocol
  Buffers.  Please see the <code>src/machi.proto</code> file for details.</p>
  <h4><a name="TODO_items">TODO items</a></h4>
  <p>TODO The per-file metadata tuple store is missing from this implementation.</p>
  TODO Section 4.1 ("The FLU") of the Machi design doc suggests that
  the FLU keep track of the epoch number of the last file write (and
  perhaps last metadata write), as an optimization for inter-FLU data
  replication/chain repair.
 <h2><a name="index">Function Index</a></h2>
 <table width="100%" border="1" cellspacing="0" cellpadding="2" summary="function index"><tr><td valign="top"><a href="#current_state-1">current_state/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#format_state-1">format_state/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#main2-4">main2/4</a></td><td></td></tr>
 <tr><td valign="top"><a href="#make_listener_regname-1">make_listener_regname/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#make_projection_server_regname-1">make_projection_server_regname/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#run_append_server-2">run_append_server/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#run_listen_server-1">run_listen_server/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#start_link-1">start_link/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#stop-1">stop/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#update_wedge_state-3">update_wedge_state/3</a></td><td></td></tr>
 <tr><td valign="top"><a href="#wedge_myself-2">wedge_myself/2</a></td><td></td></tr>
 </table>
 <h2><a name="functions">Function Details</a></h2>
 <h3 class="function"><a name="current_state-1">current_state/1</a></h3>
 <div class="spec">
 <p><tt>current_state(PidSpec) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="format_state-1">format_state/1</a></h3>
 <div class="spec">
 <p><tt>format_state(State) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="main2-4">main2/4</a></h3>
 <div class="spec">
 <p><tt>main2(FluName, TcpPort, DataDir, Props) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="make_listener_regname-1">make_listener_regname/1</a></h3>
 <div class="spec">
 <p><tt>make_listener_regname(BaseName) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="make_projection_server_regname-1">make_projection_server_regname/1</a></h3>
 <div class="spec">
 <p><tt>make_projection_server_regname(BaseName) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="run_append_server-2">run_append_server/2</a></h3>
 <div class="spec">
 <p><tt>run_append_server(FluPid, State) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="run_listen_server-1">run_listen_server/1</a></h3>
 <div class="spec">
 <p><tt>run_listen_server(State) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="start_link-1">start_link/1</a></h3>
 <div class="spec">
 <p><tt>start_link(Rest) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="stop-1">stop/1</a></h3>
 <div class="spec">
 <p><tt>stop(Pid) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="update_wedge_state-3">update_wedge_state/3</a></h3>
 <div class="spec">
 <p><tt>update_wedge_state(PidSpec, Boolean, EpochId) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="wedge_myself-2">wedge_myself/2</a></h3>
 <div class="spec">
 <p><tt>wedge_myself(PidSpec, EpochId) -&gt; any()</tt></p>
 </div>
 <hr>
 <div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <p><i>Generated by EDoc, Dec 8 2015, 21:37:38.</i></p>
 </body>
 </html>
--- a/edoc/machi_flu1_client.html
+++ b/edoc/machi_flu1_client.html
@ -0,0 +1,340 @@
 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
 <html>
 <head>
 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
 <title>Module machi_flu1_client</title>
 <link rel="stylesheet" type="text/css" href="stylesheet.css" title="EDoc">
 </head>
 <body bgcolor="white">
 <div class="navbar"><a name="#navbar_top"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <hr>
 <h1>Module machi_flu1_client</h1>
 <ul class="index"><li><a href="#description">Description</a></li><li><a href="#types">Data Types</a></li><li><a href="#index">Function Index</a></li><li><a href="#functions">Function Details</a></li></ul>Erlang API for the Machi FLU TCP protocol version 1.
 <h2><a name="description">Description</a></h2><p>Erlang API for the Machi FLU TCP protocol version 1.</p>
  <p>This client API handles low-level PDU serialization/deserialization
  and low-level TCP session management, e.g. open, receive, write,
  close.  The API for higher-level session management and Machi state
  management can be found in <a href="machi_proxy_flu1_client.html"><code>machi_proxy_flu1_client</code></a> and
  <a href="machi_cr_client.html"><code>machi_cr_client</code></a>.</p>
  <p>For the moment, this module implements a Protocol Buffers-based  
 protocol as the sole supported access method to the server,  
 sequencer, and projection store.  Conceptually, those three  
 services are independent and ought to have their own protocols.  As  
 a practical matter, there is no need for wire protocol  
 compatibility.  Furthermore, from the perspective of failure  
 detection, it is very convenient that all three FLU-related  
 services are accessed using the same single TCP port.</p>
  TODO This EDoc was written first, and the EDoc and also <code>-type</code> and
  <code>-spec</code> definitions for <a href="machi_proxy_flu1_client.html"><code>machi_proxy_flu1_client</code></a> and <a href="machi_cr_client.html"><code>machi_cr_client</code></a> must be improved.
 <h2><a name="types">Data Types</a></h2>
 <h3 class="typedecl"><a name="type-port_wrap">port_wrap()</a></h3>
 <p><tt>port_wrap() = {w, atom(), term()}</tt></p>
 <h2><a name="index">Function Index</a></h2>
 <table width="100%" border="1" cellspacing="0" cellpadding="2" summary="function index"><tr><td valign="top"><a href="#append_chunk-4">append_chunk/4</a></td><td>Append a chunk (binary- or iolist-style) of data to a file
  with <code>Prefix</code>.</td></tr>
 <tr><td valign="top"><a href="#append_chunk-5">append_chunk/5</a></td><td>Append a chunk (binary- or iolist-style) of data to a file
  with <code>Prefix</code>.</td></tr>
 <tr><td valign="top"><a href="#append_chunk_extra-5">append_chunk_extra/5</a></td><td>Append a chunk (binary- or iolist-style) of data to a file
  with <code>Prefix</code> and also request an additional <code>Extra</code> bytes.</td></tr>
 <tr><td valign="top"><a href="#append_chunk_extra-6">append_chunk_extra/6</a></td><td>Append a chunk (binary- or iolist-style) of data to a file
  with <code>Prefix</code> and also request an additional <code>Extra</code> bytes.</td></tr>
 <tr><td valign="top"><a href="#checksum_list-3">checksum_list/3</a></td><td>Fetch the list of chunk checksums for <code>File</code>.</td></tr>
 <tr><td valign="top"><a href="#checksum_list-4">checksum_list/4</a></td><td>Fetch the list of chunk checksums for <code>File</code>.</td></tr>
 <tr><td valign="top"><a href="#connect-1">connect/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#connected_p-1">connected_p/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#delete_migration-3">delete_migration/3</a></td><td>Restricted API: Delete a file after it has been successfully
  migrated.</td></tr>
 <tr><td valign="top"><a href="#delete_migration-4">delete_migration/4</a></td><td>Restricted API: Delete a file after it has been successfully
  migrated.</td></tr>
 <tr><td valign="top"><a href="#disconnect-1">disconnect/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#echo-2">echo/2</a></td><td>Echo -- test protocol round-trip.</td></tr>
 <tr><td valign="top"><a href="#echo-3">echo/3</a></td><td>Get all epoch numbers from the FLU's projection store.</td></tr>
 <tr><td valign="top"><a href="#get_all_projections-2">get_all_projections/2</a></td><td>Get all projections from the FLU's projection store.</td></tr>
 <tr><td valign="top"><a href="#get_all_projections-3">get_all_projections/3</a></td><td>Get all projections from the FLU's projection store.</td></tr>
 <tr><td valign="top"><a href="#get_latest_epochid-2">get_latest_epochid/2</a></td><td>Get the latest epoch number + checksum from the FLU's projection store.</td></tr>
 <tr><td valign="top"><a href="#get_latest_epochid-3">get_latest_epochid/3</a></td><td>Get the latest epoch number + checksum from the FLU's projection store.</td></tr>
 <tr><td valign="top"><a href="#kick_projection_reaction-2">kick_projection_reaction/2</a></td><td>Kick (politely) the remote chain manager to react to a
  projection change.</td></tr>
 <tr><td valign="top"><a href="#kick_projection_reaction-3">kick_projection_reaction/3</a></td><td>Kick (politely) the remote chain manager to react to a
  projection change.</td></tr>
 <tr><td valign="top"><a href="#list_all_projections-2">list_all_projections/2</a></td><td>Get all epoch numbers from the FLU's projection store.</td></tr>
 <tr><td valign="top"><a href="#list_all_projections-3">list_all_projections/3</a></td><td>Get all epoch numbers from the FLU's projection store.</td></tr>
 <tr><td valign="top"><a href="#list_files-2">list_files/2</a></td><td>Fetch the list of all files on the remote FLU.</td></tr>
 <tr><td valign="top"><a href="#list_files-3">list_files/3</a></td><td>Fetch the list of all files on the remote FLU.</td></tr>
 <tr><td valign="top"><a href="#quit-1">quit/1</a></td><td>Quit &amp; close the connection to remote FLU.</td></tr>
 <tr><td valign="top"><a href="#read_chunk-6">read_chunk/6</a></td><td>Read a chunk of data of size <code>Size</code> from <code>File</code> at <code>Offset</code>.</td></tr>
 <tr><td valign="top"><a href="#read_chunk-7">read_chunk/7</a></td><td>Read a chunk of data of size <code>Size</code> from <code>File</code> at <code>Offset</code>.</td></tr>
 <tr><td valign="top"><a href="#read_latest_projection-2">read_latest_projection/2</a></td><td>Get the latest projection from the FLU's projection store for <code>ProjType</code></td></tr>
 <tr><td valign="top"><a href="#read_latest_projection-3">read_latest_projection/3</a></td><td>Get the latest projection from the FLU's projection store for <code>ProjType</code></td></tr>
 <tr><td valign="top"><a href="#read_projection-3">read_projection/3</a></td><td>Read a projection <code>Proj</code> of type <code>ProjType</code>.</td></tr>
 <tr><td valign="top"><a href="#read_projection-4">read_projection/4</a></td><td>Read a projection <code>Proj</code> of type <code>ProjType</code>.</td></tr>
 <tr><td valign="top"><a href="#trim_chunk-5">trim_chunk/5</a></td><td>Restricted API: Write a chunk of already-sequenced data to
  <code>File</code> at <code>Offset</code>.</td></tr>
 <tr><td valign="top"><a href="#trunc_hack-3">trunc_hack/3</a></td><td>Restricted API: Truncate a file after it has been successfully
  erasure coded.</td></tr>
 <tr><td valign="top"><a href="#trunc_hack-4">trunc_hack/4</a></td><td>Restricted API: Truncate a file after it has been successfully
  erasure coded.</td></tr>
 <tr><td valign="top"><a href="#wedge_status-1">wedge_status/1</a></td><td>Fetch the wedge status from the remote FLU.</td></tr>
 <tr><td valign="top"><a href="#wedge_status-2">wedge_status/2</a></td><td>Fetch the wedge status from the remote FLU.</td></tr>
 <tr><td valign="top"><a href="#write_chunk-5">write_chunk/5</a></td><td>Restricted API: Write a chunk of already-sequenced data to
  <code>File</code> at <code>Offset</code>.</td></tr>
 <tr><td valign="top"><a href="#write_chunk-6">write_chunk/6</a></td><td>Restricted API: Write a chunk of already-sequenced data to
  <code>File</code> at <code>Offset</code>.</td></tr>
 <tr><td valign="top"><a href="#write_projection-3">write_projection/3</a></td><td>Write a projection <code>Proj</code> of type <code>ProjType</code>.</td></tr>
 <tr><td valign="top"><a href="#write_projection-4">write_projection/4</a></td><td>Write a projection <code>Proj</code> of type <code>ProjType</code>.</td></tr>
 </table>
 <h2><a name="functions">Function Details</a></h2>
 <h3 class="function"><a name="append_chunk-4">append_chunk/4</a></h3>
 <div class="spec">
 <p><tt>append_chunk(Sock::<a href="#type-port_wrap">port_wrap()</a>, EpochID::<a href="machi_dt.html#type-epoch_id">machi_dt:epoch_id()</a>, Prefix::<a href="machi_dt.html#type-file_prefix">machi_dt:file_prefix()</a>, Chunk::<a href="machi_dt.html#type-chunk">machi_dt:chunk()</a>) -&gt; {ok, <a href="machi_dt.html#type-chunk_pos">machi_dt:chunk_pos()</a>} | {error, <a href="machi_dt.html#type-error_general">machi_dt:error_general()</a>} | {error, term()}</tt><br></p>
 </div><p>Append a chunk (binary- or iolist-style) of data to a file
  with <code>Prefix</code>.</p>
 <h3 class="function"><a name="append_chunk-5">append_chunk/5</a></h3>
 <div class="spec">
 <p><tt>append_chunk(Host::<a href="machi_dt.html#type-inet_host">machi_dt:inet_host()</a>, TcpPort::<a href="machi_dt.html#type-inet_port">machi_dt:inet_port()</a>, EpochID::<a href="machi_dt.html#type-epoch_id">machi_dt:epoch_id()</a>, Prefix::<a href="machi_dt.html#type-file_prefix">machi_dt:file_prefix()</a>, Chunk::<a href="machi_dt.html#type-chunk">machi_dt:chunk()</a>) -&gt; {ok, <a href="machi_dt.html#type-chunk_pos">machi_dt:chunk_pos()</a>} | {error, <a href="machi_dt.html#type-error_general">machi_dt:error_general()</a>} | {error, term()}</tt><br></p>
 </div><p>Append a chunk (binary- or iolist-style) of data to a file
  with <code>Prefix</code>.</p>
 <h3 class="function"><a name="append_chunk_extra-5">append_chunk_extra/5</a></h3>
 <div class="spec">
 <p><tt>append_chunk_extra(Sock::<a href="#type-port_wrap">port_wrap()</a>, EpochID::<a href="machi_dt.html#type-epoch_id">machi_dt:epoch_id()</a>, Prefix::<a href="machi_dt.html#type-file_prefix">machi_dt:file_prefix()</a>, Chunk::<a href="machi_dt.html#type-chunk">machi_dt:chunk()</a>, ChunkExtra::<a href="machi_dt.html#type-chunk_size">machi_dt:chunk_size()</a>) -&gt; {ok, <a href="machi_dt.html#type-chunk_pos">machi_dt:chunk_pos()</a>} | {error, <a href="machi_dt.html#type-error_general">machi_dt:error_general()</a>} | {error, term()}</tt><br></p>
 </div><p><p>Append a chunk (binary- or iolist-style) of data to a file
  with <code>Prefix</code> and also request an additional <code>Extra</code> bytes.</p>
  For example, if the <code>Chunk</code> size is 1 KByte and <code>Extra</code> is 4K Bytes, then
  the file offsets that follow <code>Chunk</code>'s position for the following 4K will
  be reserved by the file sequencer for later write(s) by the
  <code>write_chunk()</code> API.</p>
 <h3 class="function"><a name="append_chunk_extra-6">append_chunk_extra/6</a></h3>
 <div class="spec">
 <p><tt>append_chunk_extra(Host::<a href="machi_dt.html#type-inet_host">machi_dt:inet_host()</a>, TcpPort::<a href="machi_dt.html#type-inet_port">machi_dt:inet_port()</a>, EpochID::<a href="machi_dt.html#type-epoch_id">machi_dt:epoch_id()</a>, Prefix::<a href="machi_dt.html#type-file_prefix">machi_dt:file_prefix()</a>, Chunk::<a href="machi_dt.html#type-chunk">machi_dt:chunk()</a>, ChunkExtra::<a href="machi_dt.html#type-chunk_size">machi_dt:chunk_size()</a>) -&gt; {ok, <a href="machi_dt.html#type-chunk_pos">machi_dt:chunk_pos()</a>} | {error, <a href="machi_dt.html#type-error_general">machi_dt:error_general()</a>} | {error, term()}</tt><br></p>
 </div><p><p>Append a chunk (binary- or iolist-style) of data to a file
  with <code>Prefix</code> and also request an additional <code>Extra</code> bytes.</p>
  For example, if the <code>Chunk</code> size is 1 KByte and <code>Extra</code> is 4K Bytes, then
  the file offsets that follow <code>Chunk</code>'s position for the following 4K will
  be reserved by the file sequencer for later write(s) by the
  <code>write_chunk()</code> API.</p>
 <h3 class="function"><a name="checksum_list-3">checksum_list/3</a></h3>
 <div class="spec">
 <p><tt>checksum_list(Sock::<a href="#type-port_wrap">port_wrap()</a>, EpochID::<a href="machi_dt.html#type-epoch_id">machi_dt:epoch_id()</a>, File::<a href="machi_dt.html#type-file_name">machi_dt:file_name()</a>) -&gt; {ok, binary()} | {error, <a href="machi_dt.html#type-error_general">machi_dt:error_general()</a> | no_such_file | partial_read} | {error, term()}</tt><br></p>
 </div><p>Fetch the list of chunk checksums for <code>File</code>.</p>
 <h3 class="function"><a name="checksum_list-4">checksum_list/4</a></h3>
 <div class="spec">
 <p><tt>checksum_list(Host::<a href="machi_dt.html#type-inet_host">machi_dt:inet_host()</a>, TcpPort::<a href="machi_dt.html#type-inet_port">machi_dt:inet_port()</a>, EpochID::<a href="machi_dt.html#type-epoch_id">machi_dt:epoch_id()</a>, File::<a href="machi_dt.html#type-file_name">machi_dt:file_name()</a>) -&gt; {ok, binary()} | {error, <a href="machi_dt.html#type-error_general">machi_dt:error_general()</a> | no_such_file} | {error, term()}</tt><br></p>
 </div><p><p>Fetch the list of chunk checksums for <code>File</code>.</p>
  Why return a simple <code>binary()</code> type rather than
  <code>[machi_dt:chunk_summary()]</code>?  The two reasons are:
  <ol>
  <li> Server overhead: the CPU required to chop up the implementation-
       specific store into zillions of very small terms is very high.
  </li>
  <li> Protocol encoding and decoding overhead: the cost is non-zero,
       and the sum of cost of encoding and decoding a zillion small terms
       is substantial.
  </li>
  </ol>
  <p>For both reasons, the server's protocol response is absurdly simple
  and very fast: send back a <code>binary()</code> blob to the client.  Then it  
 is the client's responsibility to spend the CPU time to parse the  
 blob.</p>
  Details of the encoding used inside the <code>binary()</code> blog can be found
  in the EDoc comments for <a href="machi_flu1.html#decode_csum_file_entry-1"><code>machi_flu1:decode_csum_file_entry/1</code></a>.</p>
 <h3 class="function"><a name="connect-1">connect/1</a></h3>
 <div class="spec">
 <p><tt>connect(P_srvr) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="connected_p-1">connected_p/1</a></h3>
 <div class="spec">
 <p><tt>connected_p(X1) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="delete_migration-3">delete_migration/3</a></h3>
 <div class="spec">
 <p><tt>delete_migration(Sock::<a href="#type-port_wrap">port_wrap()</a>, EpochID::<a href="machi_dt.html#type-epoch_id">machi_dt:epoch_id()</a>, File::<a href="machi_dt.html#type-file_name">machi_dt:file_name()</a>) -&gt; ok | {error, <a href="machi_dt.html#type-error_general">machi_dt:error_general()</a> | no_such_file} | {error, term()}</tt><br></p>
 </div><p>Restricted API: Delete a file after it has been successfully
  migrated.</p>
 <h3 class="function"><a name="delete_migration-4">delete_migration/4</a></h3>
 <div class="spec">
 <p><tt>delete_migration(Host::<a href="machi_dt.html#type-inet_host">machi_dt:inet_host()</a>, TcpPort::<a href="machi_dt.html#type-inet_port">machi_dt:inet_port()</a>, EpochID::<a href="machi_dt.html#type-epoch_id">machi_dt:epoch_id()</a>, File::<a href="machi_dt.html#type-file_name">machi_dt:file_name()</a>) -&gt; ok | {error, <a href="machi_dt.html#type-error_general">machi_dt:error_general()</a> | no_such_file} | {error, term()}</tt><br></p>
 </div><p>Restricted API: Delete a file after it has been successfully
  migrated.</p>
 <h3 class="function"><a name="disconnect-1">disconnect/1</a></h3>
 <div class="spec">
 <p><tt>disconnect(WS) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="echo-2">echo/2</a></h3>
 <div class="spec">
 <p><tt>echo(Sock::<a href="#type-port_wrap">port_wrap()</a>, String::string()) -&gt; string() | {error, term()}</tt><br></p>
 </div><p>Echo -- test protocol round-trip.</p>
 <h3 class="function"><a name="echo-3">echo/3</a></h3>
 <div class="spec">
 <p><tt>echo(Host::<a href="machi_dt.html#type-inet_host">machi_dt:inet_host()</a>, TcpPort::<a href="machi_dt.html#type-inet_port">machi_dt:inet_port()</a>, String::string()) -&gt; string() | {error, term()}</tt><br></p>
 </div><p>Get all epoch numbers from the FLU's projection store.</p>
 <h3 class="function"><a name="get_all_projections-2">get_all_projections/2</a></h3>
 <div class="spec">
 <p><tt>get_all_projections(Sock::<a href="#type-port_wrap">port_wrap()</a>, ProjType::<a href="machi_dt.html#type-projection_type">machi_dt:projection_type()</a>) -&gt; {ok, [<a href="machi_dt.html#type-projection">machi_dt:projection()</a>]} | {error, term()}</tt><br></p>
 </div><p>Get all projections from the FLU's projection store.</p>
 <h3 class="function"><a name="get_all_projections-3">get_all_projections/3</a></h3>
 <div class="spec">
 <p><tt>get_all_projections(Host::<a href="machi_dt.html#type-inet_host">machi_dt:inet_host()</a>, TcpPort::<a href="machi_dt.html#type-inet_port">machi_dt:inet_port()</a>, ProjType::<a href="machi_dt.html#type-projection_type">machi_dt:projection_type()</a>) -&gt; {ok, [<a href="machi_dt.html#type-projection">machi_dt:projection()</a>]} | {error, term()}</tt><br></p>
 </div><p>Get all projections from the FLU's projection store.</p>
 <h3 class="function"><a name="get_latest_epochid-2">get_latest_epochid/2</a></h3>
 <div class="spec">
 <p><tt>get_latest_epochid(Sock::<a href="#type-port_wrap">port_wrap()</a>, ProjType::<a href="machi_dt.html#type-projection_type">machi_dt:projection_type()</a>) -&gt; {ok, <a href="machi_dt.html#type-epoch_id">machi_dt:epoch_id()</a>} | {error, term()}</tt><br></p>
 </div><p>Get the latest epoch number + checksum from the FLU's projection store.</p>
 <h3 class="function"><a name="get_latest_epochid-3">get_latest_epochid/3</a></h3>
 <div class="spec">
 <p><tt>get_latest_epochid(Host::<a href="machi_dt.html#type-inet_host">machi_dt:inet_host()</a>, TcpPort::<a href="machi_dt.html#type-inet_port">machi_dt:inet_port()</a>, ProjType::<a href="machi_dt.html#type-projection_type">machi_dt:projection_type()</a>) -&gt; {ok, <a href="machi_dt.html#type-epoch_id">machi_dt:epoch_id()</a>} | {error, term()}</tt><br></p>
 </div><p>Get the latest epoch number + checksum from the FLU's projection store.</p>
 <h3 class="function"><a name="kick_projection_reaction-2">kick_projection_reaction/2</a></h3>
 <div class="spec">
 <p><tt>kick_projection_reaction(Sock::<a href="#type-port_wrap">port_wrap()</a>, Options::list()) -&gt; ok</tt><br></p>
 </div><p>Kick (politely) the remote chain manager to react to a
  projection change.</p>
 <h3 class="function"><a name="kick_projection_reaction-3">kick_projection_reaction/3</a></h3>
 <div class="spec">
 <p><tt>kick_projection_reaction(Host::<a href="machi_dt.html#type-inet_host">machi_dt:inet_host()</a>, TcpPort::<a href="machi_dt.html#type-inet_port">machi_dt:inet_port()</a>, Options::list()) -&gt; ok</tt><br></p>
 </div><p>Kick (politely) the remote chain manager to react to a
  projection change.</p>
 <h3 class="function"><a name="list_all_projections-2">list_all_projections/2</a></h3>
 <div class="spec">
 <p><tt>list_all_projections(Sock::<a href="#type-port_wrap">port_wrap()</a>, ProjType::<a href="machi_dt.html#type-projection_type">machi_dt:projection_type()</a>) -&gt; {ok, [non_neg_integer()]} | {error, term()}</tt><br></p>
 </div><p>Get all epoch numbers from the FLU's projection store.</p>
 <h3 class="function"><a name="list_all_projections-3">list_all_projections/3</a></h3>
 <div class="spec">
 <p><tt>list_all_projections(Host::<a href="machi_dt.html#type-inet_host">machi_dt:inet_host()</a>, TcpPort::<a href="machi_dt.html#type-inet_port">machi_dt:inet_port()</a>, ProjType::<a href="machi_dt.html#type-projection_type">machi_dt:projection_type()</a>) -&gt; {ok, [non_neg_integer()]} | {error, term()}</tt><br></p>
 </div><p>Get all epoch numbers from the FLU's projection store.</p>
 <h3 class="function"><a name="list_files-2">list_files/2</a></h3>
 <div class="spec">
 <p><tt>list_files(Sock::<a href="#type-port_wrap">port_wrap()</a>, EpochID::<a href="machi_dt.html#type-epoch_id">machi_dt:epoch_id()</a>) -&gt; {ok, [<a href="machi_dt.html#type-file_info">machi_dt:file_info()</a>]} | {error, term()}</tt><br></p>
 </div><p>Fetch the list of all files on the remote FLU.</p>
 <h3 class="function"><a name="list_files-3">list_files/3</a></h3>
 <div class="spec">
 <p><tt>list_files(Host::<a href="machi_dt.html#type-inet_host">machi_dt:inet_host()</a>, TcpPort::<a href="machi_dt.html#type-inet_port">machi_dt:inet_port()</a>, EpochID::<a href="machi_dt.html#type-epoch_id">machi_dt:epoch_id()</a>) -&gt; {ok, [<a href="machi_dt.html#type-file_info">machi_dt:file_info()</a>]} | {error, term()}</tt><br></p>
 </div><p>Fetch the list of all files on the remote FLU.</p>
 <h3 class="function"><a name="quit-1">quit/1</a></h3>
 <div class="spec">
 <p><tt>quit(Sock::<a href="#type-port_wrap">port_wrap()</a>) -&gt; ok</tt><br></p>
 </div><p>Quit &amp; close the connection to remote FLU.</p>
 <h3 class="function"><a name="read_chunk-6">read_chunk/6</a></h3>
 <div class="spec">
 <p><tt>read_chunk(Sock::<a href="#type-port_wrap">port_wrap()</a>, EpochID::<a href="machi_dt.html#type-epoch_id">machi_dt:epoch_id()</a>, File::<a href="machi_dt.html#type-file_name">machi_dt:file_name()</a>, Offset::<a href="machi_dt.html#type-file_offset">machi_dt:file_offset()</a>, Size::<a href="machi_dt.html#type-chunk_size">machi_dt:chunk_size()</a>, Opts::<a href="proplists.html#type-proplist">proplists:proplist()</a>) -&gt; {ok, <a href="machi_dt.html#type-chunk_s">machi_dt:chunk_s()</a>} | {error, <a href="machi_dt.html#type-error_general">machi_dt:error_general()</a> | not_written | partial_read} | {error, term()}</tt><br></p>
 </div><p>Read a chunk of data of size <code>Size</code> from <code>File</code> at <code>Offset</code>.</p>
 <h3 class="function"><a name="read_chunk-7">read_chunk/7</a></h3>
 <div class="spec">
 <p><tt>read_chunk(Host::<a href="machi_dt.html#type-inet_host">machi_dt:inet_host()</a>, TcpPort::<a href="machi_dt.html#type-inet_port">machi_dt:inet_port()</a>, EpochID::<a href="machi_dt.html#type-epoch_id">machi_dt:epoch_id()</a>, File::<a href="machi_dt.html#type-file_name">machi_dt:file_name()</a>, Offset::<a href="machi_dt.html#type-file_offset">machi_dt:file_offset()</a>, Size::<a href="machi_dt.html#type-chunk_size">machi_dt:chunk_size()</a>, Opts::<a href="proplists.html#type-proplist">proplists:proplist()</a>) -&gt; {ok, <a href="machi_dt.html#type-chunk_s">machi_dt:chunk_s()</a>} | {error, <a href="machi_dt.html#type-error_general">machi_dt:error_general()</a> | not_written | partial_read} | {error, term()}</tt><br></p>
 </div><p>Read a chunk of data of size <code>Size</code> from <code>File</code> at <code>Offset</code>.</p>
 <h3 class="function"><a name="read_latest_projection-2">read_latest_projection/2</a></h3>
 <div class="spec">
 <p><tt>read_latest_projection(Sock::<a href="#type-port_wrap">port_wrap()</a>, ProjType::<a href="machi_dt.html#type-projection_type">machi_dt:projection_type()</a>) -&gt; {ok, <a href="machi_dt.html#type-projection">machi_dt:projection()</a>} | {error, not_written} | {error, term()}</tt><br></p>
 </div><p>Get the latest projection from the FLU's projection store for <code>ProjType</code></p>
 <h3 class="function"><a name="read_latest_projection-3">read_latest_projection/3</a></h3>
 <div class="spec">
 <p><tt>read_latest_projection(Host::<a href="machi_dt.html#type-inet_host">machi_dt:inet_host()</a>, TcpPort::<a href="machi_dt.html#type-inet_port">machi_dt:inet_port()</a>, ProjType::<a href="machi_dt.html#type-projection_type">machi_dt:projection_type()</a>) -&gt; {ok, <a href="machi_dt.html#type-projection">machi_dt:projection()</a>} | {error, not_written} | {error, term()}</tt><br></p>
 </div><p>Get the latest projection from the FLU's projection store for <code>ProjType</code></p>
 <h3 class="function"><a name="read_projection-3">read_projection/3</a></h3>
 <div class="spec">
 <p><tt>read_projection(Sock::<a href="#type-port_wrap">port_wrap()</a>, ProjType::<a href="machi_dt.html#type-projection_type">machi_dt:projection_type()</a>, Epoch::<a href="machi_dt.html#type-epoch_num">machi_dt:epoch_num()</a>) -&gt; {ok, <a href="machi_dt.html#type-projection">machi_dt:projection()</a>} | {error, not_written} | {error, term()}</tt><br></p>
 </div><p>Read a projection <code>Proj</code> of type <code>ProjType</code>.</p>
 <h3 class="function"><a name="read_projection-4">read_projection/4</a></h3>
 <div class="spec">
 <p><tt>read_projection(Host::<a href="machi_dt.html#type-inet_host">machi_dt:inet_host()</a>, TcpPort::<a href="machi_dt.html#type-inet_port">machi_dt:inet_port()</a>, ProjType::<a href="machi_dt.html#type-projection_type">machi_dt:projection_type()</a>, Epoch::<a href="machi_dt.html#type-epoch_num">machi_dt:epoch_num()</a>) -&gt; {ok, <a href="machi_dt.html#type-projection">machi_dt:projection()</a>} | {error, not_written} | {error, term()}</tt><br></p>
 </div><p>Read a projection <code>Proj</code> of type <code>ProjType</code>.</p>
 <h3 class="function"><a name="trim_chunk-5">trim_chunk/5</a></h3>
 <div class="spec">
 <p><tt>trim_chunk(Sock::<a href="#type-port_wrap">port_wrap()</a>, EpochID::<a href="machi_dt.html#type-epoch_id">machi_dt:epoch_id()</a>, File0::<a href="machi_dt.html#type-file_name">machi_dt:file_name()</a>, Offset::<a href="machi_dt.html#type-file_offset">machi_dt:file_offset()</a>, Size::<a href="machi_dt.html#type-chunk_size">machi_dt:chunk_size()</a>) -&gt; ok | {error, <a href="machi_dt.html#type-error_general">machi_dt:error_general()</a>} | {error, term()}</tt><br></p>
 </div><p>Restricted API: Write a chunk of already-sequenced data to
  <code>File</code> at <code>Offset</code>.</p>
 <h3 class="function"><a name="trunc_hack-3">trunc_hack/3</a></h3>
 <div class="spec">
 <p><tt>trunc_hack(Sock::<a href="#type-port_wrap">port_wrap()</a>, EpochID::<a href="machi_dt.html#type-epoch_id">machi_dt:epoch_id()</a>, File::<a href="machi_dt.html#type-file_name">machi_dt:file_name()</a>) -&gt; ok | {error, <a href="machi_dt.html#type-error_general">machi_dt:error_general()</a> | no_such_file} | {error, term()}</tt><br></p>
 </div><p>Restricted API: Truncate a file after it has been successfully
  erasure coded.</p>
 <h3 class="function"><a name="trunc_hack-4">trunc_hack/4</a></h3>
 <div class="spec">
 <p><tt>trunc_hack(Host::<a href="machi_dt.html#type-inet_host">machi_dt:inet_host()</a>, TcpPort::<a href="machi_dt.html#type-inet_port">machi_dt:inet_port()</a>, EpochID::<a href="machi_dt.html#type-epoch_id">machi_dt:epoch_id()</a>, File::<a href="machi_dt.html#type-file_name">machi_dt:file_name()</a>) -&gt; ok | {error, <a href="machi_dt.html#type-error_general">machi_dt:error_general()</a> | no_such_file} | {error, term()}</tt><br></p>
 </div><p>Restricted API: Truncate a file after it has been successfully
  erasure coded.</p>
 <h3 class="function"><a name="wedge_status-1">wedge_status/1</a></h3>
 <div class="spec">
 <p><tt>wedge_status(Sock::<a href="#type-port_wrap">port_wrap()</a>) -&gt; {ok, {boolean(), <a href="machi_dt.html#type-epoch_id">machi_dt:epoch_id()</a>}} | {error, term()}</tt><br></p>
 </div><p>Fetch the wedge status from the remote FLU.</p>
 <h3 class="function"><a name="wedge_status-2">wedge_status/2</a></h3>
 <div class="spec">
 <p><tt>wedge_status(Host::<a href="machi_dt.html#type-inet_host">machi_dt:inet_host()</a>, TcpPort::<a href="machi_dt.html#type-inet_port">machi_dt:inet_port()</a>) -&gt; {ok, {boolean(), <a href="machi_dt.html#type-epoch_id">machi_dt:epoch_id()</a>}} | {error, term()}</tt><br></p>
 </div><p>Fetch the wedge status from the remote FLU.</p>
 <h3 class="function"><a name="write_chunk-5">write_chunk/5</a></h3>
 <div class="spec">
 <p><tt>write_chunk(Sock::<a href="#type-port_wrap">port_wrap()</a>, EpochID::<a href="machi_dt.html#type-epoch_id">machi_dt:epoch_id()</a>, File::<a href="machi_dt.html#type-file_name">machi_dt:file_name()</a>, Offset::<a href="machi_dt.html#type-file_offset">machi_dt:file_offset()</a>, Chunk::<a href="machi_dt.html#type-chunk">machi_dt:chunk()</a>) -&gt; ok | {error, <a href="machi_dt.html#type-error_general">machi_dt:error_general()</a>} | {error, term()}</tt><br></p>
 </div><p>Restricted API: Write a chunk of already-sequenced data to
  <code>File</code> at <code>Offset</code>.</p>
 <h3 class="function"><a name="write_chunk-6">write_chunk/6</a></h3>
 <div class="spec">
 <p><tt>write_chunk(Host::<a href="machi_dt.html#type-inet_host">machi_dt:inet_host()</a>, TcpPort::<a href="machi_dt.html#type-inet_port">machi_dt:inet_port()</a>, EpochID::<a href="machi_dt.html#type-epoch_id">machi_dt:epoch_id()</a>, File::<a href="machi_dt.html#type-file_name">machi_dt:file_name()</a>, Offset::<a href="machi_dt.html#type-file_offset">machi_dt:file_offset()</a>, Chunk::<a href="machi_dt.html#type-chunk">machi_dt:chunk()</a>) -&gt; ok | {error, <a href="machi_dt.html#type-error_general">machi_dt:error_general()</a>} | {error, term()}</tt><br></p>
 </div><p>Restricted API: Write a chunk of already-sequenced data to
  <code>File</code> at <code>Offset</code>.</p>
 <h3 class="function"><a name="write_projection-3">write_projection/3</a></h3>
 <div class="spec">
 <p><tt>write_projection(Sock::<a href="#type-port_wrap">port_wrap()</a>, ProjType::<a href="machi_dt.html#type-projection_type">machi_dt:projection_type()</a>, Proj::<a href="machi_dt.html#type-projection">machi_dt:projection()</a>) -&gt; ok | {error, written} | {error, term()}</tt><br></p>
 </div><p>Write a projection <code>Proj</code> of type <code>ProjType</code>.</p>
 <h3 class="function"><a name="write_projection-4">write_projection/4</a></h3>
 <div class="spec">
 <p><tt>write_projection(Host::<a href="machi_dt.html#type-inet_host">machi_dt:inet_host()</a>, TcpPort::<a href="machi_dt.html#type-inet_port">machi_dt:inet_port()</a>, ProjType::<a href="machi_dt.html#type-projection_type">machi_dt:projection_type()</a>, Proj::<a href="machi_dt.html#type-projection">machi_dt:projection()</a>) -&gt; ok | {error, written} | {error, term()}</tt><br></p>
 </div><p>Write a projection <code>Proj</code> of type <code>ProjType</code>.</p>
 <hr>
 <div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <p><i>Generated by EDoc, Dec 8 2015, 21:37:38.</i></p>
 </body>
 </html>
--- a/edoc/machi_flu_filename_mgr.html
+++ b/edoc/machi_flu_filename_mgr.html
@ -0,0 +1,119 @@
 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
 <html>
 <head>
 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
 <title>Module machi_flu_filename_mgr</title>
 <link rel="stylesheet" type="text/css" href="stylesheet.css" title="EDoc">
 </head>
 <body bgcolor="white">
 <div class="navbar"><a name="#navbar_top"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <hr>
 <h1>Module machi_flu_filename_mgr</h1>
 <ul class="index"><li><a href="#description">Description</a></li><li><a href="#index">Function Index</a></li><li><a href="#functions">Function Details</a></li></ul>This process is responsible for managing filenames assigned to
  prefixes.
 <h2><a name="description">Description</a></h2><p>This process is responsible for managing filenames assigned to
  prefixes. It's started out of <code>machi_flu_psup</code>.</p>
  <p>Supported operations include finding the "current" filename assigned to  
 a prefix. Incrementing the sequence number and returning a new file name  
 and listing all data files assigned to a given prefix.</p>
  <p>All prefixes should have the form of <code>{prefix, P}</code>. Single filename
  return values have the form of <code>{file, F}</code>.</p>
  <h2>Finding the current file associated with a sequence</h2><p>  
 First it looks up the sequence number from the prefix name. If  
 no sequence file is found, it uses 0 as the sequence number and searches  
 for a matching file with the prefix and 0 as the sequence number.  
 If no file is found, the it generates a new filename by incorporating  
 the given prefix, a randomly generated (v4) UUID and 0 as the  
 sequence number.</p>
  If the sequence number is &gt; 0, then the process scans the filesystem
  looking for a filename which matches the prefix and given sequence number and
  returns that.
 <h2><a name="index">Function Index</a></h2>
 <table width="100%" border="1" cellspacing="0" cellpadding="2" summary="function index"><tr><td valign="top"><a href="#child_spec-2">child_spec/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#code_change-3">code_change/3</a></td><td></td></tr>
 <tr><td valign="top"><a href="#find_or_make_filename_from_prefix-3">find_or_make_filename_from_prefix/3</a></td><td>Find the latest available or make a filename from a prefix.</td></tr>
 <tr><td valign="top"><a href="#handle_call-3">handle_call/3</a></td><td></td></tr>
 <tr><td valign="top"><a href="#handle_cast-2">handle_cast/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#handle_info-2">handle_info/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#increment_prefix_sequence-2">increment_prefix_sequence/2</a></td><td>Increment the sequence counter for a given prefix.</td></tr>
 <tr><td valign="top"><a href="#init-1">init/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#list_files_by_prefix-2">list_files_by_prefix/2</a></td><td>Given a prefix in the form of <code>{prefix, P}</code> return
 all the data files associated with that prefix.</td></tr>
 <tr><td valign="top"><a href="#start_link-2">start_link/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#terminate-2">terminate/2</a></td><td></td></tr>
 </table>
 <h2><a name="functions">Function Details</a></h2>
 <h3 class="function"><a name="child_spec-2">child_spec/2</a></h3>
 <div class="spec">
 <p><tt>child_spec(FluName, DataDir) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="code_change-3">code_change/3</a></h3>
 <div class="spec">
 <p><tt>code_change(OldVsn, State, Extra) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="find_or_make_filename_from_prefix-3">find_or_make_filename_from_prefix/3</a></h3>
 <div class="spec">
 <p><tt>find_or_make_filename_from_prefix(FluName::atom(), EpochId::<a href="#type-pv1_epoch_n">pv1_epoch_n()</a>, Prefix::{prefix, string()}) -&gt; {file, Filename::string()} | {error, Reason::term()} | timeout</tt><br></p>
 </div><p>Find the latest available or make a filename from a prefix. A prefix
 should be in the form of a tagged tuple <code>{prefix, P}</code>. Returns a tagged
 tuple in the form of <code>{file, F}</code> or an <code>{error, Reason}</code></p>
 <h3 class="function"><a name="handle_call-3">handle_call/3</a></h3>
 <div class="spec">
 <p><tt>handle_call(Req, From, S) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="handle_cast-2">handle_cast/2</a></h3>
 <div class="spec">
 <p><tt>handle_cast(Req, State) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="handle_info-2">handle_info/2</a></h3>
 <div class="spec">
 <p><tt>handle_info(Info, State) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="increment_prefix_sequence-2">increment_prefix_sequence/2</a></h3>
 <div class="spec">
 <p><tt>increment_prefix_sequence(FluName::atom(), Prefix::{prefix, string()}) -&gt; ok | {error, Reason::term()} | timeout</tt><br></p>
 </div><p>Increment the sequence counter for a given prefix. Prefix should
 be in the form of <code>{prefix, P}</code>.</p>
 <h3 class="function"><a name="init-1">init/1</a></h3>
 <div class="spec">
 <p><tt>init(X1) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="list_files_by_prefix-2">list_files_by_prefix/2</a></h3>
 <div class="spec">
 <p><tt>list_files_by_prefix(FluName::atom(), Prefix::{prefix, string()}) -&gt; [<a href="file.html#type-name">file:name()</a>] | timeout | {error, Reason::term()}</tt><br></p>
 </div><p>Given a prefix in the form of <code>{prefix, P}</code> return
 all the data files associated with that prefix. Returns
 a list.</p>
 <h3 class="function"><a name="start_link-2">start_link/2</a></h3>
 <div class="spec">
 <p><tt>start_link(FluName, DataDir) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="terminate-2">terminate/2</a></h3>
 <div class="spec">
 <p><tt>terminate(Reason, State) -&gt; any()</tt></p>
 </div>
 <hr>
 <div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <p><i>Generated by EDoc, Dec 8 2015, 21:37:38.</i></p>
 </body>
 </html>
--- a/edoc/machi_flu_metadata_mgr.html
+++ b/edoc/machi_flu_metadata_mgr.html
@ -0,0 +1,123 @@
 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
 <html>
 <head>
 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
 <title>Module machi_flu_metadata_mgr</title>
 <link rel="stylesheet" type="text/css" href="stylesheet.css" title="EDoc">
 </head>
 <body bgcolor="white">
 <div class="navbar"><a name="#navbar_top"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <hr>
 <h1>Module machi_flu_metadata_mgr</h1>
 <ul class="index"><li><a href="#description">Description</a></li><li><a href="#index">Function Index</a></li><li><a href="#functions">Function Details</a></li></ul>This is a metadata service for the machi FLU which currently  
 tracks the mappings between filenames and file proxies.
 <p><b>Behaviours:</b> <a href="gen_server.html"><tt>gen_server</tt></a>.</p>
 <h2><a name="description">Description</a></h2><p>This is a metadata service for the machi FLU which currently  
 tracks the mappings between filenames and file proxies.</p>
  <p>The service takes a given hash space and spreads it out over a  
 pool of N processes which are responsible for 1/Nth the hash  
 space. When a user requests an operation on a particular file  
 the filename is hashed into the hash space and the request  
 forwarded to a particular manager responsible for that slice  
 of the hash space.</p>
  The current hash implementation is <code>erlang:phash2/1</code> which has
  a range between 0..2^27-1 or 134,217,727.
 <h2><a name="index">Function Index</a></h2>
 <table width="100%" border="1" cellspacing="0" cellpadding="2" summary="function index"><tr><td valign="top"><a href="#build_metadata_mgr_name-2">build_metadata_mgr_name/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#child_spec-4">child_spec/4</a></td><td></td></tr>
 <tr><td valign="top"><a href="#code_change-3">code_change/3</a></td><td></td></tr>
 <tr><td valign="top"><a href="#handle_call-3">handle_call/3</a></td><td></td></tr>
 <tr><td valign="top"><a href="#handle_cast-2">handle_cast/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#handle_info-2">handle_info/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#init-1">init/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#lookup_manager_pid-2">lookup_manager_pid/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#lookup_proxy_pid-2">lookup_proxy_pid/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#start_link-4">start_link/4</a></td><td></td></tr>
 <tr><td valign="top"><a href="#start_proxy_pid-2">start_proxy_pid/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#stop_proxy_pid-2">stop_proxy_pid/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#terminate-2">terminate/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#trim_file-2">trim_file/2</a></td><td></td></tr>
 </table>
 <h2><a name="functions">Function Details</a></h2>
 <h3 class="function"><a name="build_metadata_mgr_name-2">build_metadata_mgr_name/2</a></h3>
 <div class="spec">
 <p><tt>build_metadata_mgr_name(FluName, N) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="child_spec-4">child_spec/4</a></h3>
 <div class="spec">
 <p><tt>child_spec(FluName, C, DataDir, N) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="code_change-3">code_change/3</a></h3>
 <div class="spec">
 <p><tt>code_change(OldVsn, State, Extra) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="handle_call-3">handle_call/3</a></h3>
 <div class="spec">
 <p><tt>handle_call(Req, From, State) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="handle_cast-2">handle_cast/2</a></h3>
 <div class="spec">
 <p><tt>handle_cast(Req, State) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="handle_info-2">handle_info/2</a></h3>
 <div class="spec">
 <p><tt>handle_info(Info, State) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="init-1">init/1</a></h3>
 <div class="spec">
 <p><tt>init(X1) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="lookup_manager_pid-2">lookup_manager_pid/2</a></h3>
 <div class="spec">
 <p><tt>lookup_manager_pid(FluName, X2) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="lookup_proxy_pid-2">lookup_proxy_pid/2</a></h3>
 <div class="spec">
 <p><tt>lookup_proxy_pid(FluName, X2) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="start_link-4">start_link/4</a></h3>
 <div class="spec">
 <p><tt>start_link(FluName, Name, DataDir, Num) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="start_proxy_pid-2">start_proxy_pid/2</a></h3>
 <div class="spec">
 <p><tt>start_proxy_pid(FluName, X2) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="stop_proxy_pid-2">stop_proxy_pid/2</a></h3>
 <div class="spec">
 <p><tt>stop_proxy_pid(FluName, X2) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="terminate-2">terminate/2</a></h3>
 <div class="spec">
 <p><tt>terminate(Reason, State) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="trim_file-2">trim_file/2</a></h3>
 <div class="spec">
 <p><tt>trim_file(FluName, X2) -&gt; any()</tt></p>
 </div>
 <hr>
 <div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <p><i>Generated by EDoc, Dec 8 2015, 21:37:38.</i></p>
 </body>
 </html>
--- a/edoc/machi_flu_metadata_mgr_sup.html
+++ b/edoc/machi_flu_metadata_mgr_sup.html
@ -0,0 +1,49 @@
 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
 <html>
 <head>
 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
 <title>Module machi_flu_metadata_mgr_sup</title>
 <link rel="stylesheet" type="text/css" href="stylesheet.css" title="EDoc">
 </head>
 <body bgcolor="white">
 <div class="navbar"><a name="#navbar_top"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <hr>
 <h1>Module machi_flu_metadata_mgr_sup</h1>
 <ul class="index"><li><a href="#description">Description</a></li><li><a href="#index">Function Index</a></li><li><a href="#functions">Function Details</a></li></ul>This is the supervisor for the collection of metadata
  managers.
 <p><b>Behaviours:</b> <a href="supervisor.html"><tt>supervisor</tt></a>.</p>
 <h2><a name="description">Description</a></h2>This is the supervisor for the collection of metadata
  managers. It's started out of <code>machi_flu_psup</code>. It reads an
  application environment variable named <code>metadata_manager_count</code>
  with a default of 10 if it is not set.
 <h2><a name="index">Function Index</a></h2>
 <table width="100%" border="1" cellspacing="0" cellpadding="2" summary="function index"><tr><td valign="top"><a href="#child_spec-3">child_spec/3</a></td><td></td></tr>
 <tr><td valign="top"><a href="#init-1">init/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#start_link-3">start_link/3</a></td><td></td></tr>
 </table>
 <h2><a name="functions">Function Details</a></h2>
 <h3 class="function"><a name="child_spec-3">child_spec/3</a></h3>
 <div class="spec">
 <p><tt>child_spec(FluName, DataDir, N) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="init-1">init/1</a></h3>
 <div class="spec">
 <p><tt>init(X1) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="start_link-3">start_link/3</a></h3>
 <div class="spec">
 <p><tt>start_link(FluName, DataDir, N) -&gt; any()</tt></p>
 </div>
 <hr>
 <div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <p><i>Generated by EDoc, Dec 8 2015, 21:37:38.</i></p>
 </body>
 </html>
--- a/edoc/machi_flu_psup.html
+++ b/edoc/machi_flu_psup.html
@ -0,0 +1,136 @@
 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
 <html>
 <head>
 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
 <title>Module machi_flu_psup</title>
 <link rel="stylesheet" type="text/css" href="stylesheet.css" title="EDoc">
 </head>
 <body bgcolor="white">
 <div class="navbar"><a name="#navbar_top"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <hr>
 <h1>Module machi_flu_psup</h1>
 <ul class="index"><li><a href="#description">Description</a></li><li><a href="#index">Function Index</a></li><li><a href="#functions">Function Details</a></li></ul>Supervisor for Machi FLU servers and their related support  
 servers.
 <p><b>Behaviours:</b> <a href="supervisor.html"><tt>supervisor</tt></a>.</p>
 <h2><a name="description">Description</a></h2><p>Supervisor for Machi FLU servers and their related support  
 servers.</p>
  <p>Our parent supervisor, <a href="machi_flu_sup.html"><code>machi_flu_sup</code></a>, is responsible for
  managing FLUs as a single entity.  However, the actual
  implementation of a FLU includes three major Erlang processes (not
  including support/worker procs): the FLU itself, the FLU's
  projection store, and the FLU's local chain manager.  This
  supervisor is responsible for managing those three major services
  as a single "package", to be started &amp; stopped together.</p>
  <p>The illustration below shows the OTP process supervision tree for
  the Machi application.  Two FLUs are running, called <code>a</code> and <code>b</code>.
  The chain is configured for a third FLU, <code>c</code>, which is not running  
 at this time.</p>
  <p><img src="/machi//images/supervisor-2flus.png"></p>
  <ul>
  <li> The FLU process itself is named <code>a</code>.
  </li>
  <li> The projection store process is named <code>a_pstore</code>.
  </li>
  <li> The chain manager process is named <code>a_chmgr</code>.  The three
       linked subprocesses are long-lived <a href="machi_proxy_flu1_client.html"><code>machi_proxy_flu1_client</code></a> processes for communicating to all
       chain participants' projection stores (including the local
       store <code>a_pstore</code>).
  </li>
  <li> A fourth major process, <code>a_listener</code>, which is responsible for
       listening on a TCP socket and creating new connections.
       Currently, each listener has two processes handling incoming
       requests, one from each chain manager proxy.
  </li>
  <li> Note that the sub-supervisor parent of <code>a</code> and <code>a_listener</code> does
       not have a registered name.
  </li>
  </ul>
 <h2><a name="index">Function Index</a></h2>
 <table width="100%" border="1" cellspacing="0" cellpadding="2" summary="function index"><tr><td valign="top"><a href="#init-1">init/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#make_fitness_regname-1">make_fitness_regname/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#make_flu_regname-1">make_flu_regname/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#make_mgr_supname-1">make_mgr_supname/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#make_p_regname-1">make_p_regname/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#make_package_spec-1">make_package_spec/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#make_package_spec-4">make_package_spec/4</a></td><td></td></tr>
 <tr><td valign="top"><a href="#make_proj_supname-1">make_proj_supname/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#start_flu_package-1">start_flu_package/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#start_flu_package-4">start_flu_package/4</a></td><td></td></tr>
 <tr><td valign="top"><a href="#start_link-4">start_link/4</a></td><td></td></tr>
 <tr><td valign="top"><a href="#stop_flu_package-1">stop_flu_package/1</a></td><td></td></tr>
 </table>
 <h2><a name="functions">Function Details</a></h2>
 <h3 class="function"><a name="init-1">init/1</a></h3>
 <div class="spec">
 <p><tt>init(X1) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="make_fitness_regname-1">make_fitness_regname/1</a></h3>
 <div class="spec">
 <p><tt>make_fitness_regname(FluName) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="make_flu_regname-1">make_flu_regname/1</a></h3>
 <div class="spec">
 <p><tt>make_flu_regname(FluName) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="make_mgr_supname-1">make_mgr_supname/1</a></h3>
 <div class="spec">
 <p><tt>make_mgr_supname(MgrName) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="make_p_regname-1">make_p_regname/1</a></h3>
 <div class="spec">
 <p><tt>make_p_regname(FluName) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="make_package_spec-1">make_package_spec/1</a></h3>
 <div class="spec">
 <p><tt>make_package_spec(P_srvr) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="make_package_spec-4">make_package_spec/4</a></h3>
 <div class="spec">
 <p><tt>make_package_spec(FluName, TcpPort, DataDir, Props) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="make_proj_supname-1">make_proj_supname/1</a></h3>
 <div class="spec">
 <p><tt>make_proj_supname(ProjName) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="start_flu_package-1">start_flu_package/1</a></h3>
 <div class="spec">
 <p><tt>start_flu_package(P_srvr) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="start_flu_package-4">start_flu_package/4</a></h3>
 <div class="spec">
 <p><tt>start_flu_package(FluName, TcpPort, DataDir, Props) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="start_link-4">start_link/4</a></h3>
 <div class="spec">
 <p><tt>start_link(FluName, TcpPort, DataDir, Props) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="stop_flu_package-1">stop_flu_package/1</a></h3>
 <div class="spec">
 <p><tt>stop_flu_package(FluName) -&gt; any()</tt></p>
 </div>
 <hr>
 <div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <p><i>Generated by EDoc, Dec 8 2015, 21:37:38.</i></p>
 </body>
 </html>
--- a/edoc/machi_flu_sup.html
+++ b/edoc/machi_flu_sup.html
@ -0,0 +1,53 @@
 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
 <html>
 <head>
 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
 <title>Module machi_flu_sup</title>
 <link rel="stylesheet" type="text/css" href="stylesheet.css" title="EDoc">
 </head>
 <body bgcolor="white">
 <div class="navbar"><a name="#navbar_top"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <hr>
 <h1>Module machi_flu_sup</h1>
 <ul class="index"><li><a href="#description">Description</a></li><li><a href="#index">Function Index</a></li><li><a href="#functions">Function Details</a></li></ul>Supervisor for Machi FLU servers and their related support  
 servers.
 <p><b>Behaviours:</b> <a href="supervisor.html"><tt>supervisor</tt></a>.</p>
 <h2><a name="description">Description</a></h2><p>Supervisor for Machi FLU servers and their related support  
 servers.</p>
  <p>Responsibility for managing FLU and chain lifecycle after the initial
  application startup is delegated to <a href="machi_lifecycle_mgr.html"><code>machi_lifecycle_mgr</code></a>.</p>
  See <a href="machi_flu_psup.html"><code>machi_flu_psup</code></a> for an illustration of the entire Machi
  application process structure.
 <h2><a name="index">Function Index</a></h2>
 <table width="100%" border="1" cellspacing="0" cellpadding="2" summary="function index"><tr><td valign="top"><a href="#init-1">init/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#load_rc_d_files_from_dir-1">load_rc_d_files_from_dir/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#start_link-0">start_link/0</a></td><td></td></tr>
 </table>
 <h2><a name="functions">Function Details</a></h2>
 <h3 class="function"><a name="init-1">init/1</a></h3>
 <div class="spec">
 <p><tt>init(X1) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="load_rc_d_files_from_dir-1">load_rc_d_files_from_dir/1</a></h3>
 <div class="spec">
 <p><tt>load_rc_d_files_from_dir(Dir) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="start_link-0">start_link/0</a></h3>
 <div class="spec">
 <p><tt>start_link() -&gt; any()</tt></p>
 </div>
 <hr>
 <div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <p><i>Generated by EDoc, Dec 8 2015, 21:37:38.</i></p>
 </body>
 </html>
--- a/edoc/machi_lifecycle_mgr.html
+++ b/edoc/machi_lifecycle_mgr.html
@ -0,0 +1,229 @@
 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
 <html>
 <head>
 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
 <title>Module machi_lifecycle_mgr</title>
 <link rel="stylesheet" type="text/css" href="stylesheet.css" title="EDoc">
 </head>
 <body bgcolor="white">
 <div class="navbar"><a name="#navbar_top"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <hr>
 <h1>Module machi_lifecycle_mgr</h1>
 <ul class="index"><li><a href="#description">Description</a></li><li><a href="#index">Function Index</a></li><li><a href="#functions">Function Details</a></li></ul>Lifecycle manager for Machi FLUs and chains.
 <p><b>Behaviours:</b> <a href="gen_server.html"><tt>gen_server</tt></a>.</p>
 <h2><a name="description">Description</a></h2><p>Lifecycle manager for Machi FLUs and chains.</p>
  <p>Over the lifetime of a Machi cluster, both the number and types of  
 FLUs and chains may change.  The lifecycle manager is responsible  
 for implementing the lifecycle changes as expressed by "policy".  
 In our case, "policy" is created by an external administrative  
 entity that creates and deletes configuration files that define  
 FLUs and chains relative to this local machine.</p>
  <p>The "master configuration" for deciding which FLUs should be
  running on this machine was inspired by BSD UNIX's <code>init(8)</code> and the  
 "rc.d" scheme.  FLU definitions are found in a single directory,  
 with one file per FLU.  Chains are defined similarly, with one  
 definition file per chain.</p>
  <p>If a definition file for a FLU (or chain) exists, then that
  FLU/chain ought to be configured into being and running.  If a
  definition file for a FLU/chain is removed, then that FLU/chain
  should be stopped gracefully.  However, deleting of a file destroys
  information that is stored inside of that file.  Therefore, we will
  <b>not allow arbitrary unlinking of lifecycle config files</b>.  If
  the administrator deletes these config files using <code>unlink(8)</code>  
 directly, then "the warranty has been broken".</p>
  <p>We will rely on using an administrative command to inform the  
 running system to stop and/or delete lifecycle resources.  If the  
 Machi application is not running, sorry, please start Machi first.</p>
  <h3><a name="Wheel_reinvention">Wheel reinvention</a></h3>
  <p>There's a whole mess of configuration management research &amp;
  libraries out there.  I hope to ignore them all by doing something
  quick &amp; dirty &amp; good enough here.  If I fail, then I'll go  
 pay attention to That Other Stuff.</p>
  <h3><a name="A_note_about_policy">A note about policy</a></h3>
  <p>It is outside of the scope of this local lifecycle manager to make  
 decisions about policy or to distribute policy info/files/whatever  
 to other machines.  This is our machine.  There are many like it,  
 but this one is ours.</p>
  <h3><a name="Machi_Application_Variables">Machi Application Variables</a></h3>
  <p>All OTP application environment variables below are defined in the
  <code>machi</code> application.</p>
  <ul>
   <li> <tt>flu_config_dir</tt>: Stores the <code>rc.d-</code>like config files for
        FLU runtime policy.
  </li>
   <li> <tt>flu_data_dir</tt>: Stores the file data and metadata for
        all FLUs.
  </li>
   <li> <tt>chain_config_dir</tt>: Stores the <code>rc.d</code>-like config files for
        chain runtime policy.
  </li>
  </ul>
  <h3><a name="The_FLU_Lifecycle">The FLU Lifecycle</a></h3>
  <p>FLUs on the local machine may be started and stopped, as defined by  
 administrative policy.  In order to do any useful work, however, a  
 running FLU must also be configured to be a member of a replication  
 chain.  Thus, as a practical matter, both a FLU and the chain that  
 the FLU participates in must both be managed by this manager.</p>
  <p>When a new <code>rc.d</code>-style config file is written to the FLU  
 definition directory, a Machi server process will discover the file  
 within a certain period of time, e.g. 15 seconds.  The FLU will be  
 started with the file's specified parameters.  A FLU should be  
 defined and started before configuring its chain membership.</p>
  <p>Usually a FLU is removed implicitly by removing that FLU from the a
  newer definition file for the chain, or by deleting the entire
  chain definition.  If a FLU has been started but never been a chain
  member, then the FLU can be stopped &amp; removed explicitly.</p>
  <p>When a FLU has been removed by policy, the FLU's data files are set  
 aside into a temporary area.  An additional policy command may be  
 used to permanently delete such FLUs' data files, i.e. to reclaim  
 disk space.</p>
  <p>Resources for the FLU are defined in <code>machi_projection.hrl</code>
  in the <code>p_srvr{}</code> record.  The major elements of this record are:</p>
  <ul>
   <li> <tt>name :: atom()</tt>: The name of the FLU.  This name
        should be unique over the lifetime of the administrative
        domain and thus managed by outside policy.  This name must be
        the same as the name of the <code>rc.d</code>-style config file that
        defines the FLU.
  </li>
   <li> <tt>address :: string()</tt>: The DNS hostname or IP address
        used by other servers to communicate with this FLU.
  </li>
   <li> <tt>port :: non_neg_integer() </tt>: The TCP port number that
        the FLU listens to for incoming Protocol Buffers-serialized
        communication.
  </li>
   <li> <tt>props :: property_list()</tt>: A general-purpose property
        list.  Its use is currently fluid &amp; not well-defined yet.
  </li>
  </ul>
  <h3><a name="The_Chain_Lifecycle">The Chain Lifecycle</a></h3>
  <p>If a FLU on the local machine is expected to participate in a
  replication chain, then an <code>rc.d</code>-style chain definition file must  
 also be present on each machine that runs a FLU in the chain.</p>
  <p>Machi's chains are self-managing, via Humming Consensus; see the
  <a href="https://github.com/basho/machi/tree/master/doc/" target="_top"><tt>https://github.com/basho/machi/tree/master/doc/</tt></a> directory for  
 much more detail about Humming Consensus.  After FLUs have received  
 their initial chain configuration for Humming Consensus, the FLUs  
 will manage each other (and the chain) themselves.</p>
  <p>However, Humming Consensus does not handle three chain management  
 problems: 1. specifying the very first chain configuration,  
 2. altering the membership of the chain (adding/removing FLUs from  
 the chain), or 3. stopping the chain permanently.</p>
  <p>FLUs in a new chain should have definition files created on each  
 FLU's respective machine prior to defining their chain.  Similarly,  
 on each machine that hosts a chain member, a chain definition file  
 created.  External policy is responsible for creating each of these  
 files.</p>
  <p>Resources for the chain are defined in <code>machi_projection.hrl</code>
  in the <code>chain_def_v1{}</code> record.  The major elements of this record are:</p>
  <ul>
   <li> <tt>name :: atom()</tt>: The name of the chain.  This name
        should be unique over the lifetime of the administrative
        domain and thus managed by outside policy.  This name must be
        the same as the name of the <code>rc.d</code>-style config file that
        defines the chain.
  </li>
   <li> <tt>mode :: 'ap_mode' | 'cp_mode'</tt>: This is the consistency
        to be used for managing the chain's replicated data: eventual
        consistency and strong consistency, respectively.
  </li>
   <li> <tt>full :: [#p_srvr{}] </tt>: A list of <code>#p_srvr{}</code> records
        to define the full-service members of the chain.
  </li>
   <li> <tt>witnesses :: [#p_srvr{}] </tt>: A list of <code>#p_srvr{}</code> records
        to define the witness-only members of the chain.  Witness servers
        may only be used with strong consistency mode.
  </li>
   <li> <tt>props :: property_list()</tt>: A general-purpose property
        list.  Its use is currently fluid &amp; not well-defined yet.
  </li>
  </ul>
  <h3><a name="Conflicts_with_TCP_ports,_FLU_&amp;_chain_names,_etc">Conflicts with TCP ports, FLU &amp; chain names, etc</a></h3>
  This manager is not responsible for managing conflicts in resource
  namespaces, e.g., TCP port numbers, FLU names, chain names, etc.
  Managing these namespaces is external policy's responsibility.
 <h2><a name="index">Function Index</a></h2>
 <table width="100%" border="1" cellspacing="0" cellpadding="2" summary="function index"><tr><td valign="top"><a href="#code_change-3">code_change/3</a></td><td></td></tr>
 <tr><td valign="top"><a href="#handle_call-3">handle_call/3</a></td><td></td></tr>
 <tr><td valign="top"><a href="#handle_cast-2">handle_cast/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#handle_info-2">handle_info/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#init-1">init/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#start_link-0">start_link/0</a></td><td></td></tr>
 <tr><td valign="top"><a href="#terminate-2">terminate/2</a></td><td></td></tr>
 </table>
 <h2><a name="functions">Function Details</a></h2>
 <h3 class="function"><a name="code_change-3">code_change/3</a></h3>
 <div class="spec">
 <p><tt>code_change(OldVsn, State, Extra) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="handle_call-3">handle_call/3</a></h3>
 <div class="spec">
 <p><tt>handle_call(Request, From, State) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="handle_cast-2">handle_cast/2</a></h3>
 <div class="spec">
 <p><tt>handle_cast(Msg, State) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="handle_info-2">handle_info/2</a></h3>
 <div class="spec">
 <p><tt>handle_info(Info, State) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="init-1">init/1</a></h3>
 <div class="spec">
 <p><tt>init(X1) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="start_link-0">start_link/0</a></h3>
 <div class="spec">
 <p><tt>start_link() -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="terminate-2">terminate/2</a></h3>
 <div class="spec">
 <p><tt>terminate(Reason, State) -&gt; any()</tt></p>
 </div>
 <hr>
 <div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <p><i>Generated by EDoc, Dec 8 2015, 21:37:38.</i></p>
 </body>
 </html>
--- a/edoc/machi_merkle_tree.html
+++ b/edoc/machi_merkle_tree.html
@ -0,0 +1,69 @@
 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
 <html>
 <head>
 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
 <title>Module machi_merkle_tree</title>
 <link rel="stylesheet" type="text/css" href="stylesheet.css" title="EDoc">
 </head>
 <body bgcolor="white">
 <div class="navbar"><a name="#navbar_top"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <hr>
 <h1>Module machi_merkle_tree</h1>
 <ul class="index"><li><a href="#description">Description</a></li><li><a href="#index">Function Index</a></li><li><a href="#functions">Function Details</a></li></ul>Creates a Merkle tree per file based on the checksum data for  
 a given data file.
 <h2><a name="description">Description</a></h2><p>Creates a Merkle tree per file based on the checksum data for  
 a given data file.</p>
  <p>The <code>naive</code> implementation representation is:</p>
  <p><code>&lt;&lt;Length:64, Offset:32, 0&gt;&gt;</code> for unwritten bytes
  <code>&lt;&lt;Length:64, Offset:32, 1&gt;&gt;</code> for trimmed bytes
  <code>&lt;&lt;Length:64, Offset:32, Csum/binary&gt;&gt;</code> for written bytes</p>
  The tree feeds these leaf nodes into hashes representing chunks of a minimum
  size of at least 1024 KB (1 MB), but if the file size is larger, we will try
  to get about 100 chunks for the first rollup "Level 1." We aim for around 10
  hashes at level 2, and then 2 hashes level 3 and finally the root.
 <h2><a name="index">Function Index</a></h2>
 <table width="100%" border="1" cellspacing="0" cellpadding="2" summary="function index"><tr><td valign="top"><a href="#diff-2">diff/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#filename-1">filename/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#open-2">open/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#open-3">open/3</a></td><td></td></tr>
 <tr><td valign="top"><a href="#tree-1">tree/1</a></td><td></td></tr>
 </table>
 <h2><a name="functions">Function Details</a></h2>
 <h3 class="function"><a name="diff-2">diff/2</a></h3>
 <div class="spec">
 <p><tt>diff(Mt, X2) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="filename-1">filename/1</a></h3>
 <div class="spec">
 <p><tt>filename(Mt) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="open-2">open/2</a></h3>
 <div class="spec">
 <p><tt>open(Filename, DataDir) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="open-3">open/3</a></h3>
 <div class="spec">
 <p><tt>open(Filename, DataDir, Type) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="tree-1">tree/1</a></h3>
 <div class="spec">
 <p><tt>tree(Mt) -&gt; any()</tt></p>
 </div>
 <hr>
 <div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <p><i>Generated by EDoc, Dec 8 2015, 21:37:38.</i></p>
 </body>
 </html>
--- a/edoc/machi_pb_high_client.html
+++ b/edoc/machi_pb_high_client.html
@ -0,0 +1,204 @@
 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
 <html>
 <head>
 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
 <title>Module machi_pb_high_client</title>
 <link rel="stylesheet" type="text/css" href="stylesheet.css" title="EDoc">
 </head>
 <body bgcolor="white">
 <div class="navbar"><a name="#navbar_top"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <hr>
 <h1>Module machi_pb_high_client</h1>
 <ul class="index"><li><a href="#description">Description</a></li><li><a href="#types">Data Types</a></li><li><a href="#index">Function Index</a></li><li><a href="#functions">Function Details</a></li></ul>Machi PB (Protocol Buffers) high-level client (prototype, API TBD).
 <h2><a name="description">Description</a></h2><p>Machi PB (Protocol Buffers) high-level client (prototype, API TBD)</p>
  At the moment, this is brittle-prototype-quality code: the API is not yet
  fully specified, there is very little error handling with respect
  to a single socket connection, and there is no code to deal with
  multiple connections/load balancing/error handling to several/all
  Machi cluster servers.
 <h2><a name="types">Data Types</a></h2>
 <h3 class="typedecl"><a name="type-machi_client_error_reason">machi_client_error_reason()</a></h3>
 <p><tt>machi_client_error_reason() = bad_arg | wedged | bad_checksum | partition | not_written | written | trimmed | no_such_file | partial_read | bad_epoch | <a href="inet.html#type-posix">inet:posix()</a></tt></p>
 <h2><a name="index">Function Index</a></h2>
 <table width="100%" border="1" cellspacing="0" cellpadding="2" summary="function index"><tr><td valign="top"><a href="#append_chunk-6">append_chunk/6</a></td><td></td></tr>
 <tr><td valign="top"><a href="#append_chunk-7">append_chunk/7</a></td><td></td></tr>
 <tr><td valign="top"><a href="#auth-3">auth/3</a></td><td></td></tr>
 <tr><td valign="top"><a href="#auth-4">auth/4</a></td><td></td></tr>
 <tr><td valign="top"><a href="#checksum_list-2">checksum_list/2</a></td><td>Returns a binary that has checksums and chunks encoded inside
  (This is because encoding-decoding them are inefficient).</td></tr>
 <tr><td valign="top"><a href="#checksum_list-3">checksum_list/3</a></td><td></td></tr>
 <tr><td valign="top"><a href="#code_change-3">code_change/3</a></td><td></td></tr>
 <tr><td valign="top"><a href="#connected_p-1">connected_p/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#convert_general_status_code-1">convert_general_status_code/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#echo-2">echo/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#echo-3">echo/3</a></td><td></td></tr>
 <tr><td valign="top"><a href="#handle_call-3">handle_call/3</a></td><td></td></tr>
 <tr><td valign="top"><a href="#handle_cast-2">handle_cast/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#handle_info-2">handle_info/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#init-1">init/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#list_files-1">list_files/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#list_files-2">list_files/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#quit-1">quit/1</a></td><td>Stops a client process.</td></tr>
 <tr><td valign="top"><a href="#read_chunk-5">read_chunk/5</a></td><td>Tries to read a chunk of a specified file.</td></tr>
 <tr><td valign="top"><a href="#read_chunk-6">read_chunk/6</a></td><td></td></tr>
 <tr><td valign="top"><a href="#start_link-1">start_link/1</a></td><td>Creates a client process.</td></tr>
 <tr><td valign="top"><a href="#terminate-2">terminate/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#trim_chunk-4">trim_chunk/4</a></td><td>Trims arbitrary binary range of any file.</td></tr>
 <tr><td valign="top"><a href="#trim_chunk-5">trim_chunk/5</a></td><td></td></tr>
 <tr><td valign="top"><a href="#write_chunk-5">write_chunk/5</a></td><td></td></tr>
 <tr><td valign="top"><a href="#write_chunk-6">write_chunk/6</a></td><td></td></tr>
 </table>
 <h2><a name="functions">Function Details</a></h2>
 <h3 class="function"><a name="append_chunk-6">append_chunk/6</a></h3>
 <div class="spec">
 <p><tt>append_chunk(PidSpec::pid(), PlacementKey::binary(), Prefix::binary(), Chunk::binary(), CSum::binary(), ChunkExtra::non_neg_integer()) -&gt; {ok, Filename::string(), Offset::<a href="machi_dt.html#type-file_offset">machi_dt:file_offset()</a>} | {error, <a href="#type-machi_client_error_reason">machi_client_error_reason()</a>}</tt><br></p>
 </div>
 <h3 class="function"><a name="append_chunk-7">append_chunk/7</a></h3>
 <div class="spec">
 <p><tt>append_chunk(PidSpec::pid(), PlacementKey::binary(), Prefix::binary(), Chunk::binary(), CSum::binary(), ChunkExtra::non_neg_integer(), Timeout::non_neg_integer()) -&gt; {ok, Filename::string(), Offset::<a href="machi_dt.html#type-file_offset">machi_dt:file_offset()</a>} | {error, <a href="#type-machi_client_error_reason">machi_client_error_reason()</a>}</tt><br></p>
 </div>
 <h3 class="function"><a name="auth-3">auth/3</a></h3>
 <div class="spec">
 <p><tt>auth(PidSpec::pid(), User::string(), Pass::string()) -&gt; ok | {error, <a href="#type-machi_client_error_reason">machi_client_error_reason()</a>}</tt><br></p>
 </div>
 <h3 class="function"><a name="auth-4">auth/4</a></h3>
 <div class="spec">
 <p><tt>auth(PidSpec::pid(), User::string(), Pass::string(), Timeout::non_neg_integer()) -&gt; ok | {error, <a href="#type-machi_client_error_reason">machi_client_error_reason()</a>}</tt><br></p>
 </div>
 <h3 class="function"><a name="checksum_list-2">checksum_list/2</a></h3>
 <div class="spec">
 <p><tt>checksum_list(PidSpec::pid(), File::string()) -&gt; {ok, binary()} | {error, <a href="#type-machi_client_error_reason">machi_client_error_reason()</a>}</tt><br></p>
 </div><p>Returns a binary that has checksums and chunks encoded inside
  (This is because encoding-decoding them are inefficient). TODO:
  return a structured list of them.</p>
 <h3 class="function"><a name="checksum_list-3">checksum_list/3</a></h3>
 <div class="spec">
 <p><tt>checksum_list(PidSpec, File, Timeout) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="code_change-3">code_change/3</a></h3>
 <div class="spec">
 <p><tt>code_change(OldVsn, S, Extra) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="connected_p-1">connected_p/1</a></h3>
 <div class="spec">
 <p><tt>connected_p(PidSpec) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="convert_general_status_code-1">convert_general_status_code/1</a></h3>
 <div class="spec">
 <p><tt>convert_general_status_code(X1) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="echo-2">echo/2</a></h3>
 <div class="spec">
 <p><tt>echo(PidSpec::pid(), String::string()) -&gt; {ok, string()} | {error, <a href="#type-machi_client_error_reason">machi_client_error_reason()</a>}</tt><br></p>
 </div>
 <h3 class="function"><a name="echo-3">echo/3</a></h3>
 <div class="spec">
 <p><tt>echo(PidSpec::pid(), String::string(), Timeout::non_neg_integer()) -&gt; {ok, string()} | {error, <a href="#type-machi_client_error_reason">machi_client_error_reason()</a>}</tt><br></p>
 </div>
 <h3 class="function"><a name="handle_call-3">handle_call/3</a></h3>
 <div class="spec">
 <p><tt>handle_call(Request, From, S) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="handle_cast-2">handle_cast/2</a></h3>
 <div class="spec">
 <p><tt>handle_cast(Msg, S) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="handle_info-2">handle_info/2</a></h3>
 <div class="spec">
 <p><tt>handle_info(Info, S) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="init-1">init/1</a></h3>
 <div class="spec">
 <p><tt>init(X1) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="list_files-1">list_files/1</a></h3>
 <div class="spec">
 <p><tt>list_files(PidSpec) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="list_files-2">list_files/2</a></h3>
 <div class="spec">
 <p><tt>list_files(PidSpec, Timeout) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="quit-1">quit/1</a></h3>
 <div class="spec">
 <p><tt>quit(PidSpec::pid()) -&gt; ok</tt><br></p>
 </div><p>Stops a client process.</p>
 <h3 class="function"><a name="read_chunk-5">read_chunk/5</a></h3>
 <div class="spec">
 <p><tt>read_chunk(PidSpec::pid(), File::string(), Offset::<a href="machi_dt.html#type-file_offset">machi_dt:file_offset()</a>, Size::<a href="machi_dt.html#type-chunk_size">machi_dt:chunk_size()</a>, Options::[{flag_no_checksum | flag_no_chunk | needs_trimmed, boolean()}]) -&gt; {ok, {Chunks::[{File::string(), <a href="machi_dt.html#type-file_offset">machi_dt:file_offset()</a>, <a href="machi_dt.html#type-chunk_size">machi_dt:chunk_size()</a>, binary()}], Trimmed::[{File::string(), <a href="machi_dt.html#type-file_offset">machi_dt:file_offset()</a>, <a href="machi_dt.html#type-chunk_size">machi_dt:chunk_size()</a>}]}} | {error, <a href="#type-machi_client_error_reason">machi_client_error_reason()</a>}</tt><br></p>
 </div><p>Tries to read a chunk of a specified file. It returns <code>{ok,
  {Chunks, TrimmedChunks}}</code> for live file while it returns <code>{error,
  trimmed}</code> if all bytes of the file was trimmed.</p>
 <h3 class="function"><a name="read_chunk-6">read_chunk/6</a></h3>
 <div class="spec">
 <p><tt>read_chunk(PidSpec::pid(), File::string(), Offset::<a href="machi_dt.html#type-file_offset">machi_dt:file_offset()</a>, Size::<a href="machi_dt.html#type-chunk_size">machi_dt:chunk_size()</a>, Options::[{flag_no_checksum | flag_no_chunk | needs_trimmed, boolean()}], Timeout::non_neg_integer()) -&gt; {ok, {Chunks::[{File::string(), <a href="machi_dt.html#type-file_offset">machi_dt:file_offset()</a>, <a href="machi_dt.html#type-chunk_size">machi_dt:chunk_size()</a>, binary()}], Trimmed::[{File::string(), <a href="machi_dt.html#type-file_offset">machi_dt:file_offset()</a>, <a href="machi_dt.html#type-chunk_size">machi_dt:chunk_size()</a>}]}} | {error, <a href="#type-machi_client_error_reason">machi_client_error_reason()</a>}</tt><br></p>
 </div>
 <h3 class="function"><a name="start_link-1">start_link/1</a></h3>
 <div class="spec">
 <p><tt>start_link(P_srvr_list::<a href="#type-p_srvr_dict">p_srvr_dict()</a>) -&gt; {ok, pid()} | {error, <a href="#type-machi_client_error_reason">machi_client_error_reason()</a>}</tt><br></p>
 </div><p>Creates a client process</p>
 <h3 class="function"><a name="terminate-2">terminate/2</a></h3>
 <div class="spec">
 <p><tt>terminate(Reason, S) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="trim_chunk-4">trim_chunk/4</a></h3>
 <div class="spec">
 <p><tt>trim_chunk(PidSpec::pid(), File::string(), Offset::non_neg_integer(), Size::<a href="machi_dt.html#type-chunk_size">machi_dt:chunk_size()</a>) -&gt; ok | {error, <a href="#type-machi_client_error_reason">machi_client_error_reason()</a>}</tt><br></p>
 </div><p>Trims arbitrary binary range of any file. If a specified range
  has any byte trimmed, it fails and returns <code>{error, trimmed}</code>.
  Otherwise it trims all bytes in that range. If there are
  overlapping chunks with client-specified checksum, they will cut
  off and checksum are re-calculated in server side.  TODO: Add
  option specifying whether to trigger GC.</p>
 <h3 class="function"><a name="trim_chunk-5">trim_chunk/5</a></h3>
 <div class="spec">
 <p><tt>trim_chunk(PidSpec, File, Offset, Size, Timeout) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="write_chunk-5">write_chunk/5</a></h3>
 <div class="spec">
 <p><tt>write_chunk(PidSpec::pid(), File::string(), Offset::<a href="machi_dt.html#type-file_offset">machi_dt:file_offset()</a>, Chunk::binary(), CSum::binary()) -&gt; ok | {error, <a href="#type-machi_client_error_reason">machi_client_error_reason()</a>}</tt><br></p>
 </div>
 <h3 class="function"><a name="write_chunk-6">write_chunk/6</a></h3>
 <div class="spec">
 <p><tt>write_chunk(PidSpec::pid(), File::string(), Offset::<a href="machi_dt.html#type-file_offset">machi_dt:file_offset()</a>, Chunk::binary(), CSum::binary(), Timeout::non_neg_integer()) -&gt; ok | {error, <a href="#type-machi_client_error_reason">machi_client_error_reason()</a>}</tt><br></p>
 </div>
 <hr>
 <div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <p><i>Generated by EDoc, Dec 8 2015, 21:37:38.</i></p>
 </body>
 </html>
--- a/edoc/machi_pb_server.html
+++ b/edoc/machi_pb_server.html
@ -0,0 +1,31 @@
 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
 <html>
 <head>
 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
 <title>Module machi_pb_server</title>
 <link rel="stylesheet" type="text/css" href="stylesheet.css" title="EDoc">
 </head>
 <body bgcolor="white">
 <div class="navbar"><a name="#navbar_top"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <hr>
 <h1>Module machi_pb_server</h1>
 <ul class="index"><li><a href="#index">Function Index</a></li><li><a href="#functions">Function Details</a></li></ul>
 <h2><a name="index">Function Index</a></h2>
 <table width="100%" border="1" cellspacing="0" cellpadding="2" summary="function index"><tr><td valign="top"><a href="#run_loop-2">run_loop/2</a></td><td></td></tr>
 </table>
 <h2><a name="functions">Function Details</a></h2>
 <h3 class="function"><a name="run_loop-2">run_loop/2</a></h3>
 <div class="spec">
 <p><tt>run_loop(Sock, P_srvr_list) -&gt; any()</tt></p>
 </div>
 <hr>
 <div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <p><i>Generated by EDoc, Jun 25 2015, 16:39:10.</i></p>
 </body>
 </html>
--- a/edoc/machi_pb_translate.html
+++ b/edoc/machi_pb_translate.html
@ -0,0 +1,55 @@
 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
 <html>
 <head>
 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
 <title>Module machi_pb_translate</title>
 <link rel="stylesheet" type="text/css" href="stylesheet.css" title="EDoc">
 </head>
 <body bgcolor="white">
 <div class="navbar"><a name="#navbar_top"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <hr>
 <h1>Module machi_pb_translate</h1>
 <ul class="index"><li><a href="#index">Function Index</a></li><li><a href="#functions">Function Details</a></li></ul>
 <h2><a name="index">Function Index</a></h2>
 <table width="100%" border="1" cellspacing="0" cellpadding="2" summary="function index"><tr><td valign="top"><a href="#conv_to_csum_tag-1">conv_to_csum_tag/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#from_pb_request-1">from_pb_request/1</a></td><td>Adapt impedence mismatches between Erlang and Protocol Buffers.</td></tr>
 <tr><td valign="top"><a href="#from_pb_response-1">from_pb_response/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#to_pb_request-2">to_pb_request/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#to_pb_response-3">to_pb_response/3</a></td><td></td></tr>
 </table>
 <h2><a name="functions">Function Details</a></h2>
 <h3 class="function"><a name="conv_to_csum_tag-1">conv_to_csum_tag/1</a></h3>
 <div class="spec">
 <p><tt>conv_to_csum_tag(Type) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="from_pb_request-1">from_pb_request/1</a></h3>
 <div class="spec">
 <p><tt>from_pb_request(Mpb_ll_request) -&gt; any()</tt></p>
 </div><p>Adapt impedence mismatches between Erlang and Protocol Buffers.</p>
 <h3 class="function"><a name="from_pb_response-1">from_pb_response/1</a></h3>
 <div class="spec">
 <p><tt>from_pb_response(Mpb_ll_response) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="to_pb_request-2">to_pb_request/2</a></h3>
 <div class="spec">
 <p><tt>to_pb_request(ReqID, X2) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="to_pb_response-3">to_pb_response/3</a></h3>
 <div class="spec">
 <p><tt>to_pb_response(ReqID, X2, X) -&gt; any()</tt></p>
 </div>
 <hr>
 <div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <p><i>Generated by EDoc, Dec 8 2015, 21:37:38.</i></p>
 </body>
 </html>
--- a/edoc/machi_pb_wrap.html
+++ b/edoc/machi_pb_wrap.html
@ -0,0 +1,77 @@
 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
 <html>
 <head>
 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
 <title>Module machi_pb_wrap</title>
 <link rel="stylesheet" type="text/css" href="stylesheet.css" title="EDoc">
 </head>
 <body bgcolor="white">
 <div class="navbar"><a name="#navbar_top"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <hr>
 <h1>Module machi_pb_wrap</h1>
 <ul class="index"><li><a href="#description">Description</a></li><li><a href="#index">Function Index</a></li><li><a href="#functions">Function Details</a></li></ul>Wrappers for Protocol Buffers encoding, including hacks to fix
       impedance mismatches between Erlang terms and PB encodings.
 <h2><a name="description">Description</a></h2>Wrappers for Protocol Buffers encoding, including hacks to fix
       impedance mismatches between Erlang terms and PB encodings.
 <h2><a name="index">Function Index</a></h2>
 <table width="100%" border="1" cellspacing="0" cellpadding="2" summary="function index"><tr><td valign="top"><a href="#dec_p_srvr-1">dec_p_srvr/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#dec_projection_v1-1">dec_projection_v1/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#enc_p_srvr-1">enc_p_srvr/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#enc_projection_v1-1">enc_projection_v1/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#make_projection_req-2">make_projection_req/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#make_projection_resp-3">make_projection_resp/3</a></td><td></td></tr>
 <tr><td valign="top"><a href="#unmake_projection_req-1">unmake_projection_req/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#unmake_projection_resp-1">unmake_projection_resp/1</a></td><td></td></tr>
 </table>
 <h2><a name="functions">Function Details</a></h2>
 <h3 class="function"><a name="dec_p_srvr-1">dec_p_srvr/1</a></h3>
 <div class="spec">
 <p><tt>dec_p_srvr(Bin) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="dec_projection_v1-1">dec_projection_v1/1</a></h3>
 <div class="spec">
 <p><tt>dec_projection_v1(Bin) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="enc_p_srvr-1">enc_p_srvr/1</a></h3>
 <div class="spec">
 <p><tt>enc_p_srvr(P) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="enc_projection_v1-1">enc_projection_v1/1</a></h3>
 <div class="spec">
 <p><tt>enc_projection_v1(P) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="make_projection_req-2">make_projection_req/2</a></h3>
 <div class="spec">
 <p><tt>make_projection_req(ID, X2) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="make_projection_resp-3">make_projection_resp/3</a></h3>
 <div class="spec">
 <p><tt>make_projection_resp(ID, X2, Status) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="unmake_projection_req-1">unmake_projection_req/1</a></h3>
 <div class="spec">
 <p><tt>unmake_projection_req(Mpb_ll_request) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="unmake_projection_resp-1">unmake_projection_resp/1</a></h3>
 <div class="spec">
 <p><tt>unmake_projection_resp(Mpb_ll_response) -&gt; any()</tt></p>
 </div>
 <hr>
 <div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <p><i>Generated by EDoc, Jun 25 2015, 16:39:10.</i></p>
 </body>
 </html>
--- a/edoc/machi_plist.html
+++ b/edoc/machi_plist.html
@ -0,0 +1,61 @@
 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
 <html>
 <head>
 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
 <title>Module machi_plist</title>
 <link rel="stylesheet" type="text/css" href="stylesheet.css" title="EDoc">
 </head>
 <body bgcolor="white">
 <div class="navbar"><a name="#navbar_top"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <hr>
 <h1>Module machi_plist</h1>
 <ul class="index"><li><a href="#types">Data Types</a></li><li><a href="#index">Function Index</a></li><li><a href="#functions">Function Details</a></li></ul>
 <h2><a name="types">Data Types</a></h2>
 <h3 class="typedecl"><a name="type-plist">plist()</a></h3>
 <p><tt>plist() = #machi_plist{}</tt></p>
 <h2><a name="index">Function Index</a></h2>
 <table width="100%" border="1" cellspacing="0" cellpadding="2" summary="function index"><tr><td valign="top"><a href="#add-2">add/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#all-1">all/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#close-1">close/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#find-2">find/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#open-2">open/2</a></td><td>persistent list of binaries.</td></tr>
 </table>
 <h2><a name="functions">Function Details</a></h2>
 <h3 class="function"><a name="add-2">add/2</a></h3>
 <div class="spec">
 <p><tt>add(Plist::<a href="#type-plist">plist()</a>, Name::string()) -&gt; {ok, <a href="#type-plist">plist()</a>} | {error, <a href="file.html#type-posix">file:posix()</a>}</tt><br></p>
 </div>
 <h3 class="function"><a name="all-1">all/1</a></h3>
 <div class="spec">
 <p><tt>all(Machi_plist::<a href="#type-plist">plist()</a>) -&gt; [<a href="file.html#type-filename">file:filename()</a>]</tt><br></p>
 </div>
 <h3 class="function"><a name="close-1">close/1</a></h3>
 <div class="spec">
 <p><tt>close(Machi_plist::<a href="#type-plist">plist()</a>) -&gt; ok</tt><br></p>
 </div>
 <h3 class="function"><a name="find-2">find/2</a></h3>
 <div class="spec">
 <p><tt>find(Machi_plist::<a href="#type-plist">plist()</a>, Name::string()) -&gt; boolean()</tt><br></p>
 </div>
 <h3 class="function"><a name="open-2">open/2</a></h3>
 <div class="spec">
 <p><tt>open(Filename::<a href="file.html#type-filename_all">file:filename_all()</a>, Opt::<a href="proplists.html#type-proplist">proplists:proplist()</a>) -&gt; {ok, <a href="#type-plist">plist()</a>} | {error, <a href="file.html#type-posix">file:posix()</a>}</tt><br></p>
 </div><p>persistent list of binaries</p>
 <hr>
 <div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <p><i>Generated by EDoc, Dec 8 2015, 21:37:38.</i></p>
 </body>
 </html>
--- a/edoc/machi_projection.html
+++ b/edoc/machi_projection.html
@ -0,0 +1,106 @@
 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
 <html>
 <head>
 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
 <title>Module machi_projection</title>
 <link rel="stylesheet" type="text/css" href="stylesheet.css" title="EDoc">
 </head>
 <body bgcolor="white">
 <div class="navbar"><a name="#navbar_top"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <hr>
 <h1>Module machi_projection</h1>
 <ul class="index"><li><a href="#description">Description</a></li><li><a href="#index">Function Index</a></li><li><a href="#functions">Function Details</a></li></ul>API for manipulating Machi projection data structures (i.e., records).
 <h2><a name="description">Description</a></h2>API for manipulating Machi projection data structures (i.e., records).
 <h2><a name="index">Function Index</a></h2>
 <table width="100%" border="1" cellspacing="0" cellpadding="2" summary="function index"><tr><td valign="top"><a href="#compare-2">compare/2</a></td><td>Compare two projection records for equality (assuming that the  
 checksum element has been correctly calculated).</td></tr>
 <tr><td valign="top"><a href="#get_epoch_id-1">get_epoch_id/1</a></td><td>Return the epoch_id of the projection.</td></tr>
 <tr><td valign="top"><a href="#make_epoch_id-1">make_epoch_id/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#make_members_dict-1">make_members_dict/1</a></td><td>Make a <code>p_srvr_dict()</code> out of a list of <code>p_srvr()</code> or out of a
  <code>p_srvr_dict()</code>.</td></tr>
 <tr><td valign="top"><a href="#make_summary-1">make_summary/1</a></td><td>Create a proplist-style summary of a projection record.</td></tr>
 <tr><td valign="top"><a href="#new-6">new/6</a></td><td>Create a new projection record.</td></tr>
 <tr><td valign="top"><a href="#new-7">new/7</a></td><td>Create a new projection record.</td></tr>
 <tr><td valign="top"><a href="#new-8">new/8</a></td><td>Create a new projection record.</td></tr>
 <tr><td valign="top"><a href="#update_checksum-1">update_checksum/1</a></td><td>Update the checksum element of a projection record.</td></tr>
 <tr><td valign="top"><a href="#update_dbg2-2">update_dbg2/2</a></td><td>Update the <code>dbg2</code> element of a projection record.</td></tr>
 </table>
 <h2><a name="functions">Function Details</a></h2>
 <h3 class="function"><a name="compare-2">compare/2</a></h3>
 <div class="spec">
 <p><tt>compare(Projection_v1::#projection_v1{}, Projection_v1::#projection_v1{}) -&gt; integer()</tt><br></p>
 </div><p><p>Compare two projection records for equality (assuming that the  
 checksum element has been correctly calculated).</p>
  The name "compare" is probably too close to "rank"?  This
  comparison has nothing to do with projection ranking.</p>
 <h3 class="function"><a name="get_epoch_id-1">get_epoch_id/1</a></h3>
 <div class="spec">
 <p><tt>get_epoch_id(Projection_v1) -&gt; any()</tt></p>
 </div><p>Return the epoch_id of the projection.</p>
 <h3 class="function"><a name="make_epoch_id-1">make_epoch_id/1</a></h3>
 <div class="spec">
 <p><tt>make_epoch_id(Projection_v1) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="make_members_dict-1">make_members_dict/1</a></h3>
 <div class="spec">
 <p><tt>make_members_dict(Ps::[<a href="#type-p_srvr">p_srvr()</a>] | <a href="#type-p_srvr_dict">p_srvr_dict()</a>) -&gt; <a href="#type-p_srvr_dict">p_srvr_dict()</a></tt><br></p>
 </div><p><p>Make a <code>p_srvr_dict()</code> out of a list of <code>p_srvr()</code> or out of a
  <code>p_srvr_dict()</code>.</p>
  <p>If <code>Ps</code> is a <code>p_srvr_dict()</code>, then this function is usually a
  no-op.  However, if someone has tampered with the list and screwed
  up its order, then we should fix it so <code>orddict</code> can work  
 correctly.</p>
  If <code>Ps</code> is simply <code>list(p_srvr())</code>, in which case we'll convert it
  to a <code>p_srvr_dict()</code>.</p>
 <h3 class="function"><a name="make_summary-1">make_summary/1</a></h3>
 <div class="spec">
 <p><tt>make_summary(Projection_v1) -&gt; any()</tt></p>
 </div><p>Create a proplist-style summary of a projection record.</p>
 <h3 class="function"><a name="new-6">new/6</a></h3>
 <div class="spec">
 <p><tt>new(MyName, MemberDict, Down_list, UPI_list, Repairing_list, Ps) -&gt; any()</tt></p>
 </div><p>Create a new projection record.</p>
 <h3 class="function"><a name="new-7">new/7</a></h3>
 <div class="spec">
 <p><tt>new(EpochNum, MyName, MemberDict, Down_list, UPI_list, Repairing_list, Dbg) -&gt; any()</tt></p>
 </div><p>Create a new projection record.</p>
 <h3 class="function"><a name="new-8">new/8</a></h3>
 <div class="spec">
 <p><tt>new(EpochNum, MyName, MembersDict0, Down_list, UPI_list, Repairing_list, Dbg, Dbg2) -&gt; any()</tt></p>
 </div><p><p>Create a new projection record.</p>
  The <code>MemberDict0</code> argument may be a true <code>p_srvr_dict()</code> (i.e, it
  is a well-formed <code>orddict</code> with the correct 2-tuple key-value form)
  or it may be simply <code>list(p_srvr())</code>, in which case we'll convert it
  to a <code>p_srvr_dict()</code>.</p>
 <h3 class="function"><a name="update_checksum-1">update_checksum/1</a></h3>
 <div class="spec">
 <p><tt>update_checksum(P) -&gt; any()</tt></p>
 </div><p>Update the checksum element of a projection record.</p>
 <h3 class="function"><a name="update_dbg2-2">update_dbg2/2</a></h3>
 <div class="spec">
 <p><tt>update_dbg2(P, Dbg2) -&gt; any()</tt></p>
 </div><p>Update the <code>dbg2</code> element of a projection record.</p>
 <hr>
 <div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <p><i>Generated by EDoc, Dec 8 2015, 21:37:38.</i></p>
 </body>
 </html>
--- a/edoc/machi_projection_store.html
+++ b/edoc/machi_projection_store.html
@ -0,0 +1,181 @@
 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
 <html>
 <head>
 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
 <title>Module machi_projection_store</title>
 <link rel="stylesheet" type="text/css" href="stylesheet.css" title="EDoc">
 </head>
 <body bgcolor="white">
 <div class="navbar"><a name="#navbar_top"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <hr>
 <h1>Module machi_projection_store</h1>
 <ul class="index"><li><a href="#description">Description</a></li><li><a href="#index">Function Index</a></li><li><a href="#functions">Function Details</a></li></ul>The Machi write-once projection store service.
 <h2><a name="description">Description</a></h2><p>The Machi write-once projection store service.</p>
  <p>This API is gen_server-style message passing, intended for use  
 within a single Erlang node to glue together the projection store  
 server with the node-local process that implements Machi's FLU  
 client access protocol (on the "server side" of the TCP connection).</p>
  <p>All Machi client access to the projection store SHOULD NOT use this
  module's API.  Instead, clients should access indirectly via <a href="machi_cr_client.html"><code>machi_cr_client</code></a>, <a href="machi_proxy_flu1_client.html"><code>machi_proxy_flu1_client</code></a>, or <a href="machi_flu1_client.html"><code>machi_flu1_client</code></a>.</p>
  The projection store is implemented by an Erlang/OTP <code>gen_server</code>
  process that is associated with each FLU.  Conceptually, the
  projection store is an array of write-once registers.  For each
  projection store register, the key is a 2-tuple of an epoch number
  (<code>non_neg_integer()</code> type) and a projection type (<code>public</code> or
  <code>private</code> type); the value is a projection data structure
  (<code>projection_v1()</code> type).
 <h2><a name="index">Function Index</a></h2>
 <table width="100%" border="1" cellspacing="0" cellpadding="2" summary="function index"><tr><td valign="top"><a href="#code_change-3">code_change/3</a></td><td></td></tr>
 <tr><td valign="top"><a href="#get_all_projections-2">get_all_projections/2</a></td><td>Fetch all projection records of type <code>ProjType</code>.</td></tr>
 <tr><td valign="top"><a href="#get_all_projections-3">get_all_projections/3</a></td><td>Fetch all projection records of type <code>ProjType</code>.</td></tr>
 <tr><td valign="top"><a href="#get_latest_epochid-2">get_latest_epochid/2</a></td><td>Fetch the latest epoch number + checksum for type <code>ProjType</code>.</td></tr>
 <tr><td valign="top"><a href="#get_latest_epochid-3">get_latest_epochid/3</a></td><td>Fetch the latest epoch number + checksum for type <code>ProjType</code>.</td></tr>
 <tr><td valign="top"><a href="#get_wedge_notify_pid-1">get_wedge_notify_pid/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#handle_call-3">handle_call/3</a></td><td></td></tr>
 <tr><td valign="top"><a href="#handle_cast-2">handle_cast/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#handle_info-2">handle_info/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#init-1">init/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#list_all_projections-2">list_all_projections/2</a></td><td>Fetch all projection epoch numbers of type <code>ProjType</code>.</td></tr>
 <tr><td valign="top"><a href="#list_all_projections-3">list_all_projections/3</a></td><td>Fetch all projection epoch numbers of type <code>ProjType</code>.</td></tr>
 <tr><td valign="top"><a href="#read-3">read/3</a></td><td>Fetch the projection record type <code>ProjType</code> for epoch number <code>Epoch</code> .</td></tr>
 <tr><td valign="top"><a href="#read-4">read/4</a></td><td>Fetch the projection record type <code>ProjType</code> for epoch number <code>Epoch</code> .</td></tr>
 <tr><td valign="top"><a href="#read_latest_projection-2">read_latest_projection/2</a></td><td>Fetch the latest projection record for type <code>ProjType</code>.</td></tr>
 <tr><td valign="top"><a href="#read_latest_projection-3">read_latest_projection/3</a></td><td>Fetch the latest projection record for type <code>ProjType</code>.</td></tr>
 <tr><td valign="top"><a href="#set_consistency_mode-2">set_consistency_mode/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#set_wedge_notify_pid-2">set_wedge_notify_pid/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#start_link-3">start_link/3</a></td><td>Start a new projection store server.</td></tr>
 <tr><td valign="top"><a href="#terminate-2">terminate/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#write-3">write/3</a></td><td>Write the projection record type <code>ProjType</code> for epoch number <code>Epoch</code> .</td></tr>
 <tr><td valign="top"><a href="#write-4">write/4</a></td><td>Write the projection record type <code>ProjType</code> for epoch number <code>Epoch</code> .</td></tr>
 </table>
 <h2><a name="functions">Function Details</a></h2>
 <h3 class="function"><a name="code_change-3">code_change/3</a></h3>
 <div class="spec">
 <p><tt>code_change(OldVsn, S, Extra) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="get_all_projections-2">get_all_projections/2</a></h3>
 <div class="spec">
 <p><tt>get_all_projections(PidSpec, ProjType) -&gt; any()</tt></p>
 </div><p>Fetch all projection records of type <code>ProjType</code>.</p>
 <h3 class="function"><a name="get_all_projections-3">get_all_projections/3</a></h3>
 <div class="spec">
 <p><tt>get_all_projections(PidSpec, ProjType, Timeout) -&gt; any()</tt></p>
 </div><p>Fetch all projection records of type <code>ProjType</code>.</p>
 <h3 class="function"><a name="get_latest_epochid-2">get_latest_epochid/2</a></h3>
 <div class="spec">
 <p><tt>get_latest_epochid(PidSpec, ProjType) -&gt; any()</tt></p>
 </div><p>Fetch the latest epoch number + checksum for type <code>ProjType</code>.</p>
 <h3 class="function"><a name="get_latest_epochid-3">get_latest_epochid/3</a></h3>
 <div class="spec">
 <p><tt>get_latest_epochid(PidSpec, ProjType, Timeout) -&gt; any()</tt></p>
 </div><p>Fetch the latest epoch number + checksum for type <code>ProjType</code>.
  projection.</p>
 <h3 class="function"><a name="get_wedge_notify_pid-1">get_wedge_notify_pid/1</a></h3>
 <div class="spec">
 <p><tt>get_wedge_notify_pid(PidSpec) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="handle_call-3">handle_call/3</a></h3>
 <div class="spec">
 <p><tt>handle_call(Request, From, S) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="handle_cast-2">handle_cast/2</a></h3>
 <div class="spec">
 <p><tt>handle_cast(Msg, S) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="handle_info-2">handle_info/2</a></h3>
 <div class="spec">
 <p><tt>handle_info(Info, S) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="init-1">init/1</a></h3>
 <div class="spec">
 <p><tt>init(X1) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="list_all_projections-2">list_all_projections/2</a></h3>
 <div class="spec">
 <p><tt>list_all_projections(PidSpec, ProjType) -&gt; any()</tt></p>
 </div><p>Fetch all projection epoch numbers of type <code>ProjType</code>.</p>
 <h3 class="function"><a name="list_all_projections-3">list_all_projections/3</a></h3>
 <div class="spec">
 <p><tt>list_all_projections(PidSpec, ProjType, Timeout) -&gt; any()</tt></p>
 </div><p>Fetch all projection epoch numbers of type <code>ProjType</code>.</p>
 <h3 class="function"><a name="read-3">read/3</a></h3>
 <div class="spec">
 <p><tt>read(PidSpec, ProjType, Epoch) -&gt; any()</tt></p>
 </div><p>Fetch the projection record type <code>ProjType</code> for epoch number <code>Epoch</code> .</p>
 <h3 class="function"><a name="read-4">read/4</a></h3>
 <div class="spec">
 <p><tt>read(PidSpec, ProjType, Epoch, Timeout) -&gt; any()</tt></p>
 </div><p>Fetch the projection record type <code>ProjType</code> for epoch number <code>Epoch</code> .</p>
 <h3 class="function"><a name="read_latest_projection-2">read_latest_projection/2</a></h3>
 <div class="spec">
 <p><tt>read_latest_projection(PidSpec, ProjType) -&gt; any()</tt></p>
 </div><p>Fetch the latest projection record for type <code>ProjType</code>.</p>
 <h3 class="function"><a name="read_latest_projection-3">read_latest_projection/3</a></h3>
 <div class="spec">
 <p><tt>read_latest_projection(PidSpec, ProjType, Timeout) -&gt; any()</tt></p>
 </div><p>Fetch the latest projection record for type <code>ProjType</code>.</p>
 <h3 class="function"><a name="set_consistency_mode-2">set_consistency_mode/2</a></h3>
 <div class="spec">
 <p><tt>set_consistency_mode(PidSpec, CMode) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="set_wedge_notify_pid-2">set_wedge_notify_pid/2</a></h3>
 <div class="spec">
 <p><tt>set_wedge_notify_pid(PidSpec, NotifyWedgeStateChanges) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="start_link-3">start_link/3</a></h3>
 <div class="spec">
 <p><tt>start_link(RegName, DataDir, NotifyWedgeStateChanges) -&gt; any()</tt></p>
 </div><p><p>Start a new projection store server.</p>
  The <code>DataDir</code> argument should be the same directory as specified
  for use by our companion FLU data server -- all file system paths
  used by this server are intended to be stored underneath a common
  file system parent directory as the FLU data server &amp; sequencer
  servers.</p>
 <h3 class="function"><a name="terminate-2">terminate/2</a></h3>
 <div class="spec">
 <p><tt>terminate(Reason, S) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="write-3">write/3</a></h3>
 <div class="spec">
 <p><tt>write(PidSpec, ProjType, Proj) -&gt; any()</tt></p>
 </div><p>Write the projection record type <code>ProjType</code> for epoch number <code>Epoch</code> .</p>
 <h3 class="function"><a name="write-4">write/4</a></h3>
 <div class="spec">
 <p><tt>write(PidSpec, ProjType, Proj, Timeout) -&gt; any()</tt></p>
 </div><p>Write the projection record type <code>ProjType</code> for epoch number <code>Epoch</code> .</p>
 <hr>
 <div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <p><i>Generated by EDoc, Dec 8 2015, 21:37:38.</i></p>
 </body>
 </html>
--- a/edoc/machi_proxy_flu1_client.html
+++ b/edoc/machi_proxy_flu1_client.html
@ -0,0 +1,323 @@
 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
 <html>
 <head>
 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
 <title>Module machi_proxy_flu1_client</title>
 <link rel="stylesheet" type="text/css" href="stylesheet.css" title="EDoc">
 </head>
 <body bgcolor="white">
 <div class="navbar"><a name="#navbar_top"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <hr>
 <h1>Module machi_proxy_flu1_client</h1>
 <ul class="index"><li><a href="#description">Description</a></li><li><a href="#index">Function Index</a></li><li><a href="#functions">Function Details</a></li></ul>Erlang API for the Machi FLU TCP protocol version 1, with a  
 proxy-process style API for hiding messy details such as TCP  
 connection/disconnection with the remote Machi server.
 <p><b>Behaviours:</b> <a href="gen_server.html"><tt>gen_server</tt></a>.</p>
 <h2><a name="description">Description</a></h2><p>Erlang API for the Machi FLU TCP protocol version 1, with a  
 proxy-process style API for hiding messy details such as TCP  
 connection/disconnection with the remote Machi server.</p>
  <p>Machi is intentionally avoiding using distributed Erlang for
  Machi's communication.  This design decision makes Erlang-side code
  more difficult &amp; complex, but it's the price to pay for some  
 language independence.  Later in Machi's life cycle, we may (?) need to  
 (re-)implement some components in a non-Erlang/BEAM-based language.</p>
  <p>This module implements a "man in the middle" proxy between the  
 Erlang client and Machi server (which is on the "far side" of a TCP  
 connection to somewhere).  This proxy process will always execute  
 on the same Erlang node as the Erlang client that uses it.  The  
 proxy is intended to be a stable, long-lived process that survives  
 TCP communication problems with the remote server.</p>
  For a higher level interface, see <a href="machi_cr_client.html"><code>machi_cr_client</code></a>.
  For a lower level interface, see <a href="machi_flu1_client.html"><code>machi_flu1_client</code></a>.
 <h2><a name="index">Function Index</a></h2>
 <table width="100%" border="1" cellspacing="0" cellpadding="2" summary="function index"><tr><td valign="top"><a href="#append_chunk-4">append_chunk/4</a></td><td>Append a chunk (binary- or iolist-style) of data to a file
  with <code>Prefix</code>.</td></tr>
 <tr><td valign="top"><a href="#append_chunk-5">append_chunk/5</a></td><td>Append a chunk (binary- or iolist-style) of data to a file
  with <code>Prefix</code>.</td></tr>
 <tr><td valign="top"><a href="#append_chunk_extra-5">append_chunk_extra/5</a></td><td>Append a chunk (binary- or iolist-style) of data to a file
  with <code>Prefix</code>.</td></tr>
 <tr><td valign="top"><a href="#append_chunk_extra-6">append_chunk_extra/6</a></td><td>Append a chunk (binary- or iolist-style) of data to a file
  with <code>Prefix</code>.</td></tr>
 <tr><td valign="top"><a href="#checksum_list-3">checksum_list/3</a></td><td>Fetch the list of chunk checksums for <code>File</code>.</td></tr>
 <tr><td valign="top"><a href="#checksum_list-4">checksum_list/4</a></td><td>Fetch the list of chunk checksums for <code>File</code>.</td></tr>
 <tr><td valign="top"><a href="#code_change-3">code_change/3</a></td><td></td></tr>
 <tr><td valign="top"><a href="#get_all_projections-2">get_all_projections/2</a></td><td>Get all projections from the FLU's projection store.</td></tr>
 <tr><td valign="top"><a href="#get_all_projections-3">get_all_projections/3</a></td><td>Get all projections from the FLU's projection store.</td></tr>
 <tr><td valign="top"><a href="#get_epoch_id-1">get_epoch_id/1</a></td><td>Get the <code>epoch_id()</code> of the FLU's current/latest projection.</td></tr>
 <tr><td valign="top"><a href="#get_epoch_id-2">get_epoch_id/2</a></td><td>Get the <code>epoch_id()</code> of the FLU's current/latest projection.</td></tr>
 <tr><td valign="top"><a href="#get_latest_epochid-2">get_latest_epochid/2</a></td><td>Get the latest epoch number + checksum from the FLU's projection store.</td></tr>
 <tr><td valign="top"><a href="#get_latest_epochid-3">get_latest_epochid/3</a></td><td>Get the latest epoch number + checksum from the FLU's projection store.</td></tr>
 <tr><td valign="top"><a href="#handle_call-3">handle_call/3</a></td><td></td></tr>
 <tr><td valign="top"><a href="#handle_cast-2">handle_cast/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#handle_info-2">handle_info/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#init-1">init/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#kick_projection_reaction-2">kick_projection_reaction/2</a></td><td>Kick (politely) the remote chain manager to react to a
  projection change.</td></tr>
 <tr><td valign="top"><a href="#kick_projection_reaction-3">kick_projection_reaction/3</a></td><td>Kick (politely) the remote chain manager to react to a
  projection change.</td></tr>
 <tr><td valign="top"><a href="#list_all_projections-2">list_all_projections/2</a></td><td>Get all epoch numbers from the FLU's projection store.</td></tr>
 <tr><td valign="top"><a href="#list_all_projections-3">list_all_projections/3</a></td><td>Get all epoch numbers from the FLU's projection store.</td></tr>
 <tr><td valign="top"><a href="#list_files-2">list_files/2</a></td><td>Fetch the list of all files on the remote FLU.</td></tr>
 <tr><td valign="top"><a href="#list_files-3">list_files/3</a></td><td>Fetch the list of all files on the remote FLU.</td></tr>
 <tr><td valign="top"><a href="#quit-1">quit/1</a></td><td>Quit &amp; close the connection to remote FLU and stop our
  proxy process.</td></tr>
 <tr><td valign="top"><a href="#read_chunk-6">read_chunk/6</a></td><td>Read a chunk of data of size <code>Size</code> from <code>File</code> at <code>Offset</code>.</td></tr>
 <tr><td valign="top"><a href="#read_chunk-7">read_chunk/7</a></td><td>Read a chunk of data of size <code>Size</code> from <code>File</code> at <code>Offset</code>.</td></tr>
 <tr><td valign="top"><a href="#read_latest_projection-2">read_latest_projection/2</a></td><td>Get the latest projection from the FLU's projection store for <code>ProjType</code></td></tr>
 <tr><td valign="top"><a href="#read_latest_projection-3">read_latest_projection/3</a></td><td>Get the latest projection from the FLU's projection store for <code>ProjType</code></td></tr>
 <tr><td valign="top"><a href="#read_projection-3">read_projection/3</a></td><td>Read a projection <code>Proj</code> of type <code>ProjType</code>.</td></tr>
 <tr><td valign="top"><a href="#read_projection-4">read_projection/4</a></td><td>Read a projection <code>Proj</code> of type <code>ProjType</code>.</td></tr>
 <tr><td valign="top"><a href="#start_link-1">start_link/1</a></td><td>Start a local, long-lived process that will be our steady
  &amp; reliable communication proxy with the fickle &amp; flaky
  remote Machi server.</td></tr>
 <tr><td valign="top"><a href="#start_proxies-1">start_proxies/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#stop_proxies-1">stop_proxies/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#terminate-2">terminate/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#trim_chunk-5">trim_chunk/5</a></td><td></td></tr>
 <tr><td valign="top"><a href="#trim_chunk-6">trim_chunk/6</a></td><td>Write a chunk (binary- or iolist-style) of data to a file
  with <code>Prefix</code> at <code>Offset</code>.</td></tr>
 <tr><td valign="top"><a href="#wedge_status-1">wedge_status/1</a></td><td>Fetch the wedge status from the remote FLU.</td></tr>
 <tr><td valign="top"><a href="#wedge_status-2">wedge_status/2</a></td><td>Fetch the wedge status from the remote FLU.</td></tr>
 <tr><td valign="top"><a href="#write_chunk-5">write_chunk/5</a></td><td>Write a chunk (binary- or iolist-style) of data to a file
  with <code>Prefix</code> at <code>Offset</code>.</td></tr>
 <tr><td valign="top"><a href="#write_chunk-6">write_chunk/6</a></td><td>Write a chunk (binary- or iolist-style) of data to a file
  with <code>Prefix</code> at <code>Offset</code>.</td></tr>
 <tr><td valign="top"><a href="#write_projection-3">write_projection/3</a></td><td>Write a projection <code>Proj</code> of type <code>ProjType</code>.</td></tr>
 <tr><td valign="top"><a href="#write_projection-4">write_projection/4</a></td><td>Write a projection <code>Proj</code> of type <code>ProjType</code>.</td></tr>
 </table>
 <h2><a name="functions">Function Details</a></h2>
 <h3 class="function"><a name="append_chunk-4">append_chunk/4</a></h3>
 <div class="spec">
 <p><tt>append_chunk(PidSpec, EpochID, Prefix, Chunk) -&gt; any()</tt></p>
 </div><p>Append a chunk (binary- or iolist-style) of data to a file
  with <code>Prefix</code>.</p>
 <h3 class="function"><a name="append_chunk-5">append_chunk/5</a></h3>
 <div class="spec">
 <p><tt>append_chunk(PidSpec, EpochID, Prefix, Chunk, Timeout) -&gt; any()</tt></p>
 </div><p>Append a chunk (binary- or iolist-style) of data to a file
  with <code>Prefix</code>.</p>
 <h3 class="function"><a name="append_chunk_extra-5">append_chunk_extra/5</a></h3>
 <div class="spec">
 <p><tt>append_chunk_extra(PidSpec, EpochID, Prefix, Chunk, ChunkExtra) -&gt; any()</tt></p>
 </div><p>Append a chunk (binary- or iolist-style) of data to a file
  with <code>Prefix</code>.</p>
 <h3 class="function"><a name="append_chunk_extra-6">append_chunk_extra/6</a></h3>
 <div class="spec">
 <p><tt>append_chunk_extra(PidSpec, EpochID, Prefix, Chunk, ChunkExtra, Timeout) -&gt; any()</tt></p>
 </div><p>Append a chunk (binary- or iolist-style) of data to a file
  with <code>Prefix</code>.</p>
 <h3 class="function"><a name="checksum_list-3">checksum_list/3</a></h3>
 <div class="spec">
 <p><tt>checksum_list(PidSpec, EpochID, File) -&gt; any()</tt></p>
 </div><p>Fetch the list of chunk checksums for <code>File</code>.</p>
 <h3 class="function"><a name="checksum_list-4">checksum_list/4</a></h3>
 <div class="spec">
 <p><tt>checksum_list(PidSpec, EpochID, File, Timeout) -&gt; any()</tt></p>
 </div><p>Fetch the list of chunk checksums for <code>File</code>.</p>
 <h3 class="function"><a name="code_change-3">code_change/3</a></h3>
 <div class="spec">
 <p><tt>code_change(OldVsn, S, Extra) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="get_all_projections-2">get_all_projections/2</a></h3>
 <div class="spec">
 <p><tt>get_all_projections(PidSpec, ProjType) -&gt; any()</tt></p>
 </div><p>Get all projections from the FLU's projection store.</p>
 <h3 class="function"><a name="get_all_projections-3">get_all_projections/3</a></h3>
 <div class="spec">
 <p><tt>get_all_projections(PidSpec, ProjType, Timeout) -&gt; any()</tt></p>
 </div><p>Get all projections from the FLU's projection store.</p>
 <h3 class="function"><a name="get_epoch_id-1">get_epoch_id/1</a></h3>
 <div class="spec">
 <p><tt>get_epoch_id(PidSpec) -&gt; any()</tt></p>
 </div><p>Get the <code>epoch_id()</code> of the FLU's current/latest projection.</p>
 <h3 class="function"><a name="get_epoch_id-2">get_epoch_id/2</a></h3>
 <div class="spec">
 <p><tt>get_epoch_id(PidSpec, Timeout) -&gt; any()</tt></p>
 </div><p>Get the <code>epoch_id()</code> of the FLU's current/latest projection.</p>
 <h3 class="function"><a name="get_latest_epochid-2">get_latest_epochid/2</a></h3>
 <div class="spec">
 <p><tt>get_latest_epochid(PidSpec, ProjType) -&gt; any()</tt></p>
 </div><p>Get the latest epoch number + checksum from the FLU's projection store.</p>
 <h3 class="function"><a name="get_latest_epochid-3">get_latest_epochid/3</a></h3>
 <div class="spec">
 <p><tt>get_latest_epochid(PidSpec, ProjType, Timeout) -&gt; any()</tt></p>
 </div><p>Get the latest epoch number + checksum from the FLU's projection store.</p>
 <h3 class="function"><a name="handle_call-3">handle_call/3</a></h3>
 <div class="spec">
 <p><tt>handle_call(Request, From, S) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="handle_cast-2">handle_cast/2</a></h3>
 <div class="spec">
 <p><tt>handle_cast(Msg, S) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="handle_info-2">handle_info/2</a></h3>
 <div class="spec">
 <p><tt>handle_info(Info, S) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="init-1">init/1</a></h3>
 <div class="spec">
 <p><tt>init(X1) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="kick_projection_reaction-2">kick_projection_reaction/2</a></h3>
 <div class="spec">
 <p><tt>kick_projection_reaction(PidSpec, Options) -&gt; any()</tt></p>
 </div><p>Kick (politely) the remote chain manager to react to a
  projection change.</p>
 <h3 class="function"><a name="kick_projection_reaction-3">kick_projection_reaction/3</a></h3>
 <div class="spec">
 <p><tt>kick_projection_reaction(PidSpec, Options, Timeout) -&gt; any()</tt></p>
 </div><p>Kick (politely) the remote chain manager to react to a
  projection change.</p>
 <h3 class="function"><a name="list_all_projections-2">list_all_projections/2</a></h3>
 <div class="spec">
 <p><tt>list_all_projections(PidSpec, ProjType) -&gt; any()</tt></p>
 </div><p>Get all epoch numbers from the FLU's projection store.</p>
 <h3 class="function"><a name="list_all_projections-3">list_all_projections/3</a></h3>
 <div class="spec">
 <p><tt>list_all_projections(PidSpec, ProjType, Timeout) -&gt; any()</tt></p>
 </div><p>Get all epoch numbers from the FLU's projection store.</p>
 <h3 class="function"><a name="list_files-2">list_files/2</a></h3>
 <div class="spec">
 <p><tt>list_files(PidSpec, EpochID) -&gt; any()</tt></p>
 </div><p>Fetch the list of all files on the remote FLU.</p>
 <h3 class="function"><a name="list_files-3">list_files/3</a></h3>
 <div class="spec">
 <p><tt>list_files(PidSpec, EpochID, Timeout) -&gt; any()</tt></p>
 </div><p>Fetch the list of all files on the remote FLU.</p>
 <h3 class="function"><a name="quit-1">quit/1</a></h3>
 <div class="spec">
 <p><tt>quit(PidSpec) -&gt; any()</tt></p>
 </div><p>Quit &amp; close the connection to remote FLU and stop our
  proxy process.</p>
 <h3 class="function"><a name="read_chunk-6">read_chunk/6</a></h3>
 <div class="spec">
 <p><tt>read_chunk(PidSpec, EpochID, File, Offset, Size, Opts) -&gt; any()</tt></p>
 </div><p>Read a chunk of data of size <code>Size</code> from <code>File</code> at <code>Offset</code>.</p>
 <h3 class="function"><a name="read_chunk-7">read_chunk/7</a></h3>
 <div class="spec">
 <p><tt>read_chunk(PidSpec, EpochID, File, Offset, Size, Opts, Timeout) -&gt; any()</tt></p>
 </div><p>Read a chunk of data of size <code>Size</code> from <code>File</code> at <code>Offset</code>.</p>
 <h3 class="function"><a name="read_latest_projection-2">read_latest_projection/2</a></h3>
 <div class="spec">
 <p><tt>read_latest_projection(PidSpec, ProjType) -&gt; any()</tt></p>
 </div><p>Get the latest projection from the FLU's projection store for <code>ProjType</code></p>
 <h3 class="function"><a name="read_latest_projection-3">read_latest_projection/3</a></h3>
 <div class="spec">
 <p><tt>read_latest_projection(PidSpec, ProjType, Timeout) -&gt; any()</tt></p>
 </div><p>Get the latest projection from the FLU's projection store for <code>ProjType</code></p>
 <h3 class="function"><a name="read_projection-3">read_projection/3</a></h3>
 <div class="spec">
 <p><tt>read_projection(PidSpec, ProjType, Epoch) -&gt; any()</tt></p>
 </div><p>Read a projection <code>Proj</code> of type <code>ProjType</code>.</p>
 <h3 class="function"><a name="read_projection-4">read_projection/4</a></h3>
 <div class="spec">
 <p><tt>read_projection(PidSpec, ProjType, Epoch, Timeout) -&gt; any()</tt></p>
 </div><p>Read a projection <code>Proj</code> of type <code>ProjType</code>.</p>
 <h3 class="function"><a name="start_link-1">start_link/1</a></h3>
 <div class="spec">
 <p><tt>start_link(P_srvr) -&gt; any()</tt></p>
 </div><p>Start a local, long-lived process that will be our steady
  &amp; reliable communication proxy with the fickle &amp; flaky
  remote Machi server.</p>
 <h3 class="function"><a name="start_proxies-1">start_proxies/1</a></h3>
 <div class="spec">
 <p><tt>start_proxies(MembersDict) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="stop_proxies-1">stop_proxies/1</a></h3>
 <div class="spec">
 <p><tt>stop_proxies(ProxiesDict) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="terminate-2">terminate/2</a></h3>
 <div class="spec">
 <p><tt>terminate(Reason, S) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="trim_chunk-5">trim_chunk/5</a></h3>
 <div class="spec">
 <p><tt>trim_chunk(PidSpec, EpochID, File, Offset, Size) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="trim_chunk-6">trim_chunk/6</a></h3>
 <div class="spec">
 <p><tt>trim_chunk(PidSpec, EpochID, File, Offset, Chunk, Timeout) -&gt; any()</tt></p>
 </div><p>Write a chunk (binary- or iolist-style) of data to a file
  with <code>Prefix</code> at <code>Offset</code>.</p>
 <h3 class="function"><a name="wedge_status-1">wedge_status/1</a></h3>
 <div class="spec">
 <p><tt>wedge_status(PidSpec) -&gt; any()</tt></p>
 </div><p>Fetch the wedge status from the remote FLU.</p>
 <h3 class="function"><a name="wedge_status-2">wedge_status/2</a></h3>
 <div class="spec">
 <p><tt>wedge_status(PidSpec, Timeout) -&gt; any()</tt></p>
 </div><p>Fetch the wedge status from the remote FLU.</p>
 <h3 class="function"><a name="write_chunk-5">write_chunk/5</a></h3>
 <div class="spec">
 <p><tt>write_chunk(PidSpec, EpochID, File, Offset, Chunk) -&gt; any()</tt></p>
 </div><p>Write a chunk (binary- or iolist-style) of data to a file
  with <code>Prefix</code> at <code>Offset</code>.</p>
 <h3 class="function"><a name="write_chunk-6">write_chunk/6</a></h3>
 <div class="spec">
 <p><tt>write_chunk(PidSpec, EpochID, File, Offset, Chunk, Timeout) -&gt; any()</tt></p>
 </div><p>Write a chunk (binary- or iolist-style) of data to a file
  with <code>Prefix</code> at <code>Offset</code>.</p>
 <h3 class="function"><a name="write_projection-3">write_projection/3</a></h3>
 <div class="spec">
 <p><tt>write_projection(PidSpec, ProjType, Proj) -&gt; any()</tt></p>
 </div><p>Write a projection <code>Proj</code> of type <code>ProjType</code>.</p>
 <h3 class="function"><a name="write_projection-4">write_projection/4</a></h3>
 <div class="spec">
 <p><tt>write_projection(PidSpec, ProjType, Proj, Timeout) -&gt; any()</tt></p>
 </div><p>Write a projection <code>Proj</code> of type <code>ProjType</code>.</p>
 <hr>
 <div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <p><i>Generated by EDoc, Dec 8 2015, 21:37:38.</i></p>
 </body>
 </html>
--- a/edoc/machi_sequencer.html
+++ b/edoc/machi_sequencer.html
@ -0,0 +1,23 @@
 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
 <html>
 <head>
 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
 <title>Module machi_sequencer</title>
 <link rel="stylesheet" type="text/css" href="stylesheet.css" title="EDoc">
 </head>
 <body bgcolor="white">
 <div class="navbar"><a name="#navbar_top"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <hr>
 <h1>Module machi_sequencer</h1>
 <ul class="index"><li><a href="#description">Description</a></li></ul>"Mothballed" sequencer code, perhaps to be reused sometime in
  the future?.
 <h2><a name="description">Description</a></h2>"Mothballed" sequencer code, perhaps to be reused sometime in
  the future?<hr>
 <div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <p><i>Generated by EDoc, Jul 1 2015, 18:32:19.</i></p>
 </body>
 </html>
--- a/edoc/machi_sup.html
+++ b/edoc/machi_sup.html
@ -0,0 +1,42 @@
 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
 <html>
 <head>
 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
 <title>Module machi_sup</title>
 <link rel="stylesheet" type="text/css" href="stylesheet.css" title="EDoc">
 </head>
 <body bgcolor="white">
 <div class="navbar"><a name="#navbar_top"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <hr>
 <h1>Module machi_sup</h1>
 <ul class="index"><li><a href="#description">Description</a></li><li><a href="#index">Function Index</a></li><li><a href="#functions">Function Details</a></li></ul>Top Machi application supervisor.
 <p><b>Behaviours:</b> <a href="supervisor.html"><tt>supervisor</tt></a>.</p>
 <h2><a name="description">Description</a></h2><p>Top Machi application supervisor.</p>
  See <a href="machi_flu_psup.html"><code>machi_flu_psup</code></a> for an illustration of the entire Machi
  application process structure.
 <h2><a name="index">Function Index</a></h2>
 <table width="100%" border="1" cellspacing="0" cellpadding="2" summary="function index"><tr><td valign="top"><a href="#init-1">init/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#start_link-0">start_link/0</a></td><td></td></tr>
 </table>
 <h2><a name="functions">Function Details</a></h2>
 <h3 class="function"><a name="init-1">init/1</a></h3>
 <div class="spec">
 <p><tt>init(X1) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="start_link-0">start_link/0</a></h3>
 <div class="spec">
 <p><tt>start_link() -&gt; any()</tt></p>
 </div>
 <hr>
 <div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <p><i>Generated by EDoc, Dec 8 2015, 21:37:38.</i></p>
 </body>
 </html>
--- a/edoc/machi_util.html
+++ b/edoc/machi_util.html
@ -0,0 +1,288 @@
 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
 <html>
 <head>
 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
 <title>Module machi_util</title>
 <link rel="stylesheet" type="text/css" href="stylesheet.css" title="EDoc">
 </head>
 <body bgcolor="white">
 <div class="navbar"><a name="#navbar_top"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <hr>
 <h1>Module machi_util</h1>
 <ul class="index"><li><a href="#description">Description</a></li><li><a href="#index">Function Index</a></li><li><a href="#functions">Function Details</a></li></ul>Miscellaneous utility functions.
 <h2><a name="description">Description</a></h2>Miscellaneous utility functions.
 <h2><a name="index">Function Index</a></h2>
 <table width="100%" border="1" cellspacing="0" cellpadding="2" summary="function index"><tr><td valign="top"><a href="#bin_to_hexstr-1">bin_to_hexstr/1</a></td><td>Convert a <code>binary()</code> to a hexadecimal string.</td></tr>
 <tr><td valign="top"><a href="#bool2int-1">bool2int/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#checksum_chunk-1">checksum_chunk/1</a></td><td>Calculate a checksum for a chunk of file data.</td></tr>
 <tr><td valign="top"><a href="#combinations-1">combinations/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#connect-2">connect/2</a></td><td>Create a TCP connection to a remote Machi server.</td></tr>
 <tr><td valign="top"><a href="#connect-3">connect/3</a></td><td>Create a TCP connection to a remote Machi server.</td></tr>
 <tr><td valign="top"><a href="#hexstr_to_bin-1">hexstr_to_bin/1</a></td><td>Convert a hexadecimal string to a <code>binary()</code>.</td></tr>
 <tr><td valign="top"><a href="#hexstr_to_int-1">hexstr_to_int/1</a></td><td>Convert a hexadecimal string to an integer.</td></tr>
 <tr><td valign="top"><a href="#increment_max_filenum-2">increment_max_filenum/2</a></td><td>Increase the file size of a config file, which is used as the
  basis for a minimum sequence number.</td></tr>
 <tr><td valign="top"><a href="#info_msg-2">info_msg/2</a></td><td>Log an 'info' level message.</td></tr>
 <tr><td valign="top"><a href="#int2bool-1">int2bool/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#int_to_hexbin-2">int_to_hexbin/2</a></td><td>Convert an integer into a hexadecimal string (in <code>binary()</code>
  form) whose length is based on <code>I_size</code>.</td></tr>
 <tr><td valign="top"><a href="#int_to_hexstr-2">int_to_hexstr/2</a></td><td>Convert an integer into a hexadecimal string whose length is
  based on <code>I_size</code>.</td></tr>
 <tr><td valign="top"><a href="#is_valid_filename-1">is_valid_filename/1</a></td><td>Given a filename, return true if it is a valid machi filename,
  false otherwise.</td></tr>
 <tr><td valign="top"><a href="#make_binary-1">make_binary/1</a></td><td>Convert a compatible Erlang data type into a <code>binary()</code> equivalent.</td></tr>
 <tr><td valign="top"><a href="#make_checksum_filename-2">make_checksum_filename/2</a></td><td>Calculate a checksum file path, by common convention.</td></tr>
 <tr><td valign="top"><a href="#make_checksum_filename-4">make_checksum_filename/4</a></td><td>Calculate a checksum file path, by common convention.</td></tr>
 <tr><td valign="top"><a href="#make_config_filename-2">make_config_filename/2</a></td><td>Calculate a config file path, by common convention.</td></tr>
 <tr><td valign="top"><a href="#make_data_filename-2">make_data_filename/2</a></td><td>Calculate a file data file path, by common convention.</td></tr>
 <tr><td valign="top"><a href="#make_data_filename-4">make_data_filename/4</a></td><td>Calculate a file data file path, by common convention.</td></tr>
 <tr><td valign="top"><a href="#make_projection_filename-2">make_projection_filename/2</a></td><td>Calculate a projection store file path, by common convention.</td></tr>
 <tr><td valign="top"><a href="#make_regname-1">make_regname/1</a></td><td>Create a registered name atom for FLU sequencer internal
  rendezvous/message passing use.</td></tr>
 <tr><td valign="top"><a href="#make_string-1">make_string/1</a></td><td>Convert a compatible Erlang data type into a <code>string()</code> equivalent.</td></tr>
 <tr><td valign="top"><a href="#make_tagged_csum-1">make_tagged_csum/1</a></td><td>Create a tagged checksum.</td></tr>
 <tr><td valign="top"><a href="#make_tagged_csum-2">make_tagged_csum/2</a></td><td>Makes tagged csum.</td></tr>
 <tr><td valign="top"><a href="#mbytes-1">mbytes/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#mk_order-2">mk_order/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#ordered_combinations-1">ordered_combinations/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#parse_filename-1">parse_filename/1</a></td><td>Given a machi filename, return a set of components in a list.</td></tr>
 <tr><td valign="top"><a href="#perms-1">perms/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#permutations-1">permutations/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#pretty_time-0">pretty_time/0</a></td><td></td></tr>
 <tr><td valign="top"><a href="#pretty_time-2">pretty_time/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#read_max_filenum-2">read_max_filenum/2</a></td><td>Read the file size of a config file, which is used as the
  basis for a minimum sequence number.</td></tr>
 <tr><td valign="top"><a href="#unmake_tagged_csum-1">unmake_tagged_csum/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#verb-1">verb/1</a></td><td>Log a verbose message.</td></tr>
 <tr><td valign="top"><a href="#verb-2">verb/2</a></td><td>Log a verbose message.</td></tr>
 <tr><td valign="top"><a href="#wait_for_death-2">wait_for_death/2</a></td><td></td></tr>
 <tr><td valign="top"><a href="#wait_for_life-2">wait_for_life/2</a></td><td></td></tr>
 </table>
 <h2><a name="functions">Function Details</a></h2>
 <h3 class="function"><a name="bin_to_hexstr-1">bin_to_hexstr/1</a></h3>
 <div class="spec">
 <p><tt>bin_to_hexstr(X1::binary()) -&gt; string()</tt><br></p>
 </div><p>Convert a <code>binary()</code> to a hexadecimal string.</p>
 <h3 class="function"><a name="bool2int-1">bool2int/1</a></h3>
 <div class="spec">
 <p><tt>bool2int(X1) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="checksum_chunk-1">checksum_chunk/1</a></h3>
 <div class="spec">
 <p><tt>checksum_chunk(Chunk::binary() | iolist()) -&gt; binary()</tt><br></p>
 </div><p>Calculate a checksum for a chunk of file data.</p>
 <h3 class="function"><a name="combinations-1">combinations/1</a></h3>
 <div class="spec">
 <p><tt>combinations(L) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="connect-2">connect/2</a></h3>
 <div class="spec">
 <p><tt>connect(Host::<a href="inet.html#type-ip_address">inet:ip_address()</a> | <a href="inet.html#type-hostname">inet:hostname()</a>, Port::<a href="inet.html#type-port_number">inet:port_number()</a>) -&gt; port()</tt><br></p>
 </div><p>Create a TCP connection to a remote Machi server.</p>
 <h3 class="function"><a name="connect-3">connect/3</a></h3>
 <div class="spec">
 <p><tt>connect(Host::<a href="inet.html#type-ip_address">inet:ip_address()</a> | <a href="inet.html#type-hostname">inet:hostname()</a>, Port::<a href="inet.html#type-port_number">inet:port_number()</a>, Timeout::timeout()) -&gt; port()</tt><br></p>
 </div><p>Create a TCP connection to a remote Machi server.</p>
 <h3 class="function"><a name="hexstr_to_bin-1">hexstr_to_bin/1</a></h3>
 <div class="spec">
 <p><tt>hexstr_to_bin(S::string() | binary()) -&gt; binary()</tt><br></p>
 </div><p>Convert a hexadecimal string to a <code>binary()</code>.</p>
 <h3 class="function"><a name="hexstr_to_int-1">hexstr_to_int/1</a></h3>
 <div class="spec">
 <p><tt>hexstr_to_int(X::string() | binary()) -&gt; non_neg_integer()</tt><br></p>
 </div><p>Convert a hexadecimal string to an integer.</p>
 <h3 class="function"><a name="increment_max_filenum-2">increment_max_filenum/2</a></h3>
 <div class="spec">
 <p><tt>increment_max_filenum(DataDir::string(), Prefix::string()) -&gt; ok | {error, term()}</tt><br></p>
 </div><p>Increase the file size of a config file, which is used as the
  basis for a minimum sequence number.</p>
 <h3 class="function"><a name="info_msg-2">info_msg/2</a></h3>
 <div class="spec">
 <p><tt>info_msg(Fmt::string(), Args::list()) -&gt; term()</tt><br></p>
 </div><p>Log an 'info' level message.</p>
 <h3 class="function"><a name="int2bool-1">int2bool/1</a></h3>
 <div class="spec">
 <p><tt>int2bool(I) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="int_to_hexbin-2">int_to_hexbin/2</a></h3>
 <div class="spec">
 <p><tt>int_to_hexbin(I::non_neg_integer(), I_size::non_neg_integer()) -&gt; binary()</tt><br></p>
 </div><p>Convert an integer into a hexadecimal string (in <code>binary()</code>
  form) whose length is based on <code>I_size</code>.</p>
 <h3 class="function"><a name="int_to_hexstr-2">int_to_hexstr/2</a></h3>
 <div class="spec">
 <p><tt>int_to_hexstr(I::non_neg_integer(), I_size::non_neg_integer()) -&gt; string()</tt><br></p>
 </div><p>Convert an integer into a hexadecimal string whose length is
  based on <code>I_size</code>.</p>
 <h3 class="function"><a name="is_valid_filename-1">is_valid_filename/1</a></h3>
 <div class="spec">
 <p><tt>is_valid_filename(Filename::string()) -&gt; true | false</tt><br></p>
 </div><p>Given a filename, return true if it is a valid machi filename,
  false otherwise.</p>
 <h3 class="function"><a name="make_binary-1">make_binary/1</a></h3>
 <div class="spec">
 <p><tt>make_binary(X::binary() | iolist()) -&gt; binary()</tt><br></p>
 </div><p>Convert a compatible Erlang data type into a <code>binary()</code> equivalent.</p>
 <h3 class="function"><a name="make_checksum_filename-2">make_checksum_filename/2</a></h3>
 <div class="spec">
 <p><tt>make_checksum_filename(DataDir::string(), FileName::[] | string() | binary()) -&gt; string()</tt><br></p>
 </div><p>Calculate a checksum file path, by common convention.</p>
 <h3 class="function"><a name="make_checksum_filename-4">make_checksum_filename/4</a></h3>
 <div class="spec">
 <p><tt>make_checksum_filename(DataDir::string(), Prefix::string(), SequencerName::atom() | string() | binary(), FileNum::integer()) -&gt; string()</tt><br></p>
 </div><p>Calculate a checksum file path, by common convention.</p>
 <h3 class="function"><a name="make_config_filename-2">make_config_filename/2</a></h3>
 <div class="spec">
 <p><tt>make_config_filename(DataDir::string(), Prefix::string()) -&gt; string()</tt><br></p>
 </div><p>Calculate a config file path, by common convention.</p>
 <h3 class="function"><a name="make_data_filename-2">make_data_filename/2</a></h3>
 <div class="spec">
 <p><tt>make_data_filename(DataDir::string(), File::[] | string() | binary()) -&gt; {binary(), string()}</tt><br></p>
 </div><p>Calculate a file data file path, by common convention.</p>
 <h3 class="function"><a name="make_data_filename-4">make_data_filename/4</a></h3>
 <div class="spec">
 <p><tt>make_data_filename(DataDir::string(), Prefix::string(), SequencerName::atom() | string() | binary(), FileNum::integer() | string()) -&gt; {binary(), string()}</tt><br></p>
 </div><p>Calculate a file data file path, by common convention.</p>
 <h3 class="function"><a name="make_projection_filename-2">make_projection_filename/2</a></h3>
 <div class="spec">
 <p><tt>make_projection_filename(DataDir::string(), File::[] | string()) -&gt; string()</tt><br></p>
 </div><p>Calculate a projection store file path, by common convention.</p>
 <h3 class="function"><a name="make_regname-1">make_regname/1</a></h3>
 <div class="spec">
 <p><tt>make_regname(Prefix::binary() | string()) -&gt; atom()</tt><br></p>
 </div><p>Create a registered name atom for FLU sequencer internal
  rendezvous/message passing use.</p>
 <h3 class="function"><a name="make_string-1">make_string/1</a></h3>
 <div class="spec">
 <p><tt>make_string(X::binary() | iolist()) -&gt; string()</tt><br></p>
 </div><p>Convert a compatible Erlang data type into a <code>string()</code> equivalent.</p>
 <h3 class="function"><a name="make_tagged_csum-1">make_tagged_csum/1</a></h3>
 <div class="spec">
 <p><tt>make_tagged_csum(X1) -&gt; any()</tt></p>
 </div><p>Create a tagged checksum</p>
 <h3 class="function"><a name="make_tagged_csum-2">make_tagged_csum/2</a></h3>
 <div class="spec">
 <p><tt>make_tagged_csum(X1::<a href="machi_dt.html#type-csum_tag">machi_dt:csum_tag()</a>, SHA::binary()) -&gt; <a href="machi_dt.html#type-chunk_csum">machi_dt:chunk_csum()</a></tt><br></p>
 </div><p>Makes tagged csum. Each meanings are:
  none / ?CSUM_TAG_NONE
   - a suspicious and nonsense checksum
  client_sha / ?CSUM_TAG_CLIENT_SHA
   - a valid checksum given by client and stored in server
  server_sha / ?CSUM_TAG_SERVER_SHA
   - a valid checksum generated by and stored in server
  server_regen_sha / ?CSUM_TAG_SERVER_REGEN_SHA
   - a valid checksum generated by server in an ad hoc manner, not stored in server</p>
 <h3 class="function"><a name="mbytes-1">mbytes/1</a></h3>
 <div class="spec">
 <p><tt>mbytes(Size) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="mk_order-2">mk_order/2</a></h3>
 <div class="spec">
 <p><tt>mk_order(UPI2, Repair1) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="ordered_combinations-1">ordered_combinations/1</a></h3>
 <div class="spec">
 <p><tt>ordered_combinations(Master) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="parse_filename-1">parse_filename/1</a></h3>
 <div class="spec">
 <p><tt>parse_filename(Filename::string()) -&gt; [string()]</tt><br></p>
 </div><p>Given a machi filename, return a set of components in a list.
  The components will be:
  <ul>
       <li>Prefix</li>
       <li>UUID</li>
       <li>Sequence number</li>
  </ul>
  Invalid filenames will return an empty list.</p>
 <h3 class="function"><a name="perms-1">perms/1</a></h3>
 <div class="spec">
 <p><tt>perms(L) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="permutations-1">permutations/1</a></h3>
 <div class="spec">
 <p><tt>permutations(L) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="pretty_time-0">pretty_time/0</a></h3>
 <div class="spec">
 <p><tt>pretty_time() -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="pretty_time-2">pretty_time/2</a></h3>
 <div class="spec">
 <p><tt>pretty_time(X1, MSec) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="read_max_filenum-2">read_max_filenum/2</a></h3>
 <div class="spec">
 <p><tt>read_max_filenum(DataDir::string(), Prefix::string()) -&gt; non_neg_integer()</tt><br></p>
 </div><p>Read the file size of a config file, which is used as the
  basis for a minimum sequence number.</p>
 <h3 class="function"><a name="unmake_tagged_csum-1">unmake_tagged_csum/1</a></h3>
 <div class="spec">
 <p><tt>unmake_tagged_csum(X1) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="verb-1">verb/1</a></h3>
 <div class="spec">
 <p><tt>verb(Fmt::string()) -&gt; ok</tt><br></p>
 </div><p>Log a verbose message.</p>
 <h3 class="function"><a name="verb-2">verb/2</a></h3>
 <div class="spec">
 <p><tt>verb(Fmt::string(), Args::list()) -&gt; ok</tt><br></p>
 </div><p>Log a verbose message.</p>
 <h3 class="function"><a name="wait_for_death-2">wait_for_death/2</a></h3>
 <div class="spec">
 <p><tt>wait_for_death(Pid, Iters) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="wait_for_life-2">wait_for_life/2</a></h3>
 <div class="spec">
 <p><tt>wait_for_life(Reg, Iters) -&gt; any()</tt></p>
 </div>
 <hr>
 <div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <p><i>Generated by EDoc, Dec 8 2015, 21:37:38.</i></p>
 </body>
 </html>
--- a/edoc/machi_yessir_client.html
+++ b/edoc/machi_yessir_client.html
@ -0,0 +1,260 @@
 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
 <html>
 <head>
 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
 <title>Module machi_yessir_client</title>
 <link rel="stylesheet" type="text/css" href="stylesheet.css" title="EDoc">
 </head>
 <body bgcolor="white">
 <div class="navbar"><a name="#navbar_top"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <hr>
 <h1>Module machi_yessir_client</h1>
 <ul class="index"><li><a href="#description">Description</a></li><li><a href="#index">Function Index</a></li><li><a href="#functions">Function Details</a></li></ul>"Yes, sir!" style dummy/mock client facade.
 <h2><a name="description">Description</a></h2>"Yes, sir!" style dummy/mock client facade.
 <h2><a name="index">Function Index</a></h2>
 <table width="100%" border="1" cellspacing="0" cellpadding="2" summary="function index"><tr><td valign="top"><a href="#append_chunk-4">append_chunk/4</a></td><td>Append a chunk (binary- or iolist-style) of data to a file
  with <code>Prefix</code>.</td></tr>
 <tr><td valign="top"><a href="#append_chunk-5">append_chunk/5</a></td><td>Append a chunk (binary- or iolist-style) of data to a file
  with <code>Prefix</code>.</td></tr>
 <tr><td valign="top"><a href="#append_chunk_extra-5">append_chunk_extra/5</a></td><td>Append a chunk (binary- or iolist-style) of data to a file
  with <code>Prefix</code> and also request an additional <code>Extra</code> bytes.</td></tr>
 <tr><td valign="top"><a href="#append_chunk_extra-6">append_chunk_extra/6</a></td><td>Append a chunk (binary- or iolist-style) of data to a file
  with <code>Prefix</code> and also request an additional <code>Extra</code> bytes.</td></tr>
 <tr><td valign="top"><a href="#checksum_list-3">checksum_list/3</a></td><td>Fetch the list of chunk checksums for <code>File</code>.</td></tr>
 <tr><td valign="top"><a href="#checksum_list-4">checksum_list/4</a></td><td>Fetch the list of chunk checksums for <code>File</code>.</td></tr>
 <tr><td valign="top"><a href="#connect-1">connect/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#connected_p-1">connected_p/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#delete_migration-3">delete_migration/3</a></td><td>Restricted API: Delete a file after it has been successfully
  migrated.</td></tr>
 <tr><td valign="top"><a href="#delete_migration-4">delete_migration/4</a></td><td>Restricted API: Delete a file after it has been successfully
  migrated.</td></tr>
 <tr><td valign="top"><a href="#disconnect-1">disconnect/1</a></td><td></td></tr>
 <tr><td valign="top"><a href="#get_all_projections-2">get_all_projections/2</a></td><td>Get all projections from the FLU's projection store.</td></tr>
 <tr><td valign="top"><a href="#get_all_projections-3">get_all_projections/3</a></td><td>Get all projections from the FLU's projection store.</td></tr>
 <tr><td valign="top"><a href="#get_latest_epoch-2">get_latest_epoch/2</a></td><td>Get the latest epoch number + checksum from the FLU's projection store.</td></tr>
 <tr><td valign="top"><a href="#get_latest_epoch-3">get_latest_epoch/3</a></td><td>Get the latest epoch number + checksum from the FLU's projection store.</td></tr>
 <tr><td valign="top"><a href="#list_all_projections-2">list_all_projections/2</a></td><td>Get all epoch numbers from the FLU's projection store.</td></tr>
 <tr><td valign="top"><a href="#list_all_projections-3">list_all_projections/3</a></td><td>Get all epoch numbers from the FLU's projection store.</td></tr>
 <tr><td valign="top"><a href="#list_files-2">list_files/2</a></td><td>Fetch the list of all files on the remote FLU.</td></tr>
 <tr><td valign="top"><a href="#list_files-3">list_files/3</a></td><td>Fetch the list of all files on the remote FLU.</td></tr>
 <tr><td valign="top"><a href="#quit-1">quit/1</a></td><td>Quit &amp; close the connection to remote FLU.</td></tr>
 <tr><td valign="top"><a href="#read_chunk-5">read_chunk/5</a></td><td>Read a chunk of data of size <code>Size</code> from <code>File</code> at <code>Offset</code>.</td></tr>
 <tr><td valign="top"><a href="#read_chunk-6">read_chunk/6</a></td><td>Read a chunk of data of size <code>Size</code> from <code>File</code> at <code>Offset</code>.</td></tr>
 <tr><td valign="top"><a href="#read_latest_projection-2">read_latest_projection/2</a></td><td>Get the latest projection from the FLU's projection store for <code>ProjType</code></td></tr>
 <tr><td valign="top"><a href="#read_latest_projection-3">read_latest_projection/3</a></td><td>Get the latest projection from the FLU's projection store for <code>ProjType</code></td></tr>
 <tr><td valign="top"><a href="#read_projection-3">read_projection/3</a></td><td>Read a projection <code>Proj</code> of type <code>ProjType</code>.</td></tr>
 <tr><td valign="top"><a href="#read_projection-4">read_projection/4</a></td><td>Read a projection <code>Proj</code> of type <code>ProjType</code>.</td></tr>
 <tr><td valign="top"><a href="#trunc_hack-3">trunc_hack/3</a></td><td>Restricted API: Truncate a file after it has been successfully
  erasure coded.</td></tr>
 <tr><td valign="top"><a href="#trunc_hack-4">trunc_hack/4</a></td><td>Restricted API: Truncate a file after it has been successfully
  erasure coded.</td></tr>
 <tr><td valign="top"><a href="#wedge_status-1">wedge_status/1</a></td><td>Fetch the wedge status from the remote FLU.</td></tr>
 <tr><td valign="top"><a href="#wedge_status-2">wedge_status/2</a></td><td>Fetch the wedge status from the remote FLU.</td></tr>
 <tr><td valign="top"><a href="#write_chunk-5">write_chunk/5</a></td><td>Restricted API: Write a chunk of already-sequenced data to
  <code>File</code> at <code>Offset</code>.</td></tr>
 <tr><td valign="top"><a href="#write_chunk-6">write_chunk/6</a></td><td>Restricted API: Write a chunk of already-sequenced data to
  <code>File</code> at <code>Offset</code>.</td></tr>
 <tr><td valign="top"><a href="#write_projection-3">write_projection/3</a></td><td>Write a projection <code>Proj</code> of type <code>ProjType</code>.</td></tr>
 <tr><td valign="top"><a href="#write_projection-4">write_projection/4</a></td><td>Write a projection <code>Proj</code> of type <code>ProjType</code>.</td></tr>
 </table>
 <h2><a name="functions">Function Details</a></h2>
 <h3 class="function"><a name="append_chunk-4">append_chunk/4</a></h3>
 <div class="spec">
 <p><tt>append_chunk(Sock, EpochID, Prefix, Chunk) -&gt; any()</tt></p>
 </div><p>Append a chunk (binary- or iolist-style) of data to a file
  with <code>Prefix</code>.</p>
 <h3 class="function"><a name="append_chunk-5">append_chunk/5</a></h3>
 <div class="spec">
 <p><tt>append_chunk(Host, TcpPort, EpochID, Prefix, Chunk) -&gt; any()</tt></p>
 </div><p>Append a chunk (binary- or iolist-style) of data to a file
  with <code>Prefix</code>.</p>
 <h3 class="function"><a name="append_chunk_extra-5">append_chunk_extra/5</a></h3>
 <div class="spec">
 <p><tt>append_chunk_extra(Yessir, EpochID, Prefix, Chunk, ChunkExtra) -&gt; any()</tt></p>
 </div><p><p>Append a chunk (binary- or iolist-style) of data to a file
  with <code>Prefix</code> and also request an additional <code>Extra</code> bytes.</p>
  For example, if the <code>Chunk</code> size is 1 KByte and <code>Extra</code> is 4K Bytes, then
  the file offsets that follow <code>Chunk</code>'s position for the following 4K will
  be reserved by the file sequencer for later write(s) by the
  <code>write_chunk()</code> API.</p>
 <h3 class="function"><a name="append_chunk_extra-6">append_chunk_extra/6</a></h3>
 <div class="spec">
 <p><tt>append_chunk_extra(Host, TcpPort, EpochID, Prefix, Chunk, ChunkExtra) -&gt; any()</tt></p>
 </div><p><p>Append a chunk (binary- or iolist-style) of data to a file
  with <code>Prefix</code> and also request an additional <code>Extra</code> bytes.</p>
  For example, if the <code>Chunk</code> size is 1 KByte and <code>Extra</code> is 4K Bytes, then
  the file offsets that follow <code>Chunk</code>'s position for the following 4K will
  be reserved by the file sequencer for later write(s) by the
  <code>write_chunk()</code> API.</p>
 <h3 class="function"><a name="checksum_list-3">checksum_list/3</a></h3>
 <div class="spec">
 <p><tt>checksum_list(Yessir, EpochID, File) -&gt; any()</tt></p>
 </div><p>Fetch the list of chunk checksums for <code>File</code>.</p>
 <h3 class="function"><a name="checksum_list-4">checksum_list/4</a></h3>
 <div class="spec">
 <p><tt>checksum_list(Host, TcpPort, EpochID, File) -&gt; any()</tt></p>
 </div><p>Fetch the list of chunk checksums for <code>File</code>.</p>
 <h3 class="function"><a name="connect-1">connect/1</a></h3>
 <div class="spec">
 <p><tt>connect(P_srvr) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="connected_p-1">connected_p/1</a></h3>
 <div class="spec">
 <p><tt>connected_p(X1) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="delete_migration-3">delete_migration/3</a></h3>
 <div class="spec">
 <p><tt>delete_migration(Yessir, EpochID, File) -&gt; any()</tt></p>
 </div><p>Restricted API: Delete a file after it has been successfully
  migrated.</p>
 <h3 class="function"><a name="delete_migration-4">delete_migration/4</a></h3>
 <div class="spec">
 <p><tt>delete_migration(Host, TcpPort, EpochID, File) -&gt; any()</tt></p>
 </div><p>Restricted API: Delete a file after it has been successfully
  migrated.</p>
 <h3 class="function"><a name="disconnect-1">disconnect/1</a></h3>
 <div class="spec">
 <p><tt>disconnect(Yessir) -&gt; any()</tt></p>
 </div>
 <h3 class="function"><a name="get_all_projections-2">get_all_projections/2</a></h3>
 <div class="spec">
 <p><tt>get_all_projections(Yessir, ProjType) -&gt; any()</tt></p>
 </div><p>Get all projections from the FLU's projection store.</p>
 <h3 class="function"><a name="get_all_projections-3">get_all_projections/3</a></h3>
 <div class="spec">
 <p><tt>get_all_projections(Host, TcpPort, ProjType) -&gt; any()</tt></p>
 </div><p>Get all projections from the FLU's projection store.</p>
 <h3 class="function"><a name="get_latest_epoch-2">get_latest_epoch/2</a></h3>
 <div class="spec">
 <p><tt>get_latest_epoch(Sock, ProjType) -&gt; any()</tt></p>
 </div><p>Get the latest epoch number + checksum from the FLU's projection store.</p>
 <h3 class="function"><a name="get_latest_epoch-3">get_latest_epoch/3</a></h3>
 <div class="spec">
 <p><tt>get_latest_epoch(Host, TcpPort, ProjType) -&gt; any()</tt></p>
 </div><p>Get the latest epoch number + checksum from the FLU's projection store.</p>
 <h3 class="function"><a name="list_all_projections-2">list_all_projections/2</a></h3>
 <div class="spec">
 <p><tt>list_all_projections(Sock, ProjType) -&gt; any()</tt></p>
 </div><p>Get all epoch numbers from the FLU's projection store.</p>
 <h3 class="function"><a name="list_all_projections-3">list_all_projections/3</a></h3>
 <div class="spec">
 <p><tt>list_all_projections(Host, TcpPort, ProjType) -&gt; any()</tt></p>
 </div><p>Get all epoch numbers from the FLU's projection store.</p>
 <h3 class="function"><a name="list_files-2">list_files/2</a></h3>
 <div class="spec">
 <p><tt>list_files(Yessir, EpochID) -&gt; any()</tt></p>
 </div><p>Fetch the list of all files on the remote FLU.</p>
 <h3 class="function"><a name="list_files-3">list_files/3</a></h3>
 <div class="spec">
 <p><tt>list_files(Host, TcpPort, EpochID) -&gt; any()</tt></p>
 </div><p>Fetch the list of all files on the remote FLU.</p>
 <h3 class="function"><a name="quit-1">quit/1</a></h3>
 <div class="spec">
 <p><tt>quit(X1) -&gt; any()</tt></p>
 </div><p>Quit &amp; close the connection to remote FLU.</p>
 <h3 class="function"><a name="read_chunk-5">read_chunk/5</a></h3>
 <div class="spec">
 <p><tt>read_chunk(Yessir, EpochID, File, Offset, Size) -&gt; any()</tt></p>
 </div><p>Read a chunk of data of size <code>Size</code> from <code>File</code> at <code>Offset</code>.</p>
 <h3 class="function"><a name="read_chunk-6">read_chunk/6</a></h3>
 <div class="spec">
 <p><tt>read_chunk(Host, TcpPort, EpochID, File, Offset, Size) -&gt; any()</tt></p>
 </div><p>Read a chunk of data of size <code>Size</code> from <code>File</code> at <code>Offset</code>.</p>
 <h3 class="function"><a name="read_latest_projection-2">read_latest_projection/2</a></h3>
 <div class="spec">
 <p><tt>read_latest_projection(Yessir, ProjType) -&gt; any()</tt></p>
 </div><p>Get the latest projection from the FLU's projection store for <code>ProjType</code></p>
 <h3 class="function"><a name="read_latest_projection-3">read_latest_projection/3</a></h3>
 <div class="spec">
 <p><tt>read_latest_projection(Host, TcpPort, ProjType) -&gt; any()</tt></p>
 </div><p>Get the latest projection from the FLU's projection store for <code>ProjType</code></p>
 <h3 class="function"><a name="read_projection-3">read_projection/3</a></h3>
 <div class="spec">
 <p><tt>read_projection(Yessir, ProjType, Epoch) -&gt; any()</tt></p>
 </div><p>Read a projection <code>Proj</code> of type <code>ProjType</code>.</p>
 <h3 class="function"><a name="read_projection-4">read_projection/4</a></h3>
 <div class="spec">
 <p><tt>read_projection(Host, TcpPort, ProjType, Epoch) -&gt; any()</tt></p>
 </div><p>Read a projection <code>Proj</code> of type <code>ProjType</code>.</p>
 <h3 class="function"><a name="trunc_hack-3">trunc_hack/3</a></h3>
 <div class="spec">
 <p><tt>trunc_hack(Yessir, EpochID, File) -&gt; any()</tt></p>
 </div><p>Restricted API: Truncate a file after it has been successfully
  erasure coded.</p>
 <h3 class="function"><a name="trunc_hack-4">trunc_hack/4</a></h3>
 <div class="spec">
 <p><tt>trunc_hack(Host, TcpPort, EpochID, File) -&gt; any()</tt></p>
 </div><p>Restricted API: Truncate a file after it has been successfully
  erasure coded.</p>
 <h3 class="function"><a name="wedge_status-1">wedge_status/1</a></h3>
 <div class="spec">
 <p><tt>wedge_status(Sock) -&gt; any()</tt></p>
 </div><p>Fetch the wedge status from the remote FLU.</p>
 <h3 class="function"><a name="wedge_status-2">wedge_status/2</a></h3>
 <div class="spec">
 <p><tt>wedge_status(Host, TcpPort) -&gt; any()</tt></p>
 </div><p>Fetch the wedge status from the remote FLU.</p>
 <h3 class="function"><a name="write_chunk-5">write_chunk/5</a></h3>
 <div class="spec">
 <p><tt>write_chunk(Yessir, EpochID, File, Offset, Chunk) -&gt; any()</tt></p>
 </div><p>Restricted API: Write a chunk of already-sequenced data to
  <code>File</code> at <code>Offset</code>.</p>
 <h3 class="function"><a name="write_chunk-6">write_chunk/6</a></h3>
 <div class="spec">
 <p><tt>write_chunk(Host, TcpPort, EpochID, File, Offset, Chunk) -&gt; any()</tt></p>
 </div><p>Restricted API: Write a chunk of already-sequenced data to
  <code>File</code> at <code>Offset</code>.</p>
 <h3 class="function"><a name="write_projection-3">write_projection/3</a></h3>
 <div class="spec">
 <p><tt>write_projection(Yessir, ProjType, Proj) -&gt; any()</tt></p>
 </div><p>Write a projection <code>Proj</code> of type <code>ProjType</code>.</p>
 <h3 class="function"><a name="write_projection-4">write_projection/4</a></h3>
 <div class="spec">
 <p><tt>write_projection(Host, TcpPort, ProjType, Proj) -&gt; any()</tt></p>
 </div><p>Write a projection <code>Proj</code> of type <code>ProjType</code>.</p>
 <hr>
 <div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <p><i>Generated by EDoc, Dec 8 2015, 21:37:38.</i></p>
 </body>
 </html>
--- a/edoc/modules-frame.html
+++ b/edoc/modules-frame.html
@ -0,0 +1,43 @@
 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
 <html>
 <head>
 <title>The machi application</title>
 <link rel="stylesheet" type="text/css" href="stylesheet.css" title="EDoc">
 </head>
 <body bgcolor="white">
 <h2 class="indextitle">Modules</h2>
 <table width="100%" border="0" summary="list of modules">
 <tr><td><a href="machi_admin_util.html" target="overviewFrame" class="module">machi_admin_util</a></td></tr>
 <tr><td><a href="machi_app.html" target="overviewFrame" class="module">machi_app</a></td></tr>
 <tr><td><a href="machi_basho_bench_driver.html" target="overviewFrame" class="module">machi_basho_bench_driver</a></td></tr>
 <tr><td><a href="machi_chain_manager1.html" target="overviewFrame" class="module">machi_chain_manager1</a></td></tr>
 <tr><td><a href="machi_chain_repair.html" target="overviewFrame" class="module">machi_chain_repair</a></td></tr>
 <tr><td><a href="machi_chash.html" target="overviewFrame" class="module">machi_chash</a></td></tr>
 <tr><td><a href="machi_cinfo.html" target="overviewFrame" class="module">machi_cinfo</a></td></tr>
 <tr><td><a href="machi_config.html" target="overviewFrame" class="module">machi_config</a></td></tr>
 <tr><td><a href="machi_cr_client.html" target="overviewFrame" class="module">machi_cr_client</a></td></tr>
 <tr><td><a href="machi_csum_table.html" target="overviewFrame" class="module">machi_csum_table</a></td></tr>
 <tr><td><a href="machi_dt.html" target="overviewFrame" class="module">machi_dt</a></td></tr>
 <tr><td><a href="machi_file_proxy.html" target="overviewFrame" class="module">machi_file_proxy</a></td></tr>
 <tr><td><a href="machi_file_proxy_sup.html" target="overviewFrame" class="module">machi_file_proxy_sup</a></td></tr>
 <tr><td><a href="machi_fitness.html" target="overviewFrame" class="module">machi_fitness</a></td></tr>
 <tr><td><a href="machi_flu1.html" target="overviewFrame" class="module">machi_flu1</a></td></tr>
 <tr><td><a href="machi_flu1_client.html" target="overviewFrame" class="module">machi_flu1_client</a></td></tr>
 <tr><td><a href="machi_flu_filename_mgr.html" target="overviewFrame" class="module">machi_flu_filename_mgr</a></td></tr>
 <tr><td><a href="machi_flu_metadata_mgr.html" target="overviewFrame" class="module">machi_flu_metadata_mgr</a></td></tr>
 <tr><td><a href="machi_flu_metadata_mgr_sup.html" target="overviewFrame" class="module">machi_flu_metadata_mgr_sup</a></td></tr>
 <tr><td><a href="machi_flu_psup.html" target="overviewFrame" class="module">machi_flu_psup</a></td></tr>
 <tr><td><a href="machi_flu_sup.html" target="overviewFrame" class="module">machi_flu_sup</a></td></tr>
 <tr><td><a href="machi_lifecycle_mgr.html" target="overviewFrame" class="module">machi_lifecycle_mgr</a></td></tr>
 <tr><td><a href="machi_merkle_tree.html" target="overviewFrame" class="module">machi_merkle_tree</a></td></tr>
 <tr><td><a href="machi_pb_high_client.html" target="overviewFrame" class="module">machi_pb_high_client</a></td></tr>
 <tr><td><a href="machi_pb_translate.html" target="overviewFrame" class="module">machi_pb_translate</a></td></tr>
 <tr><td><a href="machi_plist.html" target="overviewFrame" class="module">machi_plist</a></td></tr>
 <tr><td><a href="machi_projection.html" target="overviewFrame" class="module">machi_projection</a></td></tr>
 <tr><td><a href="machi_projection_store.html" target="overviewFrame" class="module">machi_projection_store</a></td></tr>
 <tr><td><a href="machi_proxy_flu1_client.html" target="overviewFrame" class="module">machi_proxy_flu1_client</a></td></tr>
 <tr><td><a href="machi_sup.html" target="overviewFrame" class="module">machi_sup</a></td></tr>
 <tr><td><a href="machi_util.html" target="overviewFrame" class="module">machi_util</a></td></tr>
 <tr><td><a href="machi_yessir_client.html" target="overviewFrame" class="module">machi_yessir_client</a></td></tr></table>
 </body>
 </html>
--- a/edoc/overview-summary.html
+++ b/edoc/overview-summary.html
@ -0,0 +1,185 @@
 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
 <html>
 <head>
 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
 <title>Machi: a small village of replicated files
 </title>
 <link rel="stylesheet" type="text/css" href="stylesheet.css" title="EDoc">
 </head>
 <body bgcolor="white">
 <div class="navbar"><a name="#navbar_top"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <h1>Machi: a small village of replicated files
 </h1>
 <h3><a name="About_This_EDoc_Documentation">About This EDoc Documentation</a></h3>
 <p>This EDoc-style documentation will concern itself only with Erlang
 function APIs and function &amp; data types.  Higher-level design and
 commentary will remain outside of the Erlang EDoc system; please see
 the "Pointers to Other Machi Documentation" section below for more
 details.</p>
 <p>Readers should beware that this documentation may be out-of-sync with
 the source code.  When in doubt, use the <code>make edoc</code> command to
 regenerate all HTML pages.</p>
 <p>It is the developer's responsibility to re-generate the documentation
 periodically and commit it to the Git repo.</p>
 <h3><a name="Machi_Code_Overview">Machi Code Overview</a></h3>
 <h4><a name="Chain_Manager">Chain Manager</a></h4>
 <p>The Chain Manager is responsible for managing the state of Machi's
 "Chain Replication" state.  This role is roughly analogous to the
 "Riak Core" application inside of Riak, which takes care of
 coordinating replica placement and replica repair.</p>
 <p>For each primitive data server in the cluster, a Machi FLU, there is a
 Chain Manager process that manages its FLU's role within the Machi
 cluster's Chain Replication scheme.  Each Chain Manager process
 executes locally and independently to manage the distributed state of
 a single Machi Chain Replication chain.</p>
  <ul>
  <li><p> To contrast with Riak Core ... Riak Core's claimant process is
       solely responsible for managing certain critical aspects of
       Riak Core distributed state.  Machi's Chain Manager process
       performs similar tasks as Riak Core's claimant.  However, Machi
       has several active Chain Manager processes, one per FLU server,
       instead of a single active process like Core's claimant.  Each
       Chain Manager process acts independently; each is constrained
       so that it will reach consensus via independent computation
       &amp; action.</p>
       Full discussion of this distributed consensus is outside the
       scope of this document; see the "Pointers to Other Machi
       Documentation" section below for more information.
  </li>
  <li> Machi differs from a Riak Core application because Machi's
       replica placement policy is simply, "All Machi servers store
       replicas of all Machi files".
       Machi is intended to be a primitive building block for creating larger
       cluster-of-clusters where files are
       distributed/fragmented/sharded across a large pool of
       independent Machi clusters.
  </li>
  <li> See
       <a href="https://www.usenix.org/legacy/events/osdi04/tech/renesse.html" target="_top"><tt>https://www.usenix.org/legacy/events/osdi04/tech/renesse.html</tt></a>
       for a copy of the paper, "Chain Replication for Supporting High
       Throughput and Availability" by Robbert van Renesse and Fred
       B. Schneider.
  </li>
  </ul>
 <h4><a name="FLU">FLU</a></h4>
 <p>The FLU is the basic storage server for Machi.</p>
  <ul>
  <li> The name FLU is taken from "flash storage unit" from the paper
       "CORFU: A Shared Log Design for Flash Clusters" by
       Balakrishnan, Malkhi, Prabhakaran, and Wobber.  See
       <a href="https://www.usenix.org/conference/nsdi12/technical-sessions/presentation/balakrishnan" target="_top"><tt>https://www.usenix.org/conference/nsdi12/technical-sessions/presentation/balakrishnan</tt></a>
  </li>
  <li> In CORFU, the sequencer step is a prerequisite step that is
       performed by a separate component, the Sequencer.
       In Machi, the <code>append_chunk()</code> protocol message has
       an implicit "sequencer" operation applied by the "head" of the
       Machi Chain Replication chain.  If a client wishes to write
       data that has already been assigned a sequencer position, then
       the <code>write_chunk()</code> API function is used.
  </li>
  </ul>
 <p>For each FLU, there are three independent tasks that are implemented
 using three different Erlang processes:</p>
  <ul>
  <li> A FLU server, implemented primarily by <code>machi_flu.erl</code>.
  </li>
  <li> A projection store server, implemented primarily by
      <code>machi_projection_store.erl</code>. 
  </li>
  <li> A chain state manager server, implemented primarily by
       <code>machi_chain_manager1.erl</code>.
  </li>
  </ul>
 <p>From the perspective of failure detection, it is very convenient that
 all three FLU-related services (file server, sequencer server, and
 projection server) are accessed using the same single TCP port.</p>
 <h4><a name="Projection_(data_structure)">Projection (data structure)</a></h4>
 <p>The projection is a data structure that specifies the current state
 of the Machi cluster: all FLUs, which FLUS are considered
 up/running or down/crashed/stopped, which FLUs are actively
 participants in the Chain Replication protocol, and which FLUs are
 under "repair" (i.e., having their data resyncronized when
 newly-added to a cluster or when restarting after a crash).</p>
 <h4><a name="Projection_Store_(server)">Projection Store (server)</a></h4>
 <p>The projection store is a storage service that is implemented by an
 Erlang/OTP <code>gen_server</code> process that is associated with each
 FLU.  Conceptually, the projection store is an array of
 write-once registers.  For each projection store register, the
 key is a 2-tuple of an epoch number (<code>non_neg_integer()</code> type)
 and a projection type (<code>public</code> or <code>private</code> type); the value is
 a projection data structure (<code>projection_v1()</code> type).</p>
 <h4><a name="Client_and_Proxy_Client">Client and Proxy Client</a></h4>
 <p>Machi is intentionally avoiding using distributed Erlang for Machi's
 communication.  This design decision makes Erlang-side code more
 difficult &amp; complex but allows us the freedom of implementing
 parts of Machi in other languages without major
 protocol&amp;API&amp;glue code changes later in the product's
 lifetime.</p>
 <p>There are two layers of interface for Machi clients.</p>
  <ul>
  <li> The <code>machi_flu1_client</code> module implements an API that uses a
       TCP socket directly.
  </li>
  <li> The <code>machi_proxy_flu1_client</code> module implements an API that
       uses a local, long-lived <code>gen_server</code> process as a proxy for
       the remote, perhaps disconnected-or-crashed Machi FLU server.
  </li>
  </ul>
 <p>The types for both modules ought to be the same.  However, due to
 rapid code churn, some differences might exist.  Any major difference
 is (almost by definition) a bug: please open a GitHub issue to request
 a correction.</p>
 <h3><a name="TODO_notes">TODO notes</a></h3>
 <p>Any use of the string "TODO" in upper/lower/mixed case, anywhere in
 the code, is a reminder signal of unfinished work.</p>
 <h3><a name="Pointers_to_Other_Machi_Documentation">Pointers to Other Machi Documentation</a></h3>
 <ul>
 <li> If you are viewing this document locally, please look in the
     <code>../doc/</code> directory,
 </li>
 <li> If you are viewing this document via the Web, please find the
     documentation via this link: 
     <a href="http://github.com/basho/machi/tree/master/doc/" target="_top"><tt>http://github.com/basho/machi/tree/master/doc/</tt></a>
     Please be aware that this link points to the <code>master</code> branch
     of the Machi source repository and therefore may be
     out-of-sync with non-<code>master</code> branch code.
 </li>
 </ul>
 <hr>
 <div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
 <p><i>Generated by EDoc, Dec 8 2015, 21:37:38.</i></p>
 </body>
 </html>
--- a/edoc/overview.edoc
+++ b/edoc/overview.edoc
@ -0,0 +1,14 @@
@title Machi: a small village of replicated files
@doc 
 Documentation for Machi is an ongoing challenge.  Much of the
 high-level design & commentary are outside of the Erlang EDoc system
 Zoom2 zoom zoom zoom boom boom boom boom
 Rumba tango Rumba tango Rumba tango Rumba tango Rumba tango Rumba
 tango Rumba tango Rumba tango Rumba tango Rumba tango Rumba tango
 Rumba tango Rumba tango Rumba tango Rumba tango Rumba tango Rumba
 tango Rumba tango Rumba tango Rumba tango Rumba tango
--- a/edoc/packages-frame.html
+++ b/edoc/packages-frame.html
@ -0,0 +1,11 @@
 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
 <html>
 <head>
 <title>The machi application</title>
 <link rel="stylesheet" type="text/css" href="stylesheet.css" title="EDoc">
 </head>
 <body bgcolor="white">
 <h2 class="indextitle">Packages</h2>
 <table width="100%" border="0" summary="list of packages"></table>
 </body>
 </html>
--- a/edoc/stylesheet.css
+++ b/edoc/stylesheet.css
@ -0,0 +1,55 @@
 /* standard EDoc style sheet */
 body {
 	font-family: Verdana, Arial, Helvetica, sans-serif;
      	margin-left: .25in;
       	margin-right: .2in;
       	margin-top: 0.2in;
       	margin-bottom: 0.2in;
       	color: #000000;
       	background-color: #ffffff;
 }
 h1,h2 {
 	margin-left: -0.2in;
 }
 div.navbar {
 	background-color: #add8e6;
 	padding: 0.2em;
 }
 h2.indextitle {
 	padding: 0.4em;
 	background-color: #add8e6;
 }
 h3.function,h3.typedecl {
 	background-color: #add8e6;
 	padding-left: 1em;
 }
 div.spec {
 	margin-left: 2em;
 	background-color: #eeeeee;
 }
 a.module,a.package {
 	text-decoration:none
 }
 a.module:hover,a.package:hover {
 	background-color: #eeeeee;
 }
 ul.definitions {
 	list-style-type: none;
 }
 ul.index {
 	list-style-type: none;
 	background-color: #eeeeee;
 }
 /*
 * Minor style tweaks
 */
 ul {
 	list-style-type: square;
 }
 table {
 	border-collapse: collapse;
 }
 td {
 	padding: 3
 }
--- a/images/basho_bench.example0.png
+++ b/images/basho_bench.example0.png
--- a/images/supervisor-2flus.png
+++ b/images/supervisor-2flus.png
--- a/include/machi.hrl
+++ b/include/machi.hrl
@ -1,63 +0,0 @@
 %% -------------------------------------------------------------------
 %%
 %% Copyright (c) 2007-2015 Basho Technologies, Inc.  All Rights Reserved.
 %%
 %% This file is provided to you under the Apache License,
 %% Version 2.0 (the "License"); you may not use this file
 %% except in compliance with the License.  You may obtain
 %% a copy of the License at
 %%
 %%   http://www.apache.org/licenses/LICENSE-2.0
 %%
 %% Unless required by applicable law or agreed to in writing,
 %% software distributed under the License is distributed on an
 %% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
 %% KIND, either express or implied.  See the License for the
 %% specific language governing permissions and limitations
 %% under the License.
 %%
 %% -------------------------------------------------------------------
 %% @doc Now 4GiBytes, could be up to 64bit due to PB message limit of
 %% chunk size
 -define(DEFAULT_MAX_FILE_SIZE, ((1 bsl 32) - 1)).
 -define(MINIMUM_OFFSET, 1024).
 %% 0th draft of checksum typing with 1st byte.
 -define(CSUM_TAG_NONE, 0).                   % No csum provided by client
 -define(CSUM_TAG_CLIENT_SHA, 1).             % Client-generated SHA1
 -define(CSUM_TAG_SERVER_SHA, 2).             % Server-genereated SHA1
 -define(CSUM_TAG_SERVER_REGEN_SHA, 3).       % Server-regenerated SHA1
 -define(CSUM_TAG_NONE_ATOM, none).
 -define(CSUM_TAG_CLIENT_SHA_ATOM, client_sha).
 -define(CSUM_TAG_SERVER_SHA_ATOM, server_sha).
 -define(CSUM_TAG_SERVER_REGEN_SHA_ATOM, server_regen_sha).
 %% Protocol Buffers goop
 -define(PB_MAX_MSG_SIZE, (33*1024*1024)).
 -define(PB_PACKET_OPTS, [{packet, 4}, {packet_size, ?PB_MAX_MSG_SIZE}]).
 %% TODO: it's used in flu_sup and elsewhere, change this to suitable name
 -define(TEST_ETS_TABLE, test_ets_table).
 -define(DEFAULT_COC_NAMESPACE, "").
 -define(DEFAULT_COC_LOCATOR,    0).
 -record(ns_info, {
          version = 0  :: machi_dt:namespace_version(),
          name = <<>>  :: machi_dt:namespace(),
          locator = 0  :: machi_dt:locator()
         }).
 -record(append_opts, {
          chunk_extra = 0             :: machi_dt:chunk_size(),
          preferred_file_name         :: 'undefined' | machi_dt:file_name_s(),
          flag_fail_preferred = false :: boolean()
         }).
 -record(read_opts, {
          no_checksum = false         :: boolean(),
          no_chunk = false            :: boolean(),
          needs_trimmed = false       :: boolean()
         }).
--- a/include/machi_chain_manager.hrl
+++ b/include/machi_chain_manager.hrl
@ -1,26 +0,0 @@
 %% -------------------------------------------------------------------
 %%
 %% Copyright (c) 2007-2015 Basho Technologies, Inc.  All Rights Reserved.
 %%
 %% This file is provided to you under the Apache License,
 %% Version 2.0 (the "License"); you may not use this file
 %% except in compliance with the License.  You may obtain
 %% a copy of the License at
 %%
 %%   http://www.apache.org/licenses/LICENSE-2.0
 %%
 %% Unless required by applicable law or agreed to in writing,
 %% software distributed under the License is distributed on an
 %% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
 %% KIND, either express or implied.  See the License for the
 %% specific language governing permissions and limitations
 %% under the License.
 %%
 %% -------------------------------------------------------------------
 -include("machi_projection.hrl").
 -define(NOT_FLAPPING, {0,0,0}).
 -type projection() :: #projection_v1{}.
--- a/include/machi_merkle_tree.hrl
+++ b/include/machi_merkle_tree.hrl
@ -1,20 +0,0 @@
 %% machi merkle tree records
 -record(naive, {
          chunk_size = 1048576   :: pos_integer(), %% default 1 MB
          recalc = true          :: boolean(),
          root                   :: 'undefined' | binary(),
          lvl1 = []              :: [ binary() ],
          lvl2 = []              :: [ binary() ],
          lvl3 = []              :: [ binary() ],
          leaves = []            :: [ { Offset :: pos_integer(),
                                        Size :: pos_integer(),
                                        Csum :: binary()} ]
         }).
 -record(mt, {
          filename               :: string(),
          tree                   :: #naive{},
          backend = 'naive'      :: 'naive'
         }).
--- a/include/machi_projection.hrl
+++ b/include/machi_projection.hrl
@ -1,92 +0,0 @@
 %% -------------------------------------------------------------------
 %%
 %% Copyright (c) 2007-2015 Basho Technologies, Inc.  All Rights Reserved.
 %%
 %% This file is provided to you under the Apache License,
 %% Version 2.0 (the "License"); you may not use this file
 %% except in compliance with the License.  You may obtain
 %% a copy of the License at
 %%
 %%   http://www.apache.org/licenses/LICENSE-2.0
 %%
 %% Unless required by applicable law or agreed to in writing,
 %% software distributed under the License is distributed on an
 %% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
 %% KIND, either express or implied.  See the License for the
 %% specific language governing permissions and limitations
 %% under the License.
 %%
 %% -------------------------------------------------------------------
 -ifndef(MACHI_PROJECTION_HRL).
 -define(MACHI_PROJECTION_HRL, true).
 -type pv1_consistency_mode() :: 'ap_mode' | 'cp_mode'.
 -type pv1_chain_name():: atom().
 -type pv1_csum()      :: binary().
 -type pv1_epoch()     :: {pv1_epoch_n(), pv1_csum()}.
 -type pv1_epoch_n()   :: non_neg_integer().
 -type pv1_server()    :: atom().
 -type pv1_timestamp() :: {non_neg_integer(), non_neg_integer(), non_neg_integer()}.
 -record(p_srvr, {
          name            :: pv1_server(),
          proto_mod = 'machi_flu1_client' :: atom(), % Module name
          address         :: term(), % Protocol-specific
          port            :: term(), % Protocol-specific
          props = []      :: list()  % proplist for other related info
         }).
 -record(flap_i, {
          flap_count :: {term(), term()},
          all_hosed :: list(),
          all_flap_counts :: list(),
          my_unique_prop_count :: non_neg_integer()
         }).
 -type p_srvr() :: #p_srvr{}.
 -type p_srvr_dict() :: orddict:orddict().
 -define(DUMMY_PV1_EPOCH, {0,<<0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0>>}).
 %% Kludge for spam gossip.  TODO: replace me
 -define(SPAM_PROJ_EPOCH, ((1 bsl 32) - 7)).
 -record(projection_v1, {
          epoch_number    :: pv1_epoch_n() | ?SPAM_PROJ_EPOCH,
          epoch_csum      :: pv1_csum(),
          author_server   :: pv1_server(),
          chain_name = ch_not_def_yet :: pv1_chain_name(),
          all_members     :: [pv1_server()],
          witnesses = []  :: [pv1_server()],
          creation_time   :: pv1_timestamp(),
          mode = ap_mode  :: pv1_consistency_mode(),
          upi             :: [pv1_server()],
          repairing       :: [pv1_server()],
          down            :: [pv1_server()],
          dbg             :: list(), %proplist(), is checksummed
          dbg2            :: list(), %proplist(), is not checksummed
          members_dict    :: p_srvr_dict()
         }).
 -define(MACHI_DEFAULT_TCP_PORT, 50000).
 -define(SHA_MAX, (1 bsl (20*8))).
 %% Set a limit to the maximum chain length, so that it's easier to
 %% create a consistent projection ranking score.
 -define(MAX_CHAIN_LENGTH, 64).
 -record(chain_def_v1, {
          name            :: atom(),         % chain name
          mode            :: pv1_consistency_mode(),
          full = []       :: [p_srvr()],
          witnesses = []  :: [p_srvr()],
          old_full = []   :: [pv1_server()], % guard against some races
          old_witnesses=[] :: [pv1_server()], % guard against some races
          local_run = []  :: [pv1_server()], % must be tailored to each machine!
          local_stop = [] :: [pv1_server()], % must be tailored to each machine!
          props = []      :: list()   % proplist for other related info
         }).
 -endif. % !MACHI_PROJECTION_HRL
--- a/include/machi_verbose.hrl
+++ b/include/machi_verbose.hrl
@ -1,31 +0,0 @@
 %% -------------------------------------------------------------------
 %%
 %% Machi: a small village of replicated files
 %%
 %% Copyright (c) 2014-2015 Basho Technologies, Inc.  All Rights Reserved.
 %%
 %% This file is provided to you under the Apache License,
 %% Version 2.0 (the "License"); you may not use this file
 %% except in compliance with the License.  You may obtain
 %% a copy of the License at
 %%
 %%   http://www.apache.org/licenses/LICENSE-2.0
 %%
 %% Unless required by applicable law or agreed to in writing,
 %% software distributed under the License is distributed on an
 %% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
 %% KIND, either express or implied.  See the License for the
 %% specific language governing permissions and limitations
 %% under the License.
 %%
 %% -------------------------------------------------------------------
 -ifdef(PULSE).
 -define(V(Fmt, Args), pulse:format(Fmt, Args)).
 -else.  % PULSE
 -define(V(Fmt, Args), io:format(user, Fmt, Args)).
 -endif. % PULSE
 -define(D(X), ?V("~s ~p\n", [??X, X])).
 -define(Dw(X), ?V("~s ~w\n", [??X, X])).
--- a/index.html
+++ b/index.html
@ -0,0 +1,38 @@
 <!DOCTYPE html>
 <html lang="en-us">
  <head>
    <meta charset="UTF-8">
    <title>Machi by basho</title>
    <meta name="viewport" content="width=device-width, initial-scale=1">
    <link rel="stylesheet" type="text/css" href="stylesheets/normalize.css" media="screen">
    <link href='http://fonts.googleapis.com/css?family=Open+Sans:400,700' rel='stylesheet' type='text/css'>
    <link rel="stylesheet" type="text/css" href="stylesheets/stylesheet.css" media="screen">
    <link rel="stylesheet" type="text/css" href="stylesheets/github-light.css" media="screen">
  </head>
  <body>
    <section class="page-header">
      <h1 class="project-name">Machi</h1>
      <h2 class="project-tagline">Machi file store: developer API</h2>
    </section>
    <section class="main-content">
      <p> <a href="edoc/">For a snapshot view of the HTML "EDoc" documentation for Machi, please follow this EDoc Link</a>.</p>
      <p> <a href="https://github.com/basho/machi/tree/master/doc/">All other Machi documentation.</a></p>
      <p> <a href="https://github.com/basho/machi">View README &amp; source on GitHub</a>
      <p> <a href="https://github.com/basho/machi/zipball/master">Download
      source as a ZIP file</a>
      <p> <a href="https://github.com/basho/machi/tarball/master">Download
      source as a compressed TAR file</a>
      <footer class="site-footer">
        <span class="site-footer-owner"><a href="https://github.com/basho/machi">Machi</a> is maintained by <a href="https://github.com/basho">basho</a>.</span>
        <span class="site-footer-credits">This page was generated by <a href="https://pages.github.com">GitHub Pages</a> using the <a href="https://github.com/jasonlong/cayman-theme">Cayman theme</a> by <a href="https://twitter.com/jasonlong">Jason Long</a>.</span>
      </footer>
    </section>
  </body>
 </html>
--- a/params.json
+++ b/params.json
@ -0,0 +1 @@
 {"name":"Machi","tagline":"Machi file store: developer API","body":"For a snapshot view of the HTML \"EDoc\" documentation for Machi, please follow the [EDoc Link](edoc/).","google":"","note":"Don't delete this file! It's used internally to help with page regeneration."}
--- a/priv/basho_bench.append-example.config
+++ b/priv/basho_bench.append-example.config
@ -1,45 +0,0 @@
 %% Mandatory: adjust this code path to top of your compiled Machi source distro
 {code_paths, ["/Users/fritchie/b/src/machi"]}.
 {driver, machi_basho_bench_driver}.
 %% Chose your maximum rate (per worker proc, see 'concurrent' below)
 %{mode, {rate,10}}.
 %{mode, {rate,20}}.
 {mode, max}.
 %% Runtime & reporting interval
 {duration, 10}.         % minutes
 {report_interval, 1}.   % seconds
 %% Choose your number of worker procs
 %{concurrent, 1}.
 {concurrent, 5}.
 %{concurrent, 10}.
 %% Here's a chain of (up to) length 3, all on localhost
 %% Note: if any servers are down, and your OS/TCP stack has an
 %% ICMP response limit such as OS X's "net.inet.icmp.icmplim" setting,
 %% then if that setting is very low (e.g., OS X's limit is 50), then
 %% you can have big problems with ICMP/RST responses being delayed and
 %% interactive *very* badly with your test.
 %% For OS X, fix using "sudo sysctl -w net.inet.icmp.icmplim=9999"
 {machi_server_info,
 [
  {p_srvr,a,machi_flu1_client,"localhost",4444,[]},
  {p_srvr,b,machi_flu1_client,"localhost",4445,[]},
  {p_srvr,c,machi_flu1_client,"localhost",4446,[]}
 ]}.
 {machi_ets_key_tab_type, set}.   % set | ordered_set
 %% Workload-specific definitions follow....
 %% 10 parts 'append' operation + 0 parts anything else = 100% 'append' ops
 {operations, [{append, 10}]}.
 %% For append, key = Machi file prefix name
 {key_generator, {to_binstr, "prefix~w", {uniform_int, 30}}}.
 %% Increase size of value_generator_source_size if value_generator is big!!
 {value_generator_source_size, 2111000}.
 {value_generator, {fixed_bin, 32768}}.   %  32 KB
--- a/priv/basho_bench.read-example.config
+++ b/priv/basho_bench.read-example.config
@ -1,47 +0,0 @@
 %% Mandatory: adjust this code path to top of your compiled Machi source distro
 {code_paths, ["/Users/fritchie/b/src/machi"]}.
 {driver, machi_basho_bench_driver}.
 %% Chose your maximum rate (per worker proc, see 'concurrent' below)
 %{mode, {rate,10}}.
 %{mode, {rate,20}}.
 {mode, max}.
 %% Runtime & reporting interval
 {duration, 10}.         % minutes
 {report_interval, 1}.   % seconds
 %% Choose your number of worker procs
 %{concurrent, 1}.
 {concurrent, 5}.
 %{concurrent, 10}.
 %% Here's a chain of (up to) length 3, all on localhost
 %% Note: if any servers are down, and your OS/TCP stack has an
 %% ICMP response limit such as OS X's "net.inet.icmp.icmplim" setting,
 %% then if that setting is very low (e.g., OS X's limit is 50), then
 %% you can have big problems with ICMP/RST responses being delayed and
 %% interactive *very* badly with your test.
 %% For OS X, fix using "sudo sysctl -w net.inet.icmp.icmplim=9999"
 {machi_server_info,
 [
  {p_srvr,a,machi_flu1_client,"localhost",4444,[]},
  {p_srvr,b,machi_flu1_client,"localhost",4445,[]},
  {p_srvr,c,machi_flu1_client,"localhost",4446,[]}
 ]}.
 {machi_ets_key_tab_type, set}.   % set | ordered_set
 %% Workload-specific definitions follow....
 %% 10 parts 'read' operation + 0 parts anything else = 100% 'read' ops
 {operations, [{read, 10}]}.
 %% For read, key = integer index into Machi's chunk ETS table, modulo the
 %%                ETS table size, so a huge number here is OK.
 {key_generator, {uniform_int, 999999999999}}.
 %% For read, value_generator_* isn't used, so leave these defaults as-is.
 {value_generator_source_size, 2111000}.
 {value_generator, {fixed_bin, 32768}}.   %  32 KB
--- a/priv/humming-consensus-demo.setup.sh
+++ b/priv/humming-consensus-demo.setup.sh
@ -1,56 +0,0 @@
 #!/bin/sh
 echo "Step: Verify that the required entries in /etc/hosts are present"
 for i in 1 2 3; do
    grep machi$i /etc/hosts | egrep -s '^127.0.0.1' > /dev/null 2>&1
    if [ $? -ne 0 ]; then
        echo ""
        echo "'grep -s machi$i' failed. Aborting, sorry."
        exit 1
    fi
    ping -c 1 machi$i > /dev/null 2>&1
    if [ $? -ne 0 ]; then
        echo ""
        echo "Ping attempt on host machi$i failed. Aborting."
        echo ""
        ping -c 1 machi$i
        exit 1
    fi
 done
 echo "Step: add a verbose logging option to app.config"
 for i in 1 2 3; do
    ed ./dev/dev$i/etc/app.config <<EOF > /dev/null 2>&1
 /verbose_confirm
 a
 {chain_manager_opts, [{private_write_verbose_confirm,true}]},
 {stability_time, 1},
 .
 w
 q
 EOF
 done
 echo "Step: start three three Machi application instances"
 for i in 1 2 3; do
    ./dev/dev$i/bin/machi start
    ./dev/dev$i/bin/machi ping
    if [ $? -ne 0 ]; then
        echo "Sorry, a 'ping' check for instance dev$i failed. Aborting."
        exit 1
    fi
 done
 echo "Step: configure one chain to start a Humming Consensus group with three members"
 # Note: $CWD of each Machi proc is two levels below the source code root dir.
 LIFECYCLE000=../../priv/quick-admin-examples/demo-000
 for i in 3 2 1; do
    ./dev/dev$i/bin/machi-admin quick-admin-apply $LIFECYCLE000 machi$i
    if [ $? -ne 0 ]; then
        echo "Sorry, 'machi-admin quick-admin-apply failed' on machi$i. Aborting."
        exit 1
    fi
 done
 exit 0
--- a/priv/humming-consensus-demo.vagrant/Vagrantfile
+++ b/priv/humming-consensus-demo.vagrant/Vagrantfile
@ -1,93 +0,0 @@
 # -*- mode: ruby -*-
 # vi: set ft=ruby :
 # All Vagrant configuration is done below. The "2" in Vagrant.configure
 # configures the configuration version (we support older styles for
 # backwards compatibility). Please don't change it unless you know what
 # you're doing.
 Vagrant.configure(2) do |config|
  # The most common configuration options are documented and commented below.
  # For a complete reference, please see the online documentation at
  # https://docs.vagrantup.com.
  # Every Vagrant development environment requires a box. You can search for
  # boxes at https://atlas.hashicorp.com/search.
  # If this Vagrant box has not been downloaded before (e.g. using "vagrant box add"),
  # then Vagrant will automatically download the VM image from HashiCorp.
  config.vm.box = "hashicorp/precise64"
  # If using a FreeBSD box, Bash may not be installed.
  # Use the config.ssh.shell setting to specify an alternate shell.
  # Note, however, that any code in the 'config.vm.provision' section
  # would then have to use this shell's syntax!
  #   config.ssh.shell = "/bin/csh -l"
  # Disable automatic box update checking. If you disable this, then
  # boxes will only be checked for updates when the user runs
  # `vagrant box outdated`. This is not recommended.
  # config.vm.box_check_update = false
  # Create a forwarded port mapping which allows access to a specific port
  # within the machine from a port on the host machine. In the example below,
  # accessing "localhost:8080" will access port 80 on the guest machine.
  # config.vm.network "forwarded_port", guest: 80, host: 8080
  # Create a private network, which allows host-only access to the machine
  # using a specific IP.
  # config.vm.network "private_network", ip: "192.168.33.10"
  # Create a public network, which generally matched to bridged network.
  # Bridged networks make the machine appear as another physical device on
  # your network.
  # config.vm.network "public_network"
  # Share an additional folder to the guest VM. The first argument is
  # the path on the host to the actual folder. The second argument is
  # the path on the guest to mount the folder. And the optional third
  # argument is a set of non-required options.
  # config.vm.synced_folder "../data", "/vagrant_data"
  # Provider-specific configuration so you can fine-tune various
  # backing providers for Vagrant. These expose provider-specific options.
  # Example for VirtualBox:
  #
  config.vm.provider "virtualbox" do |vb|
    # Display the VirtualBox GUI when booting the machine
    # vb.gui = true
    # Customize the amount of memory on the VM:
    vb.memory = "512"
  end
  #
  # View the documentation for the provider you are using for more
  # information on available options.
  # Define a Vagrant Push strategy for pushing to Atlas. Other push strategies
  # such as FTP and Heroku are also available. See the documentation at
  # https://docs.vagrantup.com/v2/push/atlas.html for more information.
  # config.push.define "atlas" do |push|
  #   push.app = "YOUR_ATLAS_USERNAME/YOUR_APPLICATION_NAME"
  # end
  # Enable provisioning with a shell script. Additional provisioners such as
  # Puppet, Chef, Ansible, Salt, and Docker are also available. Please see the
  # documentation for more information about their specific syntax and use.
  config.vm.provision "shell", inline: <<-SHELL
    # Install prerequsites
    # Support here for FreeBSD is experimental
    apt-get update ; sudo apt-get install -y git sudo rsync ; # Ubuntu Linux
    env ASSUME_ALWAYS_YES=yes pkg install -f git sudo rsync ; # FreeBSD 10
    # Install dependent packages, using slf-configurator
    git clone https://github.com/slfritchie/slf-configurator.git
    chown -R vagrant ./slf-configurator
    (cd slf-configurator ; sudo sh -x ./ALL.sh)
    echo 'export PATH=${PATH}:/usr/local/erlang/17.5/bin' >> ~vagrant/.bashrc
    export PATH=${PATH}:/usr/local/erlang/17.5/bin
    ## echo 'set path = ( $path  /usr/local/erlang/17.5/bin )' >> ~vagrant/.cshrc
    ## setenv PATH /usr/local/erlang/17.5/bin:$PATH
    git clone https://github.com/basho/machi.git
    (cd machi ; git checkout master ; make && make test )
    chown -R vagrant ./machi
  SHELL
 end
--- a/priv/make-faq.pl
+++ b/priv/make-faq.pl
@ -1,81 +0,0 @@
 #!/usr/bin/perl
 $input = shift;
 $tmp1 = "/tmp/my-tmp.1.$$";
 $tmp2 = "/tmp/my-tmp.2.$$";
 $l1 = 0;
 $l2 = 0;
 $l3 = 0;
 open(I, $input);
 open(T1, "> $tmp1");
 open(T2, "> $tmp2");
 while (<I>) {
    if (/^##*/) {
        $line = $_;
        chomp;
        @a = split;
        $count = length($a[0]) - 2;
        if ($count >= 0) {
            if ($count == 0) {
                $l1++;
                $l2 = 0;
                $l3 = 0;
                $label = "$l1"
            }
            if ($count == 1) {
                $l2++;
                $l3 = 0;
                $label = "$l1.$l2"
            }
            if ($count == 2) {
                $l3++;
                $label = "$l1.$l2.$l3"
            }
            $indent = " " x ($count * 4);
            s/^#*\s*[0-9. ]*//;
            $anchor = "n$label";
            printf T1 "%s+ [%s. %s](#%s)\n", $indent, $label, $_, $anchor;
            printf T2 "<a name=\"%s\">\n", $anchor;
            $line =~ s/(#+)\s*[0-9. ]*/$1 $label.  /;
            print T2 $line;
        } else {
            print T2 $_, "\n";
        }
    } else {
        next if /^<a name="n[0-9.]+">/;
        print T2 $_;
    }
 }
 close(I);
 close(T1);
 close(T2);
 open(T2, $tmp2);
 while (<T2>) {
    if (/<!\-\- OUTLINE \-\->/) {
        print;
        print "\n";
        open(T1, $tmp1);
        while (<T1>) {
            print;
        }
        close(T1);
        while (<T2>) {
            if (/<!\-\- ENDOUTLINE \-\->/) {
                print "\n";
                print;
                last;
            }
        }
    } else {
        print;
    }
 }
 close(T2);
 unlink($tmp1);
 unlink($tmp2);
 exit(0);
--- a/priv/quick-admin-examples/000
+++ b/priv/quick-admin-examples/000
@ -1 +0,0 @@
 {host, "localhost", []}.
--- a/priv/quick-admin-examples/001
+++ b/priv/quick-admin-examples/001
@ -1,4 +0,0 @@
 {flu,f1,"localhost",20401,[]}.
 {flu,f2,"localhost",20402,[]}.
 {flu,f3,"localhost",20403,[]}.
 {chain,c1,[f1,f2,f3],[]}.
--- a/priv/quick-admin-examples/002
+++ b/priv/quick-admin-examples/002
@ -1,4 +0,0 @@
 {flu,f4,"localhost",20404,[]}.
 {flu,f5,"localhost",20405,[]}.
 {flu,f6,"localhost",20406,[]}.
 {chain,c2,[f4,f5,f6],[]}.
--- a/priv/quick-admin-examples/demo-000
+++ b/priv/quick-admin-examples/demo-000
@ -1,7 +0,0 @@
 {host, "machi1", []}.
 {host, "machi2", []}.
 {host, "machi3", []}.
 {flu,f1,"machi1",20401,[]}.
 {flu,f2,"machi2",20402,[]}.
 {flu,f3,"machi3",20403,[]}.
 {chain,c1,[f1,f2,f3],[]}.
--- a/Show more
+++ b/Show more
		`@ -0,0 +1 @@`
							`{"name":"Machi","tagline":"Machi file store: developer API","body":"For a snapshot view of the HTML \"EDoc\" documentation for Machi, please follow the [EDoc Link](edoc/).","google":"","note":"Don't delete this file! It's used internally to help with page regeneration."}`