machi/doc/README.md

74 lines
3.4 KiB
Markdown
Raw Normal View History

2015-04-08 09:06:08 +00:00
## Machi Documentation Overview
For a Web-browsable version of a snapshot of the source doc "EDoc"
Erlang documentation, please use this link:
[Machi EDoc snapshot](https://basho.github.io/machi/edoc/).
## Documents in this directory
### chain-self-management-sketch.org
2015-05-05 10:29:36 +00:00
[chain-self-management-sketch.org](chain-self-management-sketch.org)
is a mostly-deprecated draft of
an introduction to the
self-management algorithm proposed for Machi. Most material has been
2015-05-05 10:29:36 +00:00
moved to the [high-level-chain-mgr.pdf](high-level-chain-mgr.pdf) document.
2015-06-17 01:44:35 +00:00
### cluster-of-clusters (directory)
This directory contains the sketch of the "cluster of clusters" design
strawman for partitioning/distributing/sharding files across a large
number of independent Machi clusters.
### high-level-machi.pdf
2015-05-05 10:29:36 +00:00
[high-level-machi.pdf](high-level-machi.pdf)
is an overview of the high level design for
Machi. Its abstract:
> Our goal is a robust & reliable, distributed, highly available large
> file store based upon write-once registers, append-only files, Chain
> Replication, and client-server style architecture. All members of
> the cluster store all of the files. Distributed load
> balancing/sharding of files is outside of the scope of this system.
> However, it is a high priority that this system be able to integrate
> easily into systems that do provide distributed load balancing,
> e.g., Riak Core. Although strong consistency is a major feature of
> Chain Replication, this document will focus mainly on eventual
> consistency features --- strong consistency design will be discussed
> in a separate document.
### high-level-chain-mgr.pdf
2015-05-05 10:29:36 +00:00
[high-level-chain-mgr.pdf](high-level-chain-mgr.pdf)
is an overview of the techniques used by
Machi to manage Chain Replication metadata state. It also provides an
introduction to the Humming Consensus algorithm. Its abstract:
> Machi is an immutable file store, now in active development by Basho
> Japan KK. Machi uses Chain Replication to maintain strong consistency
> of file updates to all replica servers in a Machi cluster. Chain
> Replication is a variation of primary/backup replication where the
> order of updates between the primary server and each of the backup
> servers is strictly ordered into a single ``chain''. Management of
> Chain Replication's metadata, e.g., ``What is the current order of
> servers in the chain?'', remains an open research problem. The
> current state of the art for Chain Replication metadata management
> relies on an external oracle (e.g., ZooKeeper) or the Elastic
> Replication algorithm.
>
> This document describes the Machi chain manager, the component
> responsible for managing Chain Replication metadata state. The chain
> manager uses a new technique, based on a variation of CORFU, called
> ``humming consensus''.
> Humming consensus does not require active participation by all or even
> a majority of participants to make decisions. Machi's chain manager
> bases its logic on humming consensus to make decisions about how to
> react to changes in its environment, e.g. server crashes, network
> partitions, and changes by Machi cluster admnistrators. Once a
> decision is made during a virtual time epoch, humming consensus will
> eventually discover if other participants have made a different
> decision during that epoch. When a differing decision is discovered,
> new time epochs are proposed in which a new consensus is reached and
> disseminated to all available participants.