Commit graph

50 commits

Author SHA1 Message Date
Scott Lystig Fritchie
a3fc1c3d68 Add namespace info to wedge_status API call; add namespace enforcement @ machi_flu1_net_server 2015-12-31 14:34:15 +09:00
Scott Lystig Fritchie
3c6f1be5d0 Change read_chunk options to use new #read_opts{} 2015-12-29 18:47:08 +09:00
Scott Lystig Fritchie
2932a17ea6 append_chunk API refactoring; all tests pass; todo tasks remain 2015-12-29 14:13:29 +09:00
Scott Lystig Fritchie
c49ccafdc6 Merge slf/flu-config-rcd-style 2015-12-18 15:41:02 +09:00
Scott Lystig Fritchie
6b7d871ada WIP: diff in progress 2 2015-12-11 16:26:13 +09:00
Scott Lystig Fritchie
9472bad37b Clean up test errors 2015-12-10 15:57:35 +09:00
Scott Lystig Fritchie
61ef7739cd Modify chain mostly works, better 2015-12-10 00:12:34 +09:00
Scott Lystig Fritchie
2871f8397c WIP: modify chain still a bit broken 2015-12-09 17:19:02 +09:00
Shunichi Shinohara
83b4466a85 Merge pull request #50 from basho/slf/doc-name-game2
Review & merge slf/doc-name-game2
2015-12-09 09:45:11 +09:00
Scott Lystig Fritchie
8285899dba Bootstrap chain @ app init: done, with an example.
For example:

% make clean
% make stage

And then configure 3 FLUs:

    % echo '{p_srvr, a, machi_flu1_client, "localhost", 39000, []}.' > rel/machi/etc/flu-config/a
    % echo '{p_srvr, b, machi_flu1_client, "localhost", 39001, []}.' > rel/machi/etc/flu-config/b
    % echo '{p_srvr, c, machi_flu1_client, "localhost", 39002, []}.' > rel/machi/etc/flu-config/c

And then configure a chain to use 2 of those 3 FLUs:

    % echo '{chain_def_v1,c1,ap_mode,[{p_srvr,a,machi_flu1_client,"localhost",39000,[]},{p_srvr,b,machi_flu1_client,"localhost",39001,[]}],[],[]}.' > rel/machi/etc/chain-config/c1

... then start Machi e.g.

    % ./rel/machi/bin/machi console

... you should see the following console messages scroll by (including a :

    =PROGRESS REPORT==== 8-Dec-2015::22:01:44 ===
              supervisor: {local,machi_flu_sup}
                 started: [{pid,<0.145.0>},
                           {name,a},
                           {mfargs,
                               {machi_flu_psup,start_link,
                                   [a,39000,"./data/flu/a",[]]}},
                           {restart_type,permanent},
                           {shutdown,5000},
                           {child_type,supervisor}]

    [... and also for the other two FLUs, including a bunch of progress
         reports for processes that started underneath that sub-supervisor.]

    22:01:44.446 [info] Running FLUs: [a,b,c]
    22:01:44.446 [info] Running FLUs at epoch 0: [a,b,c]
    22:01:44.532 [warning] The following FLUs are defined but are not also members of a defined chain: [c]
2015-12-08 21:57:29 +09:00
Scott Lystig Fritchie
ef10ebed22 WIP: now trying to diagnose fitness server bug? 2015-12-08 14:50:16 +09:00
Scott Lystig Fritchie
5aeaf872d9 WIP: machi_chain_manager1:set_chain_members() API change, all tests pass, yay 2015-12-07 14:41:56 +09:00
Scott Lystig Fritchie
3c880dc437 WIP: find 1st overlapping FLU in any #chain_def_v1{} 2015-12-04 17:47:18 +09:00
Scott Lystig Fritchie
10a27ce7dd All eunit tests now passing again 2015-12-02 15:35:38 +09:00
Mark Allen
1f56850c2b Remove merklet from header file too. 2015-11-20 22:34:41 -06:00
Mark Allen
8d3f631d84 Oops. Accidentally left this out. 2015-11-17 22:07:28 -06:00
Scott Lystig Fritchie
d2b1c7512a Merge branch 'ku/config-system' into tmp 2015-10-27 14:14:40 +09:00
UENISHI Kota
b2eb3e089c Cleanup MACROs and changed default value
* machi_file_proxy now uses application environment
  value `max_file_size` via machi_config
* changed name from MAX_FILE_SIZE to DEFAULT_MAX_FILE_SIZE
2015-10-27 11:17:59 +09:00
UENISHI Kota
0e4ae818af Clarify checksum tags, define macros to avoid typos 2015-10-23 19:32:22 +09:00
Scott Lystig Fritchie
75c94420e0 Add test_ets_table to give programmatic slowdown 2015-09-14 22:52:41 +09:00
Scott Lystig Fritchie
185c9eb313 WIP: add failing eunit placeholder for spam 2015-09-07 15:38:23 +09:00
Scott Lystig Fritchie
c7684f660c WIP: Friday evening/Monday morning, laying groundwork for spam "broadcast" 2015-09-07 15:20:10 +09:00
Scott Lystig Fritchie
4376ce9ec1 Remove all flap counting and inner projection stuff 2015-09-04 17:17:49 +09:00
Scott Lystig Fritchie
2f82fe0487 WIP: cp_mode improvements 2015-08-24 19:04:26 +09:00
Scott Lystig Fritchie
4c3051ddf1 Oops, forgot to add include dir changes 2015-08-20 21:27:17 +09:00
Scott Lystig Fritchie
6e521700bd WIP: Adding witness_smoke_test_ but it's broken (more)
So, the problem is that the chain manager isn't finishing repair
because UPI=[a], and a is a witness, and a can't do the list files etc etc
repair stuff that repairer FLUs need to do.

The best (?) way forward is to add some advance smarts to the
chain manager so that it doesn't propose a UPI of 100% witnesses?
2015-07-21 19:05:04 +09:00
Scott Lystig Fritchie
f4d16881c0 WIP: bugfix for rare flapping infinite loop (better fix I hope)
%% So, I'd tried this kind of "if everyone is doing it, then we
        %% 'agree' and we can do something different" strategy before,
        %% and it didn't work then.  Silly me.  Distributed systems
        %% lesson #823: do not forget the past.  In a situation created
        %% by PULSE, of all=[a,b,c,d,e], b & d & e were scheduled
        %% completely unfairly.  So a & c were the only authors ever to
        %% suceessfully write a suggested projection to a public store.
        %% Oops.
        %%
        %% So, we're going to keep track in #ch_mgr state for the number
        %% of times that this insane judgement has happened.
2015-07-17 14:51:39 +09:00
Scott Lystig Fritchie
73ac220d75 Add machi_verbose.hrl 2015-07-16 16:01:53 +09:00
Scott Lystig Fritchie
517e77dc4a WIP: bugfix for rare flapping infinite loop (unfinished) part III 2015-07-15 17:35:12 +09:00
Scott Lystig Fritchie
6d95d8669c WIP: giant hairball, bleh, low-level checksum_list() barely working 2015-06-26 16:25:12 +09:00
Scott Lystig Fritchie
5ef499ec73 WIP: append_chunk #1 2015-06-23 14:08:10 +09:00
Scott Lystig Fritchie
deabe14d29 Un-proplist-ify the inner projection 2015-06-02 20:55:18 +09:00
Scott Lystig Fritchie
207be8729b Un-proplist-ify the flapping_i info 2015-06-02 20:32:52 +09:00
Scott Lystig Fritchie
e3162fdcda Rudimentary client-side checksum and server-side checksum type tags 2015-06-01 14:25:55 +09:00
Scott Lystig Fritchie
d293170e92 WIP: starting machi_cr_client.erl 2015-05-17 23:48:05 +09:00
Scott Lystig Fritchie
a06055ac23 WIP: rearrange client code to approach some semblance of modularity someday 2015-05-17 16:18:30 +09:00
Scott Lystig Fritchie
1c70a46b09 Add basic process & bookkeeping structure for repair proc
=INFO REPORT==== 11-May-2015::19:50:09 ===
    Chain tail a of [a] starting repair of [c]

    =INFO REPORT==== 11-May-2015::19:50:12 ===
    Chain tail a of [a]: repair finished in 2.438 seconds: todo_yo
2015-05-11 19:50:21 +09:00
Scott Lystig Fritchie
33bfbe109e Chain manager bug fixes & enhancment (more...)
* Set max length of a chain at -define(MAX_CHAIN_LENGTH, 64).

* Perturb tick sleep time of each manager

* If a chain manager L has zero members in its chain, and then its local
public projection store (authored by some remote author R) has a projection
that contains L, then adopt R's projection and start humming consensus.

* Handle "cross-talk" across projection stores, when chain membership
is changed administratively, e.g. chain was [a,b,c] then changed to merely
[a], but that change only happens on a.  Servers b & c continue to use
stale projections and scribble their projection suggestions to a, causing
it to flap.

What's really cool about the flapping handling is that it *works*.  I
wasn't thinking about this scenario when designing the flapping logic, but
it's really nifty that this extra scenario causes a to flap and then a's
inner projection remains stable, yay!

* Add complaints when "cross-talk" is observed.

* Fix flapping sleep time throttle.

* Fix bug in the machi_projection_store.erl's bookkeeping of the
max epoch number when flapping.
2015-05-11 18:41:45 +09:00
Scott Lystig Fritchie
517941aaaa Finish chain manager restart & membership changing 2015-05-07 17:52:16 +09:00
Scott Lystig Fritchie
59936eda62 WIP: By Jove, I believe the chain manager is working 2015-04-14 15:30:24 +09:00
Scott Lystig Fritchie
0b8ea13f7a WIP: some TODO cleanup & related refactoring 2015-04-10 22:00:52 +09:00
Scott Lystig Fritchie
4f7177067e WIP: Type fixups 2015-04-09 21:32:04 +09:00
Scott Lystig Fritchie
e06adabb6a WIP: bogus flapping in nonunanimous_setup_and_fix_test() 2015-04-09 17:13:38 +09:00
Scott Lystig Fritchie
8deea3bb01 WIP: smoke1 in chain manager works 2015-04-09 14:44:58 +09:00
Scott Lystig Fritchie
ce67fb662a WIP: more projection refactoring, eunit tests pass for the moment 2015-04-09 12:16:58 +09:00
Scott Lystig Fritchie
99bfa2a3b8 Import of machi_chain_manager1.erl and friends; tests broken 2015-04-06 14:16:20 +09:00
Scott Lystig Fritchie
44bb5e1dae WIP: epoch ID added to append protocol command 2015-04-02 18:08:42 +09:00
Scott Lystig Fritchie
4c3bd81689 Add machi_projection.erl and basic new() test 2015-04-02 16:24:32 +09:00
Scott Lystig Fritchie
58fa35a674 Remove escript-related proof-of-concept stuff from machi_util.erl
I'd first thought that having that code there would be a kind of
useful reminder: please move me somewhere else.  However, there's
quite a bit there that's "cluster of clusters" stuff and not
appropriate for the current short-term work.
2015-04-02 14:36:22 +09:00
Scott Lystig Fritchie
d243ffca23 Single server client & server code (squashed) 2015-04-01 16:14:24 +09:00