Scott Lystig Fritchie
d2f1549aa3
WIP: more generic all-way file chunk merge func, part 3
2015-05-16 17:11:54 +09:00
Scott Lystig Fritchie
04bc28b9da
WIP: more generic all-way file chunk merge func, part 2
2015-05-16 16:55:48 +09:00
Scott Lystig Fritchie
a9c753ad64
WIP: more generic all-way file chunk merge func
2015-05-15 17:15:02 +09:00
Scott Lystig Fritchie
358764d403
WIP: repair common file, part 0
2015-05-14 14:04:31 +09:00
Scott Lystig Fritchie
19d3c95325
WIP: aside, damn, add missing hex encoding for epochid, derp
2015-05-13 18:57:38 +09:00
Scott Lystig Fritchie
eec029b08f
WIP: aside, fix FLU wedge status @ init()
2015-05-13 17:59:32 +09:00
Scott Lystig Fritchie
4ae0f94649
WIP: move to stats via ETS, success/failure propagates, yay!
2015-05-12 23:45:35 +09:00
Scott Lystig Fritchie
cad84442bb
WIP: stats record, hrm
2015-05-12 22:42:03 +09:00
Scott Lystig Fritchie
8807f954ff
WIP: Whole file repair is 95% complete, yay!
2015-05-12 21:45:40 +09:00
Scott Lystig Fritchie
f48720e4dc
WIP: set up proxies for repair
2015-05-12 12:56:41 +09:00
Scott Lystig Fritchie
1c70a46b09
Add basic process & bookkeeping structure for repair proc
...
=INFO REPORT==== 11-May-2015::19:50:09 ===
Chain tail a of [a] starting repair of [c]
=INFO REPORT==== 11-May-2015::19:50:12 ===
Chain tail a of [a]: repair finished in 2.438 seconds: todo_yo
2015-05-11 19:50:21 +09:00
Scott Lystig Fritchie
c82000dc30
Reduce spamminess slightly
2015-05-11 19:00:21 +09:00
Scott Lystig Fritchie
33bfbe109e
Chain manager bug fixes & enhancment (more...)
...
* Set max length of a chain at -define(MAX_CHAIN_LENGTH, 64).
* Perturb tick sleep time of each manager
* If a chain manager L has zero members in its chain, and then its local
public projection store (authored by some remote author R) has a projection
that contains L, then adopt R's projection and start humming consensus.
* Handle "cross-talk" across projection stores, when chain membership
is changed administratively, e.g. chain was [a,b,c] then changed to merely
[a], but that change only happens on a. Servers b & c continue to use
stale projections and scribble their projection suggestions to a, causing
it to flap.
What's really cool about the flapping handling is that it *works*. I
wasn't thinking about this scenario when designing the flapping logic, but
it's really nifty that this extra scenario causes a to flap and then a's
inner projection remains stable, yay!
* Add complaints when "cross-talk" is observed.
* Fix flapping sleep time throttle.
* Fix bug in the machi_projection_store.erl's bookkeeping of the
max epoch number when flapping.
2015-05-11 18:41:45 +09:00
Scott Lystig Fritchie
dc43a32945
WIP: tests for wedge state all working
2015-05-08 21:37:19 +09:00
Scott Lystig Fritchie
0dd9282789
WIP: fix other broken eunit tests, surrounding wedge state
2015-05-08 21:24:07 +09:00
Scott Lystig Fritchie
6f7818fca7
WIP: additional tests for wedge state
2015-05-08 19:50:47 +09:00
Scott Lystig Fritchie
316126fa59
WIP: additional tests for wedge state
2015-05-08 19:07:57 +09:00
Scott Lystig Fritchie
7906e6c235
WIP: basic wedge notifications now working
2015-05-08 18:17:41 +09:00
Scott Lystig Fritchie
d6d003618d
WIP: add wedge_status() query to proxy client
2015-05-08 16:58:06 +09:00
Scott Lystig Fritchie
1dc759b908
WIP: add wedge_status() query to client
2015-05-08 16:53:10 +09:00
Scott Lystig Fritchie
762aef557f
WIP: Set the stage for FLU wedging API
2015-05-08 15:36:53 +09:00
Scott Lystig Fritchie
ae1d038abe
Change default value of chmgr's use_partition_simulator to false
2015-05-08 13:40:44 +09:00
Scott Lystig Fritchie
238c8472cd
WIP: timeout comments
2015-05-07 18:52:01 +09:00
Scott Lystig Fritchie
14fc37bd0d
Add ability to start FLUs at application startup
2015-05-07 18:39:39 +09:00
Scott Lystig Fritchie
517941aaaa
Finish chain manager restart & membership changing
2015-05-07 17:52:16 +09:00
Scott Lystig Fritchie
aeb2e4ef9e
WIP: partial refactoring of chmgr 2nd start code, one test broken
2015-05-06 11:41:04 +09:00
Scott Lystig Fritchie
a7bd8e43d3
Clean up machi_flu_psup_test.erl
2015-05-02 17:10:23 +09:00
Scott Lystig Fritchie
1675020150
WIP, tests pass again, includign the newest one
2015-05-02 00:33:49 +09:00
Scott Lystig Fritchie
53f6a753f4
WIP: tests pass, but not finished yet
2015-05-01 14:51:42 +09:00
Scott Lystig Fritchie
7bafc1c28a
WIP: stop for the night, we are broken
2015-04-30 23:16:08 +09:00
Scott Lystig Fritchie
99fd7e7fe1
Add test/machi_flu_psup_test.erl, small cleanups
2015-04-30 21:20:21 +09:00
Scott Lystig Fritchie
442e79e4f1
Add machi_flu_psup.erl to supervise all 3 FLU processes (see below)
...
Introduce machi_flu_psup:start_flu_package/4 as a way to start all
related FLU processes
* The projection store
* The chain manager
* The FLU itself
... as well as linked processes.
http://www.snookles.com/scotttmp/flu-tree-20150430.png shows one FLU
running, "a". The process registered "a" is the append server,
"some-prefix" for the sequencer & writer for the current <<"some-prefix">>
file, and a process each for 3 active TCP connections to that FLU.
2015-04-30 19:15:27 +09:00
Scott Lystig Fritchie
9249663ff6
TODO updates for machi_flu1.erl based on doc rewriting work
2015-04-30 15:15:43 +09:00
Scott Lystig Fritchie
02bc7fe0bc
WIP: Fix bug that flaps inside an inner projection, oops!
2015-04-14 18:23:00 +09:00
Scott Lystig Fritchie
90df655256
WIP: Ha! There's a bug, this verbose logging change made it easier to see
2015-04-14 16:38:19 +09:00
Scott Lystig Fritchie
9e587b3d11
WIP: crufty TODO & comment cleanup
2015-04-14 16:17:49 +09:00
Scott Lystig Fritchie
59936eda62
WIP: By Jove, I believe the chain manager is working
2015-04-14 15:30:24 +09:00
Scott Lystig Fritchie
09051aecce
WIP: experiments for transitioning out of inner/nested projection state
2015-04-14 00:54:38 +09:00
Scott Lystig Fritchie
94298d90da
WIP: transitions into & out of inner proj nesting are problems, yo!
2015-04-10 22:41:22 +09:00
Scott Lystig Fritchie
0b8ea13f7a
WIP: some TODO cleanup & related refactoring
2015-04-10 22:00:52 +09:00
Scott Lystig Fritchie
876bf79835
Add debugging & TODO note about using inner projection
2015-04-10 14:15:16 +09:00
Scott Lystig Fritchie
4f7177067e
WIP: Type fixups
2015-04-09 21:32:04 +09:00
Scott Lystig Fritchie
1984c3c350
WIP: convergence demo runs, but badly!
2015-04-09 21:08:15 +09:00
Scott Lystig Fritchie
6cd9dfc977
WIP: nonunanimous_setup_and_fix_test() passes
2015-04-09 17:47:43 +09:00
Scott Lystig Fritchie
e06adabb6a
WIP: bogus flapping in nonunanimous_setup_and_fix_test()
2015-04-09 17:13:38 +09:00
Scott Lystig Fritchie
8deea3bb01
WIP: smoke1 in chain manager works
2015-04-09 14:44:58 +09:00
Scott Lystig Fritchie
a92c5fec0a
WIP: remove old smoke0 code from machi_proxy_flu1_client_test.erl
2015-04-09 12:21:15 +09:00
Scott Lystig Fritchie
ce67fb662a
WIP: more projection refactoring, eunit tests pass for the moment
2015-04-09 12:16:58 +09:00
Scott Lystig Fritchie
0b2866d102
Add -spec statements to machi_util.erl, clean up the fallout
2015-04-08 18:39:55 +09:00
Scott Lystig Fritchie
ad872e23ca
Add first basic round of EDoc documentation, 'make edoc' target
2015-04-08 17:32:01 +09:00
Scott Lystig Fritchie
0e38eddaa9
WIP: baby step, machi_chain_manager1_test:smoke0_test() works
2015-04-06 20:07:39 +09:00
Scott Lystig Fritchie
16e283fe5b
API overhaul, add machi_proxy_flu1_client.erl, add chain manager (tests commented out)
2015-04-06 18:43:52 +09:00
Scott Lystig Fritchie
1d63b93fc0
Kill append, projection, and listen pids on machi_flu1:stop()
2015-04-06 16:49:17 +09:00
Scott Lystig Fritchie
a79f385fa7
Fix type problem for return of get_latest_epoch
2015-04-06 15:49:47 +09:00
Scott Lystig Fritchie
99bfa2a3b8
Import of machi_chain_manager1.erl and friends; tests broken
2015-04-06 14:16:20 +09:00
Scott Lystig Fritchie
c27aa1f579
Projection store API complete, I think
2015-04-03 18:37:09 +09:00
Scott Lystig Fritchie
022b9c4d1f
WIP: projection store: read, get latest epoch
2015-04-03 17:55:35 +09:00
Scott Lystig Fritchie
acf54e3c21
WIP: client side projection store, 1st API op (write), part II
2015-04-03 17:10:52 +09:00
Scott Lystig Fritchie
7205c5283e
WIP: client side projection store, 1st API op (write)
2015-04-03 12:36:16 +09:00
Scott Lystig Fritchie
7627ba08a3
WIP: epoch ID added to write/delete/trunc protocol commands
2015-04-02 21:18:41 +09:00
Scott Lystig Fritchie
3aaa2c3a3d
WIP: epoch ID added to list protocol command
2015-04-02 21:01:48 +09:00
Scott Lystig Fritchie
6b8a3cf2a4
WIP: epoch ID added to checksum protocol command
2015-04-02 20:49:45 +09:00
Scott Lystig Fritchie
9479baac46
WIP: epoch ID added to read protocol command
2015-04-02 20:31:10 +09:00
Scott Lystig Fritchie
44bb5e1dae
WIP: epoch ID added to append protocol command
2015-04-02 18:08:42 +09:00
Scott Lystig Fritchie
030d2ecd10
Update TODO-shortterm.org + minor stuff
2015-04-02 17:42:26 +09:00
Scott Lystig Fritchie
5580098d49
Refactor to use record for FLU state, add dbg mode
2015-04-02 17:16:15 +09:00
Scott Lystig Fritchie
4c3bd81689
Add machi_projection.erl and basic new() test
2015-04-02 16:24:32 +09:00
Scott Lystig Fritchie
58fa35a674
Remove escript-related proof-of-concept stuff from machi_util.erl
...
I'd first thought that having that code there would be a kind of
useful reminder: please move me somewhere else. However, there's
quite a bit there that's "cluster of clusters" stuff and not
appropriate for the current short-term work.
2015-04-02 14:36:22 +09:00
Scott Lystig Fritchie
f8263c15cc
Move FLU client 'verify checksums + local path' code from prototype/demo-day-hack
2015-04-02 12:38:12 +09:00
Scott Lystig Fritchie
76fcd4d931
Move FLU client 'verify checksums' code from prototype/demo-day-hack
2015-04-01 18:35:10 +09:00
Scott Lystig Fritchie
5c20ee6337
Fix client API for file list & checksum list
2015-04-01 18:02:16 +09:00
Scott Lystig Fritchie
d243ffca23
Single server client & server code (squashed)
2015-04-01 16:14:24 +09:00
