greg/machi
1
0
Fork 0
Code Issues 11 Pull requests 1 Projects Releases Packages Wiki Activity Actions
584 commits 58 branches 0 tags 5.9 MiB
Commit graph

214 commits

Author SHA1 Message Date
Scott Lystig Fritchie
7aff9fca70 WIP: giant hairball 12 2015-06-29 16:42:05 +09:00
Scott Lystig Fritchie
b25ab3b7ac WIP: giant hairball 11 2015-06-29 16:24:57 +09:00
Scott Lystig Fritchie
64817dd7e8 WIP: giant hairball 01 2015-06-29 16:10:43 +09:00
Scott Lystig Fritchie
f45dc7829e WIP: hairball, but: Failed: 6. Skipped: 0. Passed: 13 2015-06-27 00:43:27 +09:00
Scott Lystig Fritchie
b5c824c5c0 WIP: hairball, but bad_checksum_test() works! 2015-06-27 00:06:21 +09:00
Scott Lystig Fritchie
2fd27fdae6 WIP: hairball, but flu_projection_smoke_test() works! 2015-06-26 23:58:34 +09:00
Scott Lystig Fritchie
93f64a20c0 WIP: hairball, but flu_smoke_test() works! 2015-06-26 23:03:28 +09:00
Scott Lystig Fritchie
920a5c33d7 WIP: giant hairball 6 2015-06-26 22:32:53 +09:00
Scott Lystig Fritchie
77b4da16c3 WIP: giant hairball 5 2015-06-26 21:36:07 +09:00
Scott Lystig Fritchie
9a212fb19f WIP: giant hairball 4 2015-06-26 20:47:55 +09:00
Scott Lystig Fritchie
0e32fd25c9 WIP: giant hairball 3 2015-06-26 18:59:07 +09:00
Scott Lystig Fritchie
8437d76c1c WIP: giant hairball 2 2015-06-26 18:22:15 +09:00
Scott Lystig Fritchie
fb975eea46 WIP: giant hairball 2015-06-26 16:58:24 +09:00
Scott Lystig Fritchie
6d95d8669c WIP: giant hairball, bleh, low-level checksum_list() barely working 2015-06-26 16:25:12 +09:00
Scott Lystig Fritchie
90efc41167 machi.proto definition for low-level protocol ops 2015-06-25 17:09:33 +09:00
Scott Lystig Fritchie
cf0d9a25b4 EDoc cleanup 2015-06-25 16:39:19 +09:00
Scott Lystig Fritchie
0b2b79cd0b Merge branch 'slf/pb-api-experiment1' 2015-06-25 16:36:50 +09:00
Scott Lystig Fritchie
0f4d5ed775 Silence dialyzer unused function clause 2015-06-25 16:36:29 +09:00
Scott Lystig Fritchie
c2faf9f499 yolo, un-do experimental type hack 2015-06-25 16:36:14 +09:00
Scott Lystig Fritchie
d9694a992a Alright, use term_to_binary() for opaque/sexp-style encoding, only 15x slower. 
machi_flu1_test: timing_pb_encoding_test_... speed factor=15.12 [2.678 s] ok
 2015-06-25 16:11:46 +09:00
Scott Lystig Fritchie
2763b16ca2 timing_pb_encoding_test_... speed factor=35.95 [2.730 s] ok 
So, the PB style encoding of the Mpb_LL_WriteProjectionReq message
is about 35-36 times slower than using Erlang's term_to_binary()
and binary_to_term().  {sigh}
 2015-06-25 16:11:44 +09:00
Scott Lystig Fritchie
5d8b648a24 All projection store protocol operations are now using Protocol Buffers! 
So, there's some cheating going on, because some of the parts of
the #projection_v1{} and #p_srvr{} records aren't fully specified.
Those parts are being specified as "opaque" in the field names, e.g.

    optional bytes opaque_flap = 10;
    optional bytes opaque_inner = 11;
    required bytes opaque_dbg = 12;
    required bytes opaque_dbg2 = 13;

The serialization that's being used is erlang term sexprs.  That isn't
portable.  So if/when we really need to deal with a non-Erlang
language, we'll have to straighten this out further.
 2015-06-25 15:26:35 +09:00
Scott Lystig Fritchie
841235b3b5 WIP: bugfixes, add {error, written} 2015-06-25 15:10:24 +09:00
Scott Lystig Fritchie
4fc0578a9d WIP: bugfixes, machi_flu1_test still broken 2015-06-25 15:08:40 +09:00
Scott Lystig Fritchie
d9407b76b7 WIP: dinnertime, machi_flu1_test still broken 2015-06-24 18:00:25 +09:00
Scott Lystig Fritchie
31c5bcc0c7 WIP: 1/2 of low-level projection proto finished, machi_flu1_test fails 2015-06-24 17:20:18 +09:00
Scott Lystig Fritchie
725b10ba90 Complete PB round-trip for #projection_v1{}, bleh 2015-06-24 16:13:11 +09:00
Scott Lystig Fritchie
2068f70700 WIP: encoding #p_srvr and #projection_v1, just starting. Damn tedious. 2015-06-24 12:50:37 +09:00
Scott Lystig Fritchie
817efb2b15 machi_pb_high_client: always be checksumming 2015-06-23 17:37:47 +09:00
Scott Lystig Fritchie
d3b0b7fdc5 Clean up some dialyzer complaints 2015-06-23 17:26:15 +09:00
Scott Lystig Fritchie
727b2a987d ROTFL forgot to add src/machi_pb_server.erl 2015-06-23 17:22:45 +09:00
Scott Lystig Fritchie
ceebe3d491 WIP: list_files #2 2015-06-23 17:17:14 +09:00
Scott Lystig Fritchie
73f71c406e WIP: list_files end-to-end! 2015-06-23 17:08:15 +09:00
Scott Lystig Fritchie
6722b3c0f1 WIP: checksum_list incomplete implementation.... 2015-06-23 16:53:06 +09:00
Scott Lystig Fritchie
44c22bf752 WIP: read_chunk #1 2015-06-23 15:34:48 +09:00
Scott Lystig Fritchie
a8782eed5a WIP: write_chunk #1 2015-06-23 15:13:13 +09:00
Scott Lystig Fritchie
cb06c53dc0 WIP: PB append_chunk end-to-end works! 2015-06-23 14:45:24 +09:00
Scott Lystig Fritchie
5ef499ec73 WIP: append_chunk #1 2015-06-23 14:08:10 +09:00
Scott Lystig Fritchie
bb8e725c26 WIP: 'auth' request placeholders 2015-06-22 18:16:15 +09:00
Scott Lystig Fritchie
db7f1476b9 WIP: 'echo' request works end-to-end, yay! 2015-06-22 18:04:17 +09:00
Scott Lystig Fritchie
3d05f543df WIP: new test case is failing, quick fix soon 2015-06-22 17:49:07 +09:00
Scott Lystig Fritchie
dc9f272c44 Nearly dumbest-possible Protocol Buffers client request & response round trip 2015-06-19 17:21:04 +09:00
Scott Lystig Fritchie
c4bdeee4da Oops, add missing src/machi_dt.erl 2015-06-19 17:20:49 +09:00
Scott Lystig Fritchie
0cdaee32f8 Egadz, edoc doesn't use preprocessor {sigh} 2015-06-19 16:24:57 +09:00
Scott Lystig Fritchie
984b4f7a86 Dialyzer tightening and subsequent cleanup 2015-06-19 16:04:34 +09:00
Scott Lystig Fritchie
1372bd9594 {sigh} add filter-dialyzer-dep-warnings 2015-06-19 15:22:07 +09:00
Scott Lystig Fritchie
3c300bb9f1 Add write_chunk() to machi_cr_client.erl 2015-06-19 14:49:09 +09:00
Scott Lystig Fritchie
40c0a72b48 Add test/machi_pb_test.erl, finish PB refactoring 2015-06-19 13:00:28 +09:00
Scott Lystig Fritchie
a82bd68f3c Overhaul the 0.1 PB definition. Again. 
Many thanks to @seancribbs for a suggestion to avoid the PB design
mistake/feature of the original Riak KV PB API.
 2015-06-19 12:28:31 +09:00
Scott Lystig Fritchie
87b636a349 WIP: PB wrestling 2015-06-18 17:31:48 +09:00
Scott Lystig Fritchie
37a8c1c124 WIP: PB wrestling 2015-06-18 16:16:23 +09:00
Scott Lystig Fritchie
e5673b5e20 First attempt at Protocol Buffers .proto + infrastructure 2015-06-17 16:12:20 +09:00
Scott Lystig Fritchie
22337e1819 Remove short circuit (bad idea!) from react_to_env_C100() 2015-06-15 17:22:02 +09:00
Scott Lystig Fritchie
b244a3b8e4 Reduce verbosity, try fix up convergence demo for chain len=4 2015-06-15 12:41:16 +09:00
Scott Lystig Fritchie
9bf76e0bfb Fix for correctness bug, thanks PULSE 2015-06-05 01:06:39 +09:00
Scott Lystig Fritchie
be62300b3b Bug fixes: model and real bugs, thanks PULSE and converge_demo both! 2015-06-04 17:39:29 +09:00
Scott Lystig Fritchie
0cf9627f26 Bugfix, found by inspection, yay! 2015-06-04 15:05:37 +09:00
Scott Lystig Fritchie
89b8b6a012 Bugfix, found by PULSE, yay! 2015-06-04 14:31:58 +09:00
Scott Lystig Fritchie
d3df2bd31d WIP: remove repair_always_done option, it was flawed 2015-06-03 15:26:22 +09:00
Scott Lystig Fritchie
87417d2872 WIP: get the old jalopy into runnable shape 2015-06-03 11:48:55 +09:00
Scott Lystig Fritchie
2207151eba Fix projection_transition_is_sane() bug 2015-06-02 21:20:50 +09:00
Scott Lystig Fritchie
deabe14d29 Un-proplist-ify the inner projection 2015-06-02 20:55:18 +09:00
Scott Lystig Fritchie
207be8729b Un-proplist-ify the flapping_i info 2015-06-02 20:32:52 +09:00
Scott Lystig Fritchie
0f10b45161 Dialyzer fixes, derp! 2015-06-02 19:07:13 +09:00
Scott Lystig Fritchie
000d687588 Fix creation_time bug in inner projection 2015-06-02 16:26:49 +09:00
Scott Lystig Fritchie
cffbd3c50c Add checksum handling strawman to strawman HTTP interface 2015-06-02 13:23:36 +09:00
Scott Lystig Fritchie
dd4160b963 Add basic {error, bad_checksum} tests to proxy & CR clients 2015-06-02 12:36:51 +09:00
Scott Lystig Fritchie
e3162fdcda Rudimentary client-side checksum and server-side checksum type tags 2015-06-01 14:25:55 +09:00
Scott Lystig Fritchie
6cebf39723 Damn ugly HTTP interface "equivalent" for machi_cr_client.erl basic API 
This goes to show that mixing implementation and protocol and API
and lots of other stuff ... is cool for the quick hack to do one thing
but really sucks when trying to do more than one thing.

* Proof-of-concept only: add HTTP/1.0'ish 'PUT' interface to be the
rough equivalent of machi_cr_client:append_chunk/3
* Proof-of-concept only: add HTTP/1.0'ish 'GET' interface to be the
rough equivalent of machi_cr_client:read_chunk/4

Example use: `append_chunk`

    % curl http://127.0.0.1:4444/foo -0 -T /etc/hosts -v
    * Hostname was NOT found in DNS cache
    *   Trying 127.0.0.1...
    * Connected to 127.0.0.1 (127.0.0.1) port 4444 (#0)
    > PUT /foo HTTP/1.0
    > User-Agent: curl/7.37.1
    > Host: 127.0.0.1:4444
    > Accept: */*
    > Content-Length: 338
    >
    * We are completely uploaded and fine
    * HTTP 1.0, assume close after body
    < HTTP/1.0 201 Created
    < Location: foo.50EI18AX.21
    < X-Offset: 3052
    < X-Size: 338
    <
    * Closing connection 0

Example_use: `read_chunk`

    curl 'http://127.0.0.1:4444/foo.50EI18AX.21?offset=3052&size=338' -0 -v
    * Hostname was NOT found in DNS cache
    *   Trying 127.0.0.1...
    * Connected to 127.0.0.1 (127.0.0.1) port 4444 (#0)
    > GET /foo.50EI18AX.21?offset=3052&size=338 HTTP/1.0
    > User-Agent: curl/7.37.1
    > Host: 127.0.0.1:4444
    > Accept: */*
    >
    * HTTP 1.0, assume close after body
    < HTTP/1.0 200 OK
    < Content-Length: 338
    <
    ##
    # Host Database
    #
    # localhost is used to configure the loopback interface
    # when the system is booting.  Do not change this entry.
    ##
    127.0.0.1	localhost
    127.0.0.1	test.localhost
    255.255.255.255	broadcasthost
    ::1             localhost
    fe80::1%lo0	localhost

    # Xxxxxxx Yyyyy
    192.168.99.222	zzzzz

    127.0.0.1   aaaaaaaa.bb.ccccccccc.com
    * Closing connection 0
 2015-05-22 17:51:06 +09:00
Scott Lystig Fritchie
7ba962e9fb Fix handling of {error, partial_read} 2015-05-21 15:12:46 +09:00
Scott Lystig Fritchie
9e41162e65 Minor machi_basho_bench_driver changes 2015-05-20 18:44:04 +09:00
Scott Lystig Fritchie
1c13273efd Add simple basho_bench driver, no schmancy bells or whistles 2015-05-20 17:30:37 +09:00
Scott Lystig Fritchie
69244691f4 Such wonder when one *reads* the docs... 2015-05-20 14:12:48 +09:00
Scott Lystig Fritchie
b5ddfaf019 Finish basic API for machi_cr_client.erl 2015-05-19 20:04:36 +09:00
Scott Lystig Fritchie
a4266e8aa4 Fix known chain repair bugs, add basic smoke test 2015-05-19 19:32:48 +09:00
Scott Lystig Fritchie
152e487060 WIP: read-repair, new test is failing, yay 2015-05-19 15:15:05 +09:00
Scott Lystig Fritchie
079d15dd5c Derp, remove debugging goop + fix eunit @ write_chunk() response change 2015-05-19 14:05:18 +09:00
Scott Lystig Fritchie
eaf007ec08 Fix read repair FLU tracking 2015-05-19 13:56:12 +09:00
Scott Lystig Fritchie
f7274e7106 WIP: brute-force read-repair 2015-05-18 23:26:21 +09:00
Scott Lystig Fritchie
185c670b2f WIP: refactoring machi_cr_client:append_chunk* 2015-05-18 19:06:06 +09:00
Scott Lystig Fritchie
a347722a15 Fix {error,not_written} type bugs in chmgr 2015-05-18 17:32:22 +09:00
Scott Lystig Fritchie
966d2edae8 WIP: refactoring machi_cr_client:append_chunk* 2015-05-18 15:49:05 +09:00
Scott Lystig Fritchie
a7f53cf21a WIP: starting machi_cr_client:append_chunk* 2015-05-18 00:59:24 +09:00
Scott Lystig Fritchie
b0607ae815 WIP: starting machi_cr_client:append_chunk* 2015-05-18 00:33:15 +09:00
Scott Lystig Fritchie
d293170e92 WIP: starting machi_cr_client.erl 2015-05-17 23:48:05 +09:00
Scott Lystig Fritchie
10364834de Add a dummy client-side implementation module:machi_yessir_client.erl 2015-05-17 19:00:51 +09:00
Scott Lystig Fritchie
d4080b78d8 WIP: rearrange client code to approach some semblance of modularity someday 2015-05-17 16:46:50 +09:00
Scott Lystig Fritchie
a8c5879d21 WIP: rearrange client code to approach some semblance of modularity someday 2015-05-17 16:25:58 +09:00
Scott Lystig Fritchie
a06055ac23 WIP: rearrange client code to approach some semblance of modularity someday 2015-05-17 16:18:30 +09:00
Scott Lystig Fritchie
c7d4131a44 Remove unused verb() 2015-05-17 14:19:37 +09:00
Scott Lystig Fritchie
6c07522359 Add new API func, append_chunk_extra() 2015-05-17 14:10:42 +09:00
Scott Lystig Fritchie
5c2635346f Basic multi-party chain repair for ap_mode finished 2015-05-16 17:39:58 +09:00
Scott Lystig Fritchie
d2f1549aa3 WIP: more generic all-way file chunk merge func, part 3 2015-05-16 17:11:54 +09:00
Scott Lystig Fritchie
04bc28b9da WIP: more generic all-way file chunk merge func, part 2 2015-05-16 16:55:48 +09:00
Scott Lystig Fritchie
a9c753ad64 WIP: more generic all-way file chunk merge func 2015-05-15 17:15:02 +09:00
Scott Lystig Fritchie
358764d403 WIP: repair common file, part 0 2015-05-14 14:04:31 +09:00
Scott Lystig Fritchie
19d3c95325 WIP: aside, damn, add missing hex encoding for epochid, derp 2015-05-13 18:57:38 +09:00
Scott Lystig Fritchie
eec029b08f WIP: aside, fix FLU wedge status @ init() 2015-05-13 17:59:32 +09:00
Scott Lystig Fritchie
4ae0f94649 WIP: move to stats via ETS, success/failure propagates, yay! 2015-05-12 23:45:35 +09:00
Scott Lystig Fritchie
cad84442bb WIP: stats record, hrm 2015-05-12 22:42:03 +09:00