Scott Lystig Fritchie
0321e05b46
Fix usage of checksum_list by machi_basho_bench_driver.erl
2015-07-01 15:03:56 +09:00
Scott Lystig Fritchie
f5ae417b9e
Clarify verify_file_checksums_test_
2015-07-01 14:16:31 +09:00
Scott Lystig Fritchie
670bd2cafc
Add some flexibility to machi_chain_manager1_converge_demo:t/1 and t/2
2015-07-01 14:08:17 +09:00
Scott Lystig Fritchie
e3b80c6ac2
Docuemntation updates
2015-06-30 19:04:23 +09:00
Scott Lystig Fritchie
00c8cf0ef7
Rename temporary HTTP server hack functions
2015-06-30 16:19:44 +09:00
Scott Lystig Fritchie
7542fe8225
WIP: all eunit tests are passing again, yay
2015-06-30 16:12:23 +09:00
Scott Lystig Fritchie
e9d50a2128
WIP: Reinstate one eunit test, fix type bugs
2015-06-30 15:51:03 +09:00
Scott Lystig Fritchie
3d2b49b7e5
WIP: refactoring & edoc'ing
2015-06-30 15:20:35 +09:00
Scott Lystig Fritchie
310fdb1f6a
Add crude file size check to do_server_checksum_listing()
2015-06-30 14:13:26 +09:00
Scott Lystig Fritchie
2d070bf1e3
Minor refactoring + add demo/exploratory time measurement code
...
%% Demo/exploratory hackery to check relative speeds of dealing with
%% checksum data in different ways.
%%
%% Summary:
%%
%% * Use compact binary encoding, with 1 byte header for entry length.
%% * Because the hex-style code is *far* slower just for enc & dec ops.
%% * For 1M entries of enc+dec: 0.215 sec vs. 15.5 sec.
%% * File sorter when sorting binaries as-is is only 30-40% slower
%% than an in-memory split (of huge binary emulated by file:read_file()
%% "big slurp") and sort of the same as-is sortable binaries.
%% * File sorter slows by a factor of about 2.5 if {order, fun compare/2}
%% function must be used, i.e. because the checksum entry lengths differ.
%% * File sorter + {order, fun compare/2} is still *far* faster than external
%% sort by OS X's sort(1) of sortable ASCII hex-style:
%% 4.5 sec vs. 21 sec.
%% * File sorter {order, fun compare/2} is faster than in-memory sort
%% of order-friendly 3-tuple-style: 4.5 sec vs. 15 sec.
2015-06-30 14:08:46 +09:00
Scott Lystig Fritchie
2a4ae1ba52
Merge branch 'slf/pb-api-experiment2'
2015-06-29 17:31:52 +09:00
Scott Lystig Fritchie
34b046acbd
Remove machi_pb_wrap.erl
2015-06-29 17:31:07 +09:00
Scott Lystig Fritchie
55db22efff
Merge branch 'slf/pb-api-experiment2'
2015-06-29 17:20:35 +09:00
Scott Lystig Fritchie
dba7041929
Change names to indicate we're no longer in PB land
2015-06-29 17:20:17 +09:00
Scott Lystig Fritchie
151e696324
WIP: yank out more unused cruft
2015-06-29 17:14:33 +09:00
Scott Lystig Fritchie
87ec988353
WIP: yank out more unused cruft
2015-06-29 17:06:28 +09:00
Scott Lystig Fritchie
6cd3b8d0ec
WIP: yank out lots of unused cruft
2015-06-29 17:02:58 +09:00
Scott Lystig Fritchie
d54c74f58a
WIP: yank out io:format
2015-06-29 16:53:41 +09:00
Scott Lystig Fritchie
3089288338
WIP: giant hairball 13: all unit tests are passing again, yay!
2015-06-29 16:48:06 +09:00
Scott Lystig Fritchie
7aff9fca70
WIP: giant hairball 12
2015-06-29 16:42:05 +09:00
Scott Lystig Fritchie
b25ab3b7ac
WIP: giant hairball 11
2015-06-29 16:24:57 +09:00
Scott Lystig Fritchie
64817dd7e8
WIP: giant hairball 01
2015-06-29 16:10:43 +09:00
Scott Lystig Fritchie
f45dc7829e
WIP: hairball, but: Failed: 6. Skipped: 0. Passed: 13
2015-06-27 00:43:27 +09:00
Scott Lystig Fritchie
3cf18817df
WIP: hairball, but timing_pb_encoding_test() works!
2015-06-27 00:12:42 +09:00
Scott Lystig Fritchie
b5c824c5c0
WIP: hairball, but bad_checksum_test() works!
2015-06-27 00:06:21 +09:00
Scott Lystig Fritchie
2fd27fdae6
WIP: hairball, but flu_projection_smoke_test() works!
2015-06-26 23:58:34 +09:00
Scott Lystig Fritchie
93f64a20c0
WIP: hairball, but flu_smoke_test() works!
2015-06-26 23:03:28 +09:00
Scott Lystig Fritchie
920a5c33d7
WIP: giant hairball 6
2015-06-26 22:32:53 +09:00
Scott Lystig Fritchie
77b4da16c3
WIP: giant hairball 5
2015-06-26 21:36:07 +09:00
Scott Lystig Fritchie
9a212fb19f
WIP: giant hairball 4
2015-06-26 20:47:55 +09:00
Scott Lystig Fritchie
0e32fd25c9
WIP: giant hairball 3
2015-06-26 18:59:07 +09:00
Scott Lystig Fritchie
8437d76c1c
WIP: giant hairball 2
2015-06-26 18:22:15 +09:00
Scott Lystig Fritchie
fb975eea46
WIP: giant hairball
2015-06-26 16:58:24 +09:00
Scott Lystig Fritchie
6d95d8669c
WIP: giant hairball, bleh, low-level checksum_list() barely working
2015-06-26 16:25:12 +09:00
Scott Lystig Fritchie
90efc41167
machi.proto definition for low-level protocol ops
2015-06-25 17:09:33 +09:00
Scott Lystig Fritchie
cf0d9a25b4
EDoc cleanup
2015-06-25 16:39:19 +09:00
Scott Lystig Fritchie
0b2b79cd0b
Merge branch 'slf/pb-api-experiment1'
2015-06-25 16:36:50 +09:00
Scott Lystig Fritchie
0f4d5ed775
Silence dialyzer unused function clause
2015-06-25 16:36:29 +09:00
Scott Lystig Fritchie
c2faf9f499
yolo, un-do experimental type hack
2015-06-25 16:36:14 +09:00
Scott Lystig Fritchie
d9694a992a
Alright, use term_to_binary() for opaque/sexp-style encoding, only 15x slower.
...
machi_flu1_test: timing_pb_encoding_test_... speed factor=15.12 [2.678 s] ok
2015-06-25 16:11:46 +09:00
Scott Lystig Fritchie
2763b16ca2
timing_pb_encoding_test_... speed factor=35.95 [2.730 s] ok
...
So, the PB style encoding of the Mpb_LL_WriteProjectionReq message
is about 35-36 times slower than using Erlang's term_to_binary()
and binary_to_term(). {sigh}
2015-06-25 16:11:44 +09:00
Scott Lystig Fritchie
5d8b648a24
All projection store protocol operations are now using Protocol Buffers!
...
So, there's some cheating going on, because some of the parts of
the #projection_v1{} and #p_srvr{} records aren't fully specified.
Those parts are being specified as "opaque" in the field names, e.g.
optional bytes opaque_flap = 10;
optional bytes opaque_inner = 11;
required bytes opaque_dbg = 12;
required bytes opaque_dbg2 = 13;
The serialization that's being used is erlang term sexprs. That isn't
portable. So if/when we really need to deal with a non-Erlang
language, we'll have to straighten this out further.
2015-06-25 15:26:35 +09:00
Scott Lystig Fritchie
841235b3b5
WIP: bugfixes, add {error, written}
2015-06-25 15:10:24 +09:00
Scott Lystig Fritchie
4fc0578a9d
WIP: bugfixes, machi_flu1_test still broken
2015-06-25 15:08:40 +09:00
Scott Lystig Fritchie
d9407b76b7
WIP: dinnertime, machi_flu1_test still broken
2015-06-24 18:00:25 +09:00
Scott Lystig Fritchie
31c5bcc0c7
WIP: 1/2 of low-level projection proto finished, machi_flu1_test fails
2015-06-24 17:20:18 +09:00
Scott Lystig Fritchie
725b10ba90
Complete PB round-trip for #projection_v1{}, bleh
2015-06-24 16:13:11 +09:00
Scott Lystig Fritchie
1b0cf06f1c
Fix type problem, oops
2015-06-24 14:06:17 +09:00
Scott Lystig Fritchie
2068f70700
WIP: encoding #p_srvr and #projection_v1, just starting. Damn tedious.
2015-06-24 12:50:37 +09:00
Scott Lystig Fritchie
817efb2b15
machi_pb_high_client: always be checksumming
2015-06-23 17:37:47 +09:00
