Commit graph

925 commits

Author SHA1 Message Date
Scott Lystig Fritchie
f741a8a413 Avoid io:format() calls that can consume huge amounts of time
Example CLI session:

    f(Leaves).
    Leaves = [{{Pos*65,65}, begin {A,B,C} = now(), crypto:hash(sha, <<A:32, B:32, C:32>>) end} || Pos <- lists:seq(1,1*1000*1000)].
    timer:tc(fun() -> catch machi_merkle_tree_mgr:build_tree(x, {mt, "/tmp/foo", true, undefined, [], [], [], Leaves}) end).

On my MPB, I see about 550000 - 650000 msec, depending on the run. I've
made no attempt yet to be more rigorous about making measurements.

A result looks like this.  If we ignore the timer:tc() tuple wrapper and
time report, then the output list is [Root, Lvl3, Lvl2, Lvl1].

    Root: <<250,83,19,148,149,195,105,253,231,116,144,49,227,168,224,51,76,63,162,
            70>>
    {551957,
     [<<250,83,19,148,149,195,105,253,231,116,144,49,227,168,
        224,51,76,63,162,70>>,
      [<<102,159,171,226,30,76,65,38,81,168,229,143,240,112,
         208,13,169,216,57,35>>],
      [<<67,189,3,58,14,201,74,206,240,202,163,83,182,232,130,
         14,107,29,84,181>>],
      [<<210,57,128,48,28,28,87,185,240,55,129,158,119,77,172,0,
         183,166,205,157>>,
       <<78,154,155,117,1,220,48,116,74,66,172,146,29,39,7,7,247,
         188,142,72>>,
       <<4,50,223,192,38,141,54,249,168,79,213,90,94,32,91,117,
         36,186,41,139>>,
       <<20,181,105,237,50,97,98,91,178,26,127,20,134,174,146,96,
         133,208,245,...>>,
       <<227,25,30,162,182,226,172,29,132,240,125,248,65,109,82,
         6,116,102,...>>,
       <<229,221,31,166,106,212,120,146,96,229,224,85,237,206,
         160,38,137,...>>,
       <<124,195,225,167,238,245,75,161,71,53,43,154,103,188,
         180,171,...>>]]}
2015-10-29 12:58:03 +09:00
Mark Allen
4480f47ee6 Fixes after testing 2015-10-23 22:59:40 -05:00
Mark Allen
cdabe33753 Initial smoke test 2015-10-23 22:59:23 -05:00
Mark Allen
14b2b54c41 WIP
Almost certainly broken
2015-10-22 21:17:34 -05:00
Scott Lystig Fritchie
41bd8fa64e Merge pull request #24 from basho/ku/tools-mk
Replace some make targets with tools.mk
2015-10-22 15:47:05 +09:00
Scott Lystig Fritchie
51c97da133 Remove now-unused filter-dialyzer-dep-warnings 2015-10-22 15:39:41 +09:00
Scott Lystig Fritchie
49b4b1c304 Silence remaining warnings 2015-10-22 15:37:09 +09:00
Scott Lystig Fritchie
058de6dc9c Reduce all dialyzer runtimes by approx 13 seconds (on my MacBook Pro) 2015-10-22 15:36:50 +09:00
Scott Lystig Fritchie
a0588cbaed Avoid warnings 2015-10-22 12:59:40 +09:00
Scott Lystig Fritchie
3751ca14b5 dialyzer.ignore-warnings .gitignore 2015-10-22 12:59:19 +09:00
Scott Lystig Fritchie
3bb5ffa50e Remove src/machi_sequencer.erl 2015-10-22 12:56:24 +09:00
UENISHI Kota
3d3d26b3af Replace some make targets with tools.mk 2015-10-22 10:53:09 +09:00
Scott Lystig Fritchie
224a293c03 Merge pull request #23 from basho/slf/dialyzer1
Dialyzer cleanup
2015-10-22 10:32:01 +09:00
Scott Lystig Fritchie
8a51230760 I don't understand riak_dt type problem, but machi_fitness.erl works in common case 2015-10-21 18:37:31 +09:00
Scott Lystig Fritchie
b8c5d21876 cp_mode repair warning fix 2015-10-21 18:37:31 +09:00
Scott Lystig Fritchie
5008cbd2d0 Add filter for known pattern in machi_chain_manger1.erl 2015-10-21 18:37:31 +09:00
Scott Lystig Fritchie
9c31139b62 Whitespace 2015-10-21 18:37:31 +09:00
Scott Lystig Fritchie
b3b24b1178 Type corrections & remove dead code 2015-10-21 18:37:31 +09:00
Scott Lystig Fritchie
5122ee00a9 Bugfixes 2015-10-21 18:37:30 +09:00
Scott Lystig Fritchie
0c4c42cc52 Bugfixes 2015-10-21 18:37:30 +09:00
Scott Lystig Fritchie
d2ac5b0583 Bugfix: arg type to machi_util:parse_filename() 2015-10-21 18:37:30 +09:00
Scott Lystig Fritchie
028ddc79ff Data type cleanups, other 2015-10-21 18:37:30 +09:00
Scott Lystig Fritchie
edce9d6463 Remove -Wunderspecs, the pickiness level is too high IMO for maturity level now 2015-10-21 18:37:30 +09:00
Scott Lystig Fritchie
bd91167093 Changes to filter & make target 2015-10-21 18:37:30 +09:00
Scott Lystig Fritchie
595f9a463e Unexported funcs 2015-10-21 18:37:30 +09:00
Scott Lystig Fritchie
177aca0a68 Merge pull request #22 from basho/ss-flu1-init-sync
Make flu1 initialization synchronous
2015-10-21 18:36:12 +09:00
Shunichi Shinohara
478107915b Make flu1 initialization synchronous 2015-10-21 16:16:03 +09:00
Scott Lystig Fritchie
30d7e592a3 Merge pull request #20 from basho/ku/read-all-chunks
Allow reading multiple chunks at once
2015-10-21 15:28:10 +09:00
UENISHI Kota
79e0ae9fe5 Merge pull request #21 from basho/slf/eunit-fixes1
Slf/eunit fixes1
2015-10-21 15:08:08 +09:00
Scott Lystig Fritchie
1c8e436a64 Fix race #3 2015-10-21 15:01:11 +09:00
Scott Lystig Fritchie
9d177c6b54 Fix race #2 2015-10-21 14:45:21 +09:00
Scott Lystig Fritchie
976a701e0c Fix timeout problem in test/machi_proxy_flu1_client_test.erl 2015-10-21 14:31:58 +09:00
Scott Lystig Fritchie
981b55c070 Fix race #1 2015-10-21 14:31:41 +09:00
UENISHI Kota
a43397a7b8 Update to review comments 2015-10-21 10:58:00 +09:00
Scott Lystig Fritchie
84f9ccc4f5 markdown fix 2015-10-21 10:44:52 +09:00
UENISHI Kota
ebb9bc3f5a Allow reading multiple chunks at once
* When repairing multiple chunks at once and any of its repair
  failed, the whole read request and repair work will fail
* Rename read_repair3 and read_repair4 to do_repair_chunks and
  do_repair chunk in machi_file_proxy
* This pull request changes return semantics of read_chunk(), that
  returns any chunk included in requested range
* First and last chunk may be cut to fit the requested range
* In machi_file_proxy, unwritten_bytes are removed and replaced by
  machi_csum_table
2015-10-20 17:59:09 +09:00
Scott Lystig Fritchie
a369f0666a Update README.me for mid-Oct 2015 2015-10-20 14:50:54 +09:00
Scott Lystig Fritchie
1193fb8510 Merge branch 'slf/doc-name-game' 2015-10-19 16:46:17 +09:00
Scott Lystig Fritchie
6f9814ffb4 Merge ss/deps-for-debugging (with rebar.config conflict fix) 2015-10-19 16:41:03 +09:00
Scott Lystig Fritchie
bf61b2900b Merge branch 'master' of github.com:basho/machi 2015-10-19 16:38:10 +09:00
Scott Lystig Fritchie
ecd7eb195a Merge pull request #18 from basho/ku/read-all-valid-chunks
Allow read_chunk() to return partial chunks
2015-10-19 16:25:43 +09:00
Scott Lystig Fritchie
980d646d64 Merge branch 'ku/read-all-valid-chunks' 2015-10-19 16:24:12 +09:00
UENISHI Kota
3e975f53b8 Allow read_chunk() to return partial chunks
This is simply a change of read_chunk() protocol, where a response of
read_chunk() becomes list of written bytes along with checksum. All
related code including repair is changed as such. This is to pass all
tests and not actually supporting partial chunks.
2015-10-19 15:37:17 +09:00
Shunichi Shinohara
208c02853f Add cluster_info to deps and small callback module
For debuging from shell, some functions in machi_cinfo are exported:

- public_projection/1
- private_projection/1
- fitness/1
- chain_manager/1
- flu1/1
2015-10-19 15:36:05 +09:00
Scott Lystig Fritchie
c407ee23f2 Add section '9. Other considerations for FLU/sequencer implementations' 2015-10-19 14:47:13 +09:00
Scott Lystig Fritchie
96c037e799 Notation change 2015-10-19 12:49:33 +09:00
UENISHI Kota
6961930b0f Update eper 2015-10-19 09:40:05 +09:00
Scott Lystig Fritchie
8ab9d3ee50 Name consistency: CoC 'locator', variable name = L 2015-10-17 16:47:00 +09:00
Scott Lystig Fritchie
1485fa416f Clarifications... 2015-10-17 14:28:16 +09:00
Scott Lystig Fritchie
39774bc70f Simplify (I hope!), add CoC namespace 2015-10-17 14:14:27 +09:00