greg/machi
1
0
Fork 0
Code Issues 11 Pull requests 1 Projects Releases Packages Wiki Activity Actions
1114 commits 58 branches 0 tags 5.9 MiB
Commit graph

1114 commits

This branch
This branch
All branches
Author SHA1 Message Date
Shunichi Shinohara
059f591d3f Exclude FLUs which can not send to cr client as well as receive from 
in terms of partition simulator
 2015-11-02 17:57:16 +09:00
UENISHI Kota
3122f2cf54 Support OTP 18.1 2015-11-02 17:43:38 +09:00
Mark Allen
3c5a9e6f53 Torture tests for merkle tree 
1,000,000 entries - timings and size
 2015-11-02 00:12:58 -06:00
Mark Allen
72a4fab49d Add a naive diff function 2015-10-29 22:18:20 -05:00
Shunichi Shinohara
b5005c3526 Add EQC test case for AP mode repair w/ part. sim. 2015-10-30 10:25:51 +09:00
Shunichi Shinohara
6fa2de28cd Add self pid to debug print 2015-10-30 09:44:54 +09:00
Shunichi Shinohara
93b168415d Change log level to debug for multiple files with certain seq num 2015-10-30 09:40:07 +09:00
Shunichi Shinohara
bf5768eb47 Make CR client partition-simulator-aware 2015-10-30 09:39:21 +09:00
Shunichi Shinohara
447c8c8d48 Add 2-tuple timeout setting to CR client 2015-10-30 09:39:21 +09:00
Scott Lystig Fritchie
b859e23a37 Some long-overdue minor editing, prior to working on issue #4 2015-10-29 21:21:05 +09:00
Scott Lystig Fritchie
44497d5af8 Merge pull request #32 from basho/ku/trim-and-gc 
Trim command and GC prototype implementation
 2015-10-29 15:07:10 +09:00
Scott Lystig Fritchie
611f33e81b Change trigger_gc default -> 0 2015-10-29 15:05:15 +09:00
UENISHI Kota
028135d927 Update some comments for concise and sound description 2015-10-29 12:07:34 +09:00
Mark Allen
7086899941 Reorg merkle tree code into a library 
Was a service previously. Now contains both merklet
and the naive implementations.  Put construction
timing stuff into the test.

Tests are not truly meaningful yet.
 2015-10-28 16:59:49 -05:00
UENISHI Kota
170b3cd797 Dialyzer fix 2015-10-28 12:48:50 +09:00
UENISHI Kota
f7358424e4 Trim command and GC prototype implementation 
* maybe_gc/2 is triggered at machi_file_proxy, when chunk is deleted
  and the file is larger than `max_file_size`
* A file is deleted if all chunks except 1024 bytes header are trimmed
* If a file is going to be deleted, file_proxy notifies metadata_mgr
  to remember the filename persistently, whose filename is
  `known_files_<FluName>`
* Such trimmed filenames are stored in a machi_plist file per flu
* machi_file_proxy could not be started if the filename is in the
  manager's list. Consequently, any write, read and trim operations
  cannot happen against deleted file.
* After the file was trimmed, any read request to the file returns
  `{error, trimmed}`
* Disclaimer: no tests written yet and machi_plist does not support
  any recovery from partial writes.
* Add some thoughts as comments for repairing trims.

* State diagram of every byte is as follows:

```
state\action| write/append   | read_chunk       | trim_chunk
------------+----------------+------------------+---------------
 unwritten  |  -> written    | fail (+repair)   | -> trimmed
 written    | noop or repair | return content   | -> trimmed
 trimmed    |  fail          | fail             | noop
```
 2015-10-28 12:34:03 +09:00
Mark Allen
5e571f6009 Switch to merklet 
Still a WIP
 2015-10-27 16:33:18 -05:00
Mark Allen
7f561f34e0 Ignore vim cruft too 2015-10-27 16:33:07 -05:00
Mark Allen
77096c5f82 Add merklet as a dependency 2015-10-27 11:57:38 -05:00
Mark Allen
b710517c64 Fixes after testing 2015-10-27 11:57:38 -05:00
Mark Allen
1b8401e7de Initial smoke test 2015-10-27 11:57:38 -05:00
Mark Allen
48fabdcd5f WIP 
Almost certainly broken
 2015-10-27 11:57:38 -05:00
Scott Lystig Fritchie
61f02dfc9f Merge pull request #30 from basho/ss-fix-opt-timeout-arg-mismatch 
Fix missing "options" arg for a few calls
 2015-10-27 14:36:52 +09:00
Scott Lystig Fritchie
5ec35773e3 Merge pull request #28 from basho/ku/csum-table-anybytes 
Change machi_csum_table to support arbitrary bytes writes and trims
 2015-10-27 14:21:09 +09:00
Scott Lystig Fritchie
d2b1c7512a Merge branch 'ku/config-system' into tmp 2015-10-27 14:14:40 +09:00
Scott Lystig Fritchie
cfaed63fa7 Experimental: add 'make dialyzer' to priv/test-for-gh-pr.sh 2015-10-27 14:12:37 +09:00
Scott Lystig Fritchie
b500d5f449 config hell 2015-10-27 14:07:45 +09:00
Scott Lystig Fritchie
dd17b1de0a Remove compiler warnings 2015-10-27 14:07:07 +09:00
UENISHI Kota
7377624579 Dialyzer cleanup 2015-10-27 13:44:12 +09:00
UENISHI Kota
6e3347f727 Support log replay 2015-10-27 13:43:45 +09:00
UENISHI Kota
8a61055f55 Support arbitrary bytes write by using find_(left|right)neighbor/2 2015-10-27 13:43:45 +09:00
UENISHI Kota
d59c1fae31 Fix dialyzer issue 2015-10-27 13:27:20 +09:00
Shunichi Shinohara
39ac71048b Fix missing "options" arg for a few calls 2015-10-27 13:03:56 +09:00
Scott Lystig Fritchie
9fb19aa8ee Merge pull request #27 from basho/ku/border-checksum 
Regenerate checksum when chunks are to be sliced
 2015-10-27 12:51:02 +09:00
Scott Lystig Fritchie
bbbd9748f3 Fix compiler & dialyzer warnings 2015-10-27 12:45:48 +09:00
UENISHI Kota
b2eb3e089c Cleanup MACROs and changed default value 
* machi_file_proxy now uses application environment
  value `max_file_size` via machi_config
* changed name from MAX_FILE_SIZE to DEFAULT_MAX_FILE_SIZE
 2015-10-27 11:17:59 +09:00
UENISHI Kota
5913531e32 Introduce machi_config.erl 2015-10-27 11:00:05 +09:00
UENISHI Kota
60364fe0ca Fix typo 2015-10-26 10:10:08 +09:00
UENISHI Kota
0e4ae818af Clarify checksum tags, define macros to avoid typos 2015-10-23 19:32:22 +09:00
UENISHI Kota
3d6d4d8be3 Do the slicing in flu server rather than in CR client 2015-10-23 18:49:49 +09:00
UENISHI Kota
c5661571e3 Regenerate checksum when chunks are to be sliced 2015-10-23 17:33:08 +09:00
UENISHI Kota
5fd225bcdb Merge pull request #25 from basho/ku/trim-pb-protocol-2 
Update read_chunk() PB protocol to return trimmed chunks
 2015-10-23 17:10:21 +09:00
UENISHI Kota
0f688d6279 Update read_chunk() PB protocol to return trimmed chunks 2015-10-22 23:11:43 +09:00
Scott Lystig Fritchie
41bd8fa64e Merge pull request #24 from basho/ku/tools-mk 
Replace some make targets with tools.mk
 2015-10-22 15:47:05 +09:00
Scott Lystig Fritchie
51c97da133 Remove now-unused filter-dialyzer-dep-warnings 2015-10-22 15:39:41 +09:00
Scott Lystig Fritchie
49b4b1c304 Silence remaining warnings 2015-10-22 15:37:09 +09:00
Scott Lystig Fritchie
058de6dc9c Reduce all dialyzer runtimes by approx 13 seconds (on my MacBook Pro) 2015-10-22 15:36:50 +09:00
Scott Lystig Fritchie
a0588cbaed Avoid warnings 2015-10-22 12:59:40 +09:00
Scott Lystig Fritchie
3751ca14b5 dialyzer.ignore-warnings .gitignore 2015-10-22 12:59:19 +09:00
Scott Lystig Fritchie
3bb5ffa50e Remove src/machi_sequencer.erl 2015-10-22 12:56:24 +09:00