UENISHI Kota
3b087c0388
Add eqc trim tests to machi_file_proxy
...
* Add description on high client APIs
* Add notes to rethink high client specification
2015-11-04 16:02:29 +09:00
Scott Lystig Fritchie
44497d5af8
Merge pull request #32 from basho/ku/trim-and-gc
...
Trim command and GC prototype implementation
2015-10-29 15:07:10 +09:00
Scott Lystig Fritchie
611f33e81b
Change trigger_gc default -> 0
2015-10-29 15:05:15 +09:00
UENISHI Kota
028135d927
Update some comments for concise and sound description
2015-10-29 12:07:34 +09:00
UENISHI Kota
170b3cd797
Dialyzer fix
2015-10-28 12:48:50 +09:00
UENISHI Kota
f7358424e4
Trim command and GC prototype implementation
...
* maybe_gc/2 is triggered at machi_file_proxy, when chunk is deleted
and the file is larger than `max_file_size`
* A file is deleted if all chunks except 1024 bytes header are trimmed
* If a file is going to be deleted, file_proxy notifies metadata_mgr
to remember the filename persistently, whose filename is
`known_files_<FluName>`
* Such trimmed filenames are stored in a machi_plist file per flu
* machi_file_proxy could not be started if the filename is in the
manager's list. Consequently, any write, read and trim operations
cannot happen against deleted file.
* After the file was trimmed, any read request to the file returns
`{error, trimmed}`
* Disclaimer: no tests written yet and machi_plist does not support
any recovery from partial writes.
* Add some thoughts as comments for repairing trims.
* State diagram of every byte is as follows:
```
state\action| write/append | read_chunk | trim_chunk
------------+----------------+------------------+---------------
unwritten | -> written | fail (+repair) | -> trimmed
written | noop or repair | return content | -> trimmed
trimmed | fail | fail | noop
```
2015-10-28 12:34:03 +09:00
Scott Lystig Fritchie
61f02dfc9f
Merge pull request #30 from basho/ss-fix-opt-timeout-arg-mismatch
...
Fix missing "options" arg for a few calls
2015-10-27 14:36:52 +09:00
Scott Lystig Fritchie
5ec35773e3
Merge pull request #28 from basho/ku/csum-table-anybytes
...
Change machi_csum_table to support arbitrary bytes writes and trims
2015-10-27 14:21:09 +09:00
Scott Lystig Fritchie
d2b1c7512a
Merge branch 'ku/config-system' into tmp
2015-10-27 14:14:40 +09:00
Scott Lystig Fritchie
cfaed63fa7
Experimental: add 'make dialyzer' to priv/test-for-gh-pr.sh
2015-10-27 14:12:37 +09:00
Scott Lystig Fritchie
b500d5f449
config hell
2015-10-27 14:07:45 +09:00
Scott Lystig Fritchie
dd17b1de0a
Remove compiler warnings
2015-10-27 14:07:07 +09:00
UENISHI Kota
7377624579
Dialyzer cleanup
2015-10-27 13:44:12 +09:00
UENISHI Kota
6e3347f727
Support log replay
2015-10-27 13:43:45 +09:00
UENISHI Kota
8a61055f55
Support arbitrary bytes write by using find_(left|right)neighbor/2
2015-10-27 13:43:45 +09:00
UENISHI Kota
d59c1fae31
Fix dialyzer issue
2015-10-27 13:27:20 +09:00
Shunichi Shinohara
39ac71048b
Fix missing "options" arg for a few calls
2015-10-27 13:03:56 +09:00
Scott Lystig Fritchie
9fb19aa8ee
Merge pull request #27 from basho/ku/border-checksum
...
Regenerate checksum when chunks are to be sliced
2015-10-27 12:51:02 +09:00
Scott Lystig Fritchie
bbbd9748f3
Fix compiler & dialyzer warnings
2015-10-27 12:45:48 +09:00
UENISHI Kota
b2eb3e089c
Cleanup MACROs and changed default value
...
* machi_file_proxy now uses application environment
value `max_file_size` via machi_config
* changed name from MAX_FILE_SIZE to DEFAULT_MAX_FILE_SIZE
2015-10-27 11:17:59 +09:00
UENISHI Kota
5913531e32
Introduce machi_config.erl
2015-10-27 11:00:05 +09:00
UENISHI Kota
60364fe0ca
Fix typo
2015-10-26 10:10:08 +09:00
UENISHI Kota
0e4ae818af
Clarify checksum tags, define macros to avoid typos
2015-10-23 19:32:22 +09:00
UENISHI Kota
3d6d4d8be3
Do the slicing in flu server rather than in CR client
2015-10-23 18:49:49 +09:00
UENISHI Kota
c5661571e3
Regenerate checksum when chunks are to be sliced
2015-10-23 17:33:08 +09:00
UENISHI Kota
5fd225bcdb
Merge pull request #25 from basho/ku/trim-pb-protocol-2
...
Update read_chunk() PB protocol to return trimmed chunks
2015-10-23 17:10:21 +09:00
UENISHI Kota
0f688d6279
Update read_chunk() PB protocol to return trimmed chunks
2015-10-22 23:11:43 +09:00
Scott Lystig Fritchie
41bd8fa64e
Merge pull request #24 from basho/ku/tools-mk
...
Replace some make targets with tools.mk
2015-10-22 15:47:05 +09:00
Scott Lystig Fritchie
51c97da133
Remove now-unused filter-dialyzer-dep-warnings
2015-10-22 15:39:41 +09:00
Scott Lystig Fritchie
49b4b1c304
Silence remaining warnings
2015-10-22 15:37:09 +09:00
Scott Lystig Fritchie
058de6dc9c
Reduce all dialyzer runtimes by approx 13 seconds (on my MacBook Pro)
2015-10-22 15:36:50 +09:00
Scott Lystig Fritchie
a0588cbaed
Avoid warnings
2015-10-22 12:59:40 +09:00
Scott Lystig Fritchie
3751ca14b5
dialyzer.ignore-warnings .gitignore
2015-10-22 12:59:19 +09:00
Scott Lystig Fritchie
3bb5ffa50e
Remove src/machi_sequencer.erl
2015-10-22 12:56:24 +09:00
UENISHI Kota
3d3d26b3af
Replace some make targets with tools.mk
2015-10-22 10:53:09 +09:00
Scott Lystig Fritchie
224a293c03
Merge pull request #23 from basho/slf/dialyzer1
...
Dialyzer cleanup
2015-10-22 10:32:01 +09:00
Scott Lystig Fritchie
8a51230760
I don't understand riak_dt type problem, but machi_fitness.erl works in common case
2015-10-21 18:37:31 +09:00
Scott Lystig Fritchie
b8c5d21876
cp_mode repair warning fix
2015-10-21 18:37:31 +09:00
Scott Lystig Fritchie
5008cbd2d0
Add filter for known pattern in machi_chain_manger1.erl
2015-10-21 18:37:31 +09:00
Scott Lystig Fritchie
9c31139b62
Whitespace
2015-10-21 18:37:31 +09:00
Scott Lystig Fritchie
b3b24b1178
Type corrections & remove dead code
2015-10-21 18:37:31 +09:00
Scott Lystig Fritchie
5122ee00a9
Bugfixes
2015-10-21 18:37:30 +09:00
Scott Lystig Fritchie
0c4c42cc52
Bugfixes
2015-10-21 18:37:30 +09:00
Scott Lystig Fritchie
d2ac5b0583
Bugfix: arg type to machi_util:parse_filename()
2015-10-21 18:37:30 +09:00
Scott Lystig Fritchie
028ddc79ff
Data type cleanups, other
2015-10-21 18:37:30 +09:00
Scott Lystig Fritchie
edce9d6463
Remove -Wunderspecs, the pickiness level is too high IMO for maturity level now
2015-10-21 18:37:30 +09:00
Scott Lystig Fritchie
bd91167093
Changes to filter & make target
2015-10-21 18:37:30 +09:00
Scott Lystig Fritchie
595f9a463e
Unexported funcs
2015-10-21 18:37:30 +09:00
Scott Lystig Fritchie
177aca0a68
Merge pull request #22 from basho/ss-flu1-init-sync
...
Make flu1 initialization synchronous
2015-10-21 18:36:12 +09:00
Shunichi Shinohara
478107915b
Make flu1 initialization synchronous
2015-10-21 16:16:03 +09:00
