Shunichi Shinohara
0e21581e47
Temporary dirty fix for filename reuse bug for AP-mode
...
Sequence numbers are only unique in the scope of single FLU, but not
unique (nor monotonic) in the scope of full chain. Additionally,
monotinicity in the scope of single FLU is good property for reusing
appendable files, simple randomness does not work.
Maybe, strict monotonicity is not necessary if most recent file names
for each prefix are tracked or trackable on demand.
This fix adds FLU name just before sequence number part.
Biggest downside is that it prevents file reuse to append by different
FLUs in CP-mode. Minor poor point is it does not work for FLU name like
`a-1'.
2015-11-05 23:42:53 +09:00
UENISHI Kota
6786820401
Merge pull request #35 from basho/ku/making-file-proxy-spec
...
Add eqc trim tests to machi_file_proxy
2015-11-05 16:27:48 +09:00
UENISHI Kota
81fae32539
Remove unused test function
2015-11-05 16:19:46 +09:00
UENISHI Kota
2e6e6dd9e8
Merge pull request #40 from basho/ss/missing-test-cleanups
...
Add missing test cleanups
2015-11-05 16:16:40 +09:00
UENISHI Kota
ce41f9005e
Fix machi_file_proxy_eqc:write_post to proper assertion
2015-11-05 14:48:35 +09:00
Shunichi Shinohara
922baaf433
Make rebar unit output verbose
2015-11-05 11:55:47 +09:00
Shunichi Shinohara
9e4dc83f2a
Add missing cleanup tasks, suppress some not-so-useful logs
2015-11-05 11:47:47 +09:00
Shunichi Shinohara
1b0711f151
Stop flu1 client under CR client
2015-11-05 11:46:22 +09:00
Shunichi Shinohara
39a937db9b
Merge pull request #38 from basho/ku/otp-18
...
Support OTP 18.1
2015-11-05 10:49:56 +09:00
UENISHI Kota
f56037240e
Plan trim commands in eqc tests
2015-11-04 16:32:53 +09:00
UENISHI Kota
d0e6417f5d
Remove unnecessary output
2015-11-04 16:15:33 +09:00
UENISHI Kota
3f6f9e2c6b
Address one offset+length issue
2015-11-04 16:08:43 +09:00
UENISHI Kota
c1e5426034
Address PR comments
2015-11-04 16:08:09 +09:00
UENISHI Kota
3b087c0388
Add eqc trim tests to machi_file_proxy
...
* Add description on high client APIs
* Add notes to rethink high client specification
2015-11-04 16:02:29 +09:00
UENISHI Kota
62c8dacc65
Merge pull request #33 from basho/ss-repair-with-partition-simulator
...
Add test for append and repair with partition simulator
2015-11-04 14:44:04 +09:00
UENISHI Kota
b956f9e1f0
Fix dialyzer issue
2015-11-04 11:43:00 +09:00
Scott Lystig Fritchie
850a8786b6
Merge branch 'slf/doc-converge-demo'
...
Docs only.
2015-11-03 00:37:53 +09:00
Scott Lystig Fritchie
557525af05
Clarify chain length
2015-11-03 00:34:51 +09:00
Scott Lystig Fritchie
30000d6602
Add doc/machi_chain_manager1_converge_demo.md
2015-11-03 00:27:09 +09:00
Shunichi Shinohara
059f591d3f
Exclude FLUs which can not send to cr client as well as receive from
...
in terms of partition simulator
2015-11-02 17:57:16 +09:00
UENISHI Kota
3122f2cf54
Support OTP 18.1
2015-11-02 17:43:38 +09:00
Shunichi Shinohara
b5005c3526
Add EQC test case for AP mode repair w/ part. sim.
2015-10-30 10:25:51 +09:00
Shunichi Shinohara
6fa2de28cd
Add self pid to debug print
2015-10-30 09:44:54 +09:00
Shunichi Shinohara
93b168415d
Change log level to debug for multiple files with certain seq num
2015-10-30 09:40:07 +09:00
Shunichi Shinohara
bf5768eb47
Make CR client partition-simulator-aware
2015-10-30 09:39:21 +09:00
Shunichi Shinohara
447c8c8d48
Add 2-tuple timeout setting to CR client
2015-10-30 09:39:21 +09:00
Scott Lystig Fritchie
44497d5af8
Merge pull request #32 from basho/ku/trim-and-gc
...
Trim command and GC prototype implementation
2015-10-29 15:07:10 +09:00
Scott Lystig Fritchie
611f33e81b
Change trigger_gc default -> 0
2015-10-29 15:05:15 +09:00
UENISHI Kota
028135d927
Update some comments for concise and sound description
2015-10-29 12:07:34 +09:00
UENISHI Kota
170b3cd797
Dialyzer fix
2015-10-28 12:48:50 +09:00
UENISHI Kota
f7358424e4
Trim command and GC prototype implementation
...
* maybe_gc/2 is triggered at machi_file_proxy, when chunk is deleted
and the file is larger than `max_file_size`
* A file is deleted if all chunks except 1024 bytes header are trimmed
* If a file is going to be deleted, file_proxy notifies metadata_mgr
to remember the filename persistently, whose filename is
`known_files_<FluName>`
* Such trimmed filenames are stored in a machi_plist file per flu
* machi_file_proxy could not be started if the filename is in the
manager's list. Consequently, any write, read and trim operations
cannot happen against deleted file.
* After the file was trimmed, any read request to the file returns
`{error, trimmed}`
* Disclaimer: no tests written yet and machi_plist does not support
any recovery from partial writes.
* Add some thoughts as comments for repairing trims.
* State diagram of every byte is as follows:
```
state\action| write/append | read_chunk | trim_chunk
------------+----------------+------------------+---------------
unwritten | -> written | fail (+repair) | -> trimmed
written | noop or repair | return content | -> trimmed
trimmed | fail | fail | noop
```
2015-10-28 12:34:03 +09:00
Scott Lystig Fritchie
61f02dfc9f
Merge pull request #30 from basho/ss-fix-opt-timeout-arg-mismatch
...
Fix missing "options" arg for a few calls
2015-10-27 14:36:52 +09:00
Scott Lystig Fritchie
5ec35773e3
Merge pull request #28 from basho/ku/csum-table-anybytes
...
Change machi_csum_table to support arbitrary bytes writes and trims
2015-10-27 14:21:09 +09:00
Scott Lystig Fritchie
d2b1c7512a
Merge branch 'ku/config-system' into tmp
2015-10-27 14:14:40 +09:00
Scott Lystig Fritchie
cfaed63fa7
Experimental: add 'make dialyzer' to priv/test-for-gh-pr.sh
2015-10-27 14:12:37 +09:00
Scott Lystig Fritchie
b500d5f449
config hell
2015-10-27 14:07:45 +09:00
Scott Lystig Fritchie
dd17b1de0a
Remove compiler warnings
2015-10-27 14:07:07 +09:00
UENISHI Kota
7377624579
Dialyzer cleanup
2015-10-27 13:44:12 +09:00
UENISHI Kota
6e3347f727
Support log replay
2015-10-27 13:43:45 +09:00
UENISHI Kota
8a61055f55
Support arbitrary bytes write by using find_(left|right)neighbor/2
2015-10-27 13:43:45 +09:00
UENISHI Kota
d59c1fae31
Fix dialyzer issue
2015-10-27 13:27:20 +09:00
Shunichi Shinohara
39ac71048b
Fix missing "options" arg for a few calls
2015-10-27 13:03:56 +09:00
Scott Lystig Fritchie
9fb19aa8ee
Merge pull request #27 from basho/ku/border-checksum
...
Regenerate checksum when chunks are to be sliced
2015-10-27 12:51:02 +09:00
Scott Lystig Fritchie
bbbd9748f3
Fix compiler & dialyzer warnings
2015-10-27 12:45:48 +09:00
UENISHI Kota
b2eb3e089c
Cleanup MACROs and changed default value
...
* machi_file_proxy now uses application environment
value `max_file_size` via machi_config
* changed name from MAX_FILE_SIZE to DEFAULT_MAX_FILE_SIZE
2015-10-27 11:17:59 +09:00
UENISHI Kota
5913531e32
Introduce machi_config.erl
2015-10-27 11:00:05 +09:00
UENISHI Kota
60364fe0ca
Fix typo
2015-10-26 10:10:08 +09:00
UENISHI Kota
0e4ae818af
Clarify checksum tags, define macros to avoid typos
2015-10-23 19:32:22 +09:00
UENISHI Kota
3d6d4d8be3
Do the slicing in flu server rather than in CR client
2015-10-23 18:49:49 +09:00
UENISHI Kota
c5661571e3
Regenerate checksum when chunks are to be sliced
2015-10-23 17:33:08 +09:00
