Add file0_test.escript (and big squash)
Small cleanups
Small cleanups
Refactoring argnames & order for more consistency
Add server-side-calculated MD5 checksum + logging
file:consult() style checksum management, too slow! 513K csums = 105 seconds, ouch
Much faster checksum recording
Add checksum_list. Alas, line-by-line I/O is slow, neh?
Much faster checksum listing
Add file0_verify_checksums.escript and supporting code
Adjust escript +A and -smp flags
Add file0_compare_filelists.escript
First draft of file0_repair_server.escript
First draft of file0_repair_server.escript, part 2
WIP of file0_repair_server.escript, part 3
WIP of file0_repair_server.escript, part 4
Basic repair works, it seems, hooray!
When checksum file ordering is different, try a cheap(?) 'cmp' on sorted results instead
Add README.md
Initial import of szone_chash.erl
Add file0_cc_make_projection.escript and supporting code
Add file0_cc_map_prefix.escript and supporting code
Change think-o: hash output is a chain, silly boy
Add file0_cc_1file_write_redundant.escript and support
Add file0_cc_read_client.escript and supporting code
Add examples/servers.map & file0_start_servers.escript
WIP: working on file0_cc_migrate_files.escript
File migration finished, works, yay!
Add basic 'what am I' docs to each script
Add file0_server_daemon.escript
Minor fixes
Fix broken unit test
Add basho_bench run() commands for append & read ops with projection
Add to examples dir
WIP: erasure coding hack, part 1
Fix broken unit test
WIP: erasure coding hack, part 2
WIP: erasure coding hack, part 3, EC data write is finished!
WIP: erasure coding hack, part 4, EC data read still in progress
WIP: erasure coding hack, part 5, EC data read still in progress
WIP: erasure coding hack, part 5b, EC data read still in progress
WIP: erasure coding hack, EC data read finished!
README update, part 1
README update, part 2
Oops, put back the printed ouput for file-write-client and 1file-write-redundant-client
README update, part 3
Fix 'user' output bug in list-client
Ugly hacks to get output/no-output from write clients
Clean up minor output bugs
Clean up minor output bugs, part 2
README update, part 4
Clean up minor output bugs, part 3
Clean up minor output bugs, part 5
Clean up minor output bugs, part 6
README update, part 6
README update, part 7
README update, part 7
README update, part 8
Final edits/fixes for demo day
Fix another oops in the README/demo day script
2014-12-21 08:52:38 +00:00
|
|
|
{mode, {rate,50}}.
|
|
|
|
%% {mode, max}.
|
2014-12-21 07:31:45 +00:00
|
|
|
|
|
|
|
{duration, 1}.
|
Add file0_test.escript (and big squash)
Small cleanups
Small cleanups
Refactoring argnames & order for more consistency
Add server-side-calculated MD5 checksum + logging
file:consult() style checksum management, too slow! 513K csums = 105 seconds, ouch
Much faster checksum recording
Add checksum_list. Alas, line-by-line I/O is slow, neh?
Much faster checksum listing
Add file0_verify_checksums.escript and supporting code
Adjust escript +A and -smp flags
Add file0_compare_filelists.escript
First draft of file0_repair_server.escript
First draft of file0_repair_server.escript, part 2
WIP of file0_repair_server.escript, part 3
WIP of file0_repair_server.escript, part 4
Basic repair works, it seems, hooray!
When checksum file ordering is different, try a cheap(?) 'cmp' on sorted results instead
Add README.md
Initial import of szone_chash.erl
Add file0_cc_make_projection.escript and supporting code
Add file0_cc_map_prefix.escript and supporting code
Change think-o: hash output is a chain, silly boy
Add file0_cc_1file_write_redundant.escript and support
Add file0_cc_read_client.escript and supporting code
Add examples/servers.map & file0_start_servers.escript
WIP: working on file0_cc_migrate_files.escript
File migration finished, works, yay!
Add basic 'what am I' docs to each script
Add file0_server_daemon.escript
Minor fixes
Fix broken unit test
Add basho_bench run() commands for append & read ops with projection
Add to examples dir
WIP: erasure coding hack, part 1
Fix broken unit test
WIP: erasure coding hack, part 2
WIP: erasure coding hack, part 3, EC data write is finished!
WIP: erasure coding hack, part 4, EC data read still in progress
WIP: erasure coding hack, part 5, EC data read still in progress
WIP: erasure coding hack, part 5b, EC data read still in progress
WIP: erasure coding hack, EC data read finished!
README update, part 1
README update, part 2
Oops, put back the printed ouput for file-write-client and 1file-write-redundant-client
README update, part 3
Fix 'user' output bug in list-client
Ugly hacks to get output/no-output from write clients
Clean up minor output bugs
Clean up minor output bugs, part 2
README update, part 4
Clean up minor output bugs, part 3
Clean up minor output bugs, part 5
Clean up minor output bugs, part 6
README update, part 6
README update, part 7
README update, part 7
README update, part 8
Final edits/fixes for demo day
Fix another oops in the README/demo day script
2014-12-21 08:52:38 +00:00
|
|
|
%{duration, 15}.
|
2014-12-21 07:31:45 +00:00
|
|
|
{report_interval, 1}.
|
|
|
|
|
Add file0_test.escript (and big squash)
Small cleanups
Small cleanups
Refactoring argnames & order for more consistency
Add server-side-calculated MD5 checksum + logging
file:consult() style checksum management, too slow! 513K csums = 105 seconds, ouch
Much faster checksum recording
Add checksum_list. Alas, line-by-line I/O is slow, neh?
Much faster checksum listing
Add file0_verify_checksums.escript and supporting code
Adjust escript +A and -smp flags
Add file0_compare_filelists.escript
First draft of file0_repair_server.escript
First draft of file0_repair_server.escript, part 2
WIP of file0_repair_server.escript, part 3
WIP of file0_repair_server.escript, part 4
Basic repair works, it seems, hooray!
When checksum file ordering is different, try a cheap(?) 'cmp' on sorted results instead
Add README.md
Initial import of szone_chash.erl
Add file0_cc_make_projection.escript and supporting code
Add file0_cc_map_prefix.escript and supporting code
Change think-o: hash output is a chain, silly boy
Add file0_cc_1file_write_redundant.escript and support
Add file0_cc_read_client.escript and supporting code
Add examples/servers.map & file0_start_servers.escript
WIP: working on file0_cc_migrate_files.escript
File migration finished, works, yay!
Add basic 'what am I' docs to each script
Add file0_server_daemon.escript
Minor fixes
Fix broken unit test
Add basho_bench run() commands for append & read ops with projection
Add to examples dir
WIP: erasure coding hack, part 1
Fix broken unit test
WIP: erasure coding hack, part 2
WIP: erasure coding hack, part 3, EC data write is finished!
WIP: erasure coding hack, part 4, EC data read still in progress
WIP: erasure coding hack, part 5, EC data read still in progress
WIP: erasure coding hack, part 5b, EC data read still in progress
WIP: erasure coding hack, EC data read finished!
README update, part 1
README update, part 2
Oops, put back the printed ouput for file-write-client and 1file-write-redundant-client
README update, part 3
Fix 'user' output bug in list-client
Ugly hacks to get output/no-output from write clients
Clean up minor output bugs
Clean up minor output bugs, part 2
README update, part 4
Clean up minor output bugs, part 3
Clean up minor output bugs, part 5
Clean up minor output bugs, part 6
README update, part 6
README update, part 7
README update, part 7
README update, part 8
Final edits/fixes for demo day
Fix another oops in the README/demo day script
2014-12-21 08:52:38 +00:00
|
|
|
%% {concurrent, 1}.
|
|
|
|
{concurrent, 32}.
|
2014-12-21 07:31:45 +00:00
|
|
|
|
|
|
|
{code_paths, ["/tmp"]}.
|
|
|
|
{driver, file0}.
|
Add file0_test.escript (and big squash)
Small cleanups
Small cleanups
Refactoring argnames & order for more consistency
Add server-side-calculated MD5 checksum + logging
file:consult() style checksum management, too slow! 513K csums = 105 seconds, ouch
Much faster checksum recording
Add checksum_list. Alas, line-by-line I/O is slow, neh?
Much faster checksum listing
Add file0_verify_checksums.escript and supporting code
Adjust escript +A and -smp flags
Add file0_compare_filelists.escript
First draft of file0_repair_server.escript
First draft of file0_repair_server.escript, part 2
WIP of file0_repair_server.escript, part 3
WIP of file0_repair_server.escript, part 4
Basic repair works, it seems, hooray!
When checksum file ordering is different, try a cheap(?) 'cmp' on sorted results instead
Add README.md
Initial import of szone_chash.erl
Add file0_cc_make_projection.escript and supporting code
Add file0_cc_map_prefix.escript and supporting code
Change think-o: hash output is a chain, silly boy
Add file0_cc_1file_write_redundant.escript and support
Add file0_cc_read_client.escript and supporting code
Add examples/servers.map & file0_start_servers.escript
WIP: working on file0_cc_migrate_files.escript
File migration finished, works, yay!
Add basic 'what am I' docs to each script
Add file0_server_daemon.escript
Minor fixes
Fix broken unit test
Add basho_bench run() commands for append & read ops with projection
Add to examples dir
WIP: erasure coding hack, part 1
Fix broken unit test
WIP: erasure coding hack, part 2
WIP: erasure coding hack, part 3, EC data write is finished!
WIP: erasure coding hack, part 4, EC data read still in progress
WIP: erasure coding hack, part 5, EC data read still in progress
WIP: erasure coding hack, part 5b, EC data read still in progress
WIP: erasure coding hack, EC data read finished!
README update, part 1
README update, part 2
Oops, put back the printed ouput for file-write-client and 1file-write-redundant-client
README update, part 3
Fix 'user' output bug in list-client
Ugly hacks to get output/no-output from write clients
Clean up minor output bugs
Clean up minor output bugs, part 2
README update, part 4
Clean up minor output bugs, part 3
Clean up minor output bugs, part 5
Clean up minor output bugs, part 6
README update, part 6
README update, part 7
README update, part 7
README update, part 8
Final edits/fixes for demo day
Fix another oops in the README/demo day script
2014-12-21 08:52:38 +00:00
|
|
|
{file0_ip_list, [{{127,0,0,1}, 7071}]}.
|
|
|
|
%%% {file0_start_listener, {7071, "/tmp/SAM1/seq-tests/data"}}.
|
2014-12-21 07:31:45 +00:00
|
|
|
{value_generator_source_size, 16001001}.
|
Add file0_test.escript (and big squash)
Small cleanups
Small cleanups
Refactoring argnames & order for more consistency
Add server-side-calculated MD5 checksum + logging
file:consult() style checksum management, too slow! 513K csums = 105 seconds, ouch
Much faster checksum recording
Add checksum_list. Alas, line-by-line I/O is slow, neh?
Much faster checksum listing
Add file0_verify_checksums.escript and supporting code
Adjust escript +A and -smp flags
Add file0_compare_filelists.escript
First draft of file0_repair_server.escript
First draft of file0_repair_server.escript, part 2
WIP of file0_repair_server.escript, part 3
WIP of file0_repair_server.escript, part 4
Basic repair works, it seems, hooray!
When checksum file ordering is different, try a cheap(?) 'cmp' on sorted results instead
Add README.md
Initial import of szone_chash.erl
Add file0_cc_make_projection.escript and supporting code
Add file0_cc_map_prefix.escript and supporting code
Change think-o: hash output is a chain, silly boy
Add file0_cc_1file_write_redundant.escript and support
Add file0_cc_read_client.escript and supporting code
Add examples/servers.map & file0_start_servers.escript
WIP: working on file0_cc_migrate_files.escript
File migration finished, works, yay!
Add basic 'what am I' docs to each script
Add file0_server_daemon.escript
Minor fixes
Fix broken unit test
Add basho_bench run() commands for append & read ops with projection
Add to examples dir
WIP: erasure coding hack, part 1
Fix broken unit test
WIP: erasure coding hack, part 2
WIP: erasure coding hack, part 3, EC data write is finished!
WIP: erasure coding hack, part 4, EC data read still in progress
WIP: erasure coding hack, part 5, EC data read still in progress
WIP: erasure coding hack, part 5b, EC data read still in progress
WIP: erasure coding hack, EC data read finished!
README update, part 1
README update, part 2
Oops, put back the printed ouput for file-write-client and 1file-write-redundant-client
README update, part 3
Fix 'user' output bug in list-client
Ugly hacks to get output/no-output from write clients
Clean up minor output bugs
Clean up minor output bugs, part 2
README update, part 4
Clean up minor output bugs, part 3
Clean up minor output bugs, part 5
Clean up minor output bugs, part 6
README update, part 6
README update, part 7
README update, part 7
README update, part 8
Final edits/fixes for demo day
Fix another oops in the README/demo day script
2014-12-21 08:52:38 +00:00
|
|
|
%{key_generator, <<"32k-archive">>}.
|
|
|
|
{key_generator, {to_binstr, "32k-prefix-~w", {uniform_int, 1000}}}.
|
|
|
|
{file0_projection_path, "/Users/fritchie/b/src/misc/sequencer/examples/1.proj"}.
|
2014-12-21 07:31:45 +00:00
|
|
|
|
|
|
|
%% variations of put
|
|
|
|
|
Add file0_test.escript (and big squash)
Small cleanups
Small cleanups
Refactoring argnames & order for more consistency
Add server-side-calculated MD5 checksum + logging
file:consult() style checksum management, too slow! 513K csums = 105 seconds, ouch
Much faster checksum recording
Add checksum_list. Alas, line-by-line I/O is slow, neh?
Much faster checksum listing
Add file0_verify_checksums.escript and supporting code
Adjust escript +A and -smp flags
Add file0_compare_filelists.escript
First draft of file0_repair_server.escript
First draft of file0_repair_server.escript, part 2
WIP of file0_repair_server.escript, part 3
WIP of file0_repair_server.escript, part 4
Basic repair works, it seems, hooray!
When checksum file ordering is different, try a cheap(?) 'cmp' on sorted results instead
Add README.md
Initial import of szone_chash.erl
Add file0_cc_make_projection.escript and supporting code
Add file0_cc_map_prefix.escript and supporting code
Change think-o: hash output is a chain, silly boy
Add file0_cc_1file_write_redundant.escript and support
Add file0_cc_read_client.escript and supporting code
Add examples/servers.map & file0_start_servers.escript
WIP: working on file0_cc_migrate_files.escript
File migration finished, works, yay!
Add basic 'what am I' docs to each script
Add file0_server_daemon.escript
Minor fixes
Fix broken unit test
Add basho_bench run() commands for append & read ops with projection
Add to examples dir
WIP: erasure coding hack, part 1
Fix broken unit test
WIP: erasure coding hack, part 2
WIP: erasure coding hack, part 3, EC data write is finished!
WIP: erasure coding hack, part 4, EC data read still in progress
WIP: erasure coding hack, part 5, EC data read still in progress
WIP: erasure coding hack, part 5b, EC data read still in progress
WIP: erasure coding hack, EC data read finished!
README update, part 1
README update, part 2
Oops, put back the printed ouput for file-write-client and 1file-write-redundant-client
README update, part 3
Fix 'user' output bug in list-client
Ugly hacks to get output/no-output from write clients
Clean up minor output bugs
Clean up minor output bugs, part 2
README update, part 4
Clean up minor output bugs, part 3
Clean up minor output bugs, part 5
Clean up minor output bugs, part 6
README update, part 6
README update, part 7
README update, part 7
README update, part 8
Final edits/fixes for demo day
Fix another oops in the README/demo day script
2014-12-21 08:52:38 +00:00
|
|
|
%{operations, [{keygen_valuegen_then_null, 1}]}.
|
2014-12-21 07:31:45 +00:00
|
|
|
{value_generator, {fixed_bin, 32768}}.
|
|
|
|
%{operations, [{append_local_server, 1}]}.
|
Add file0_test.escript (and big squash)
Small cleanups
Small cleanups
Refactoring argnames & order for more consistency
Add server-side-calculated MD5 checksum + logging
file:consult() style checksum management, too slow! 513K csums = 105 seconds, ouch
Much faster checksum recording
Add checksum_list. Alas, line-by-line I/O is slow, neh?
Much faster checksum listing
Add file0_verify_checksums.escript and supporting code
Adjust escript +A and -smp flags
Add file0_compare_filelists.escript
First draft of file0_repair_server.escript
First draft of file0_repair_server.escript, part 2
WIP of file0_repair_server.escript, part 3
WIP of file0_repair_server.escript, part 4
Basic repair works, it seems, hooray!
When checksum file ordering is different, try a cheap(?) 'cmp' on sorted results instead
Add README.md
Initial import of szone_chash.erl
Add file0_cc_make_projection.escript and supporting code
Add file0_cc_map_prefix.escript and supporting code
Change think-o: hash output is a chain, silly boy
Add file0_cc_1file_write_redundant.escript and support
Add file0_cc_read_client.escript and supporting code
Add examples/servers.map & file0_start_servers.escript
WIP: working on file0_cc_migrate_files.escript
File migration finished, works, yay!
Add basic 'what am I' docs to each script
Add file0_server_daemon.escript
Minor fixes
Fix broken unit test
Add basho_bench run() commands for append & read ops with projection
Add to examples dir
WIP: erasure coding hack, part 1
Fix broken unit test
WIP: erasure coding hack, part 2
WIP: erasure coding hack, part 3, EC data write is finished!
WIP: erasure coding hack, part 4, EC data read still in progress
WIP: erasure coding hack, part 5, EC data read still in progress
WIP: erasure coding hack, part 5b, EC data read still in progress
WIP: erasure coding hack, EC data read finished!
README update, part 1
README update, part 2
Oops, put back the printed ouput for file-write-client and 1file-write-redundant-client
README update, part 3
Fix 'user' output bug in list-client
Ugly hacks to get output/no-output from write clients
Clean up minor output bugs
Clean up minor output bugs, part 2
README update, part 4
Clean up minor output bugs, part 3
Clean up minor output bugs, part 5
Clean up minor output bugs, part 6
README update, part 6
README update, part 7
README update, part 7
README update, part 8
Final edits/fixes for demo day
Fix another oops in the README/demo day script
2014-12-21 08:52:38 +00:00
|
|
|
%{operations, [{append_remote_server, 1}]}.
|
|
|
|
%{operations, [{cc_append_remote_server, 1}]}.
|
|
|
|
{operations, [{append_remote_server, 1}, {cc_append_remote_server, 1}]}.
|
2014-12-21 07:31:45 +00:00
|
|
|
%{operations, [{append_local_server, 1}, {append_remote_server, 1}]}.
|
|
|
|
|
|
|
|
%% variations of get
|
|
|
|
|
Add file0_test.escript (and big squash)
Small cleanups
Small cleanups
Refactoring argnames & order for more consistency
Add server-side-calculated MD5 checksum + logging
file:consult() style checksum management, too slow! 513K csums = 105 seconds, ouch
Much faster checksum recording
Add checksum_list. Alas, line-by-line I/O is slow, neh?
Much faster checksum listing
Add file0_verify_checksums.escript and supporting code
Adjust escript +A and -smp flags
Add file0_compare_filelists.escript
First draft of file0_repair_server.escript
First draft of file0_repair_server.escript, part 2
WIP of file0_repair_server.escript, part 3
WIP of file0_repair_server.escript, part 4
Basic repair works, it seems, hooray!
When checksum file ordering is different, try a cheap(?) 'cmp' on sorted results instead
Add README.md
Initial import of szone_chash.erl
Add file0_cc_make_projection.escript and supporting code
Add file0_cc_map_prefix.escript and supporting code
Change think-o: hash output is a chain, silly boy
Add file0_cc_1file_write_redundant.escript and support
Add file0_cc_read_client.escript and supporting code
Add examples/servers.map & file0_start_servers.escript
WIP: working on file0_cc_migrate_files.escript
File migration finished, works, yay!
Add basic 'what am I' docs to each script
Add file0_server_daemon.escript
Minor fixes
Fix broken unit test
Add basho_bench run() commands for append & read ops with projection
Add to examples dir
WIP: erasure coding hack, part 1
Fix broken unit test
WIP: erasure coding hack, part 2
WIP: erasure coding hack, part 3, EC data write is finished!
WIP: erasure coding hack, part 4, EC data read still in progress
WIP: erasure coding hack, part 5, EC data read still in progress
WIP: erasure coding hack, part 5b, EC data read still in progress
WIP: erasure coding hack, EC data read finished!
README update, part 1
README update, part 2
Oops, put back the printed ouput for file-write-client and 1file-write-redundant-client
README update, part 3
Fix 'user' output bug in list-client
Ugly hacks to get output/no-output from write clients
Clean up minor output bugs
Clean up minor output bugs, part 2
README update, part 4
Clean up minor output bugs, part 3
Clean up minor output bugs, part 5
Clean up minor output bugs, part 6
README update, part 6
README update, part 7
README update, part 7
README update, part 8
Final edits/fixes for demo day
Fix another oops in the README/demo day script
2014-12-21 08:52:38 +00:00
|
|
|
%% perl -e 'srand(time); $chunksize = 32768 ; foreach $file (@ARGV) { @fi = stat($file); $size = $fi[7]; $base = $file; $base =~ s|[^/]*/||g; for ($i = 0; $i < $size - $chunksize; $i += $chunksize) { printf "%d R %016x %08x %s\n", rand(999999), $i, $chunksize, $base; } }' data/*.* | sort -n | awk '{print $2, $3, $4, $5}' > /tmp/input.txt
|
2014-12-21 07:31:45 +00:00
|
|
|
%% {key_generator, {file_line_bin, "/tmp/input.txt"}}.
|
|
|
|
%% {value_generator_source_size, 1}.
|
|
|
|
%% {operations, [{read_raw_line_local, 1}]}.
|
Add file0_test.escript (and big squash)
Small cleanups
Small cleanups
Refactoring argnames & order for more consistency
Add server-side-calculated MD5 checksum + logging
file:consult() style checksum management, too slow! 513K csums = 105 seconds, ouch
Much faster checksum recording
Add checksum_list. Alas, line-by-line I/O is slow, neh?
Much faster checksum listing
Add file0_verify_checksums.escript and supporting code
Adjust escript +A and -smp flags
Add file0_compare_filelists.escript
First draft of file0_repair_server.escript
First draft of file0_repair_server.escript, part 2
WIP of file0_repair_server.escript, part 3
WIP of file0_repair_server.escript, part 4
Basic repair works, it seems, hooray!
When checksum file ordering is different, try a cheap(?) 'cmp' on sorted results instead
Add README.md
Initial import of szone_chash.erl
Add file0_cc_make_projection.escript and supporting code
Add file0_cc_map_prefix.escript and supporting code
Change think-o: hash output is a chain, silly boy
Add file0_cc_1file_write_redundant.escript and support
Add file0_cc_read_client.escript and supporting code
Add examples/servers.map & file0_start_servers.escript
WIP: working on file0_cc_migrate_files.escript
File migration finished, works, yay!
Add basic 'what am I' docs to each script
Add file0_server_daemon.escript
Minor fixes
Fix broken unit test
Add basho_bench run() commands for append & read ops with projection
Add to examples dir
WIP: erasure coding hack, part 1
Fix broken unit test
WIP: erasure coding hack, part 2
WIP: erasure coding hack, part 3, EC data write is finished!
WIP: erasure coding hack, part 4, EC data read still in progress
WIP: erasure coding hack, part 5, EC data read still in progress
WIP: erasure coding hack, part 5b, EC data read still in progress
WIP: erasure coding hack, EC data read finished!
README update, part 1
README update, part 2
Oops, put back the printed ouput for file-write-client and 1file-write-redundant-client
README update, part 3
Fix 'user' output bug in list-client
Ugly hacks to get output/no-output from write clients
Clean up minor output bugs
Clean up minor output bugs, part 2
README update, part 4
Clean up minor output bugs, part 3
Clean up minor output bugs, part 5
Clean up minor output bugs, part 6
README update, part 6
README update, part 7
README update, part 7
README update, part 8
Final edits/fixes for demo day
Fix another oops in the README/demo day script
2014-12-21 08:52:38 +00:00
|
|
|
|
|
|
|
%% {key_generator, {file_line_bin, "/tmp/input.txt"}}.
|
|
|
|
%% {value_generator_source_size, 1}.
|
|
|
|
%% %{operations, [{cc_read_raw_line_local, 1}]}.
|
|
|
|
%% {operations, [{read_raw_line_local, 1}, {cc_read_raw_line_local, 1}]}.
|