greg/hanoidb
1
0
Fork 0
Code Issues 10 Pull requests 3 Projects Releases Packages Wiki Activity
hanoidb/test/hanoidb_merger_tests.erl

64 lines
2.1 KiB
Erlang
Raw Normal View History

Put the copyright/license header into a few overlooked files.
2012-04-19 22:09:01 +00:00
%% ----------------------------------------------------------------------------
%%
Rename hanoi -> hanoidb
2012-05-07 15:22:55 +00:00
%% hanoidb: LSM-trees (Log-Structured Merge Trees) Indexed Storage
Put the copyright/license header into a few overlooked files.
2012-04-19 22:09:01 +00:00
%%
%% Copyright 2011-2012 (c) Trifork A/S. All Rights Reserved.
%% http://trifork.com/ info@trifork.com
%%
%% Copyright 2012 (c) Basho Technologies, Inc. All Rights Reserved.
%% http://basho.com/ info@basho.com
%%
%% This file is provided to you under the Apache License, Version 2.0 (the
%% "License"); you may not use this file except in compliance with the License.
%% You may obtain a copy of the License at
%%
%% http://www.apache.org/licenses/LICENSE-2.0
%%
%% Unless required by applicable law or agreed to in writing, software
%% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
%% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
%% License for the specific language governing permissions and limitations
%% under the License.
%%
%% ----------------------------------------------------------------------------
Rename hanoi -> hanoidb
2012-05-07 15:22:55 +00:00
-module(hanoidb_merger_tests).
Split tests to where they belong. Add simple statem test. ; Split the writer tests to the writer parts. This is not entirely correct since they both read and write, but oh, well. ; Provide a simple statem test which opens trees and puts values.
2012-01-06 21:56:23 +00:00
-ifdef(TEST).
-include_lib("eunit/include/eunit.hrl").
-endif.
-compile(export_all).
merge_test() ->
Add option to evict tombstones in merge
2012-01-06 23:28:26 +00:00
file:delete("test1"),
file:delete("test2"),
file:delete("test3"),
Refactor for expiry_secs option Tree nodes now hold entries at the form {Key, ?TOMBSTONE | BinValue | {?TOMBSTONE, TStamp} | {BinValue, TStamp}} We use the form without TStamp when expiry_secs is unset or set to 0 (i.e., values don't expire). merger/writer: Move KV count into writer, because now the writer:add determines if a value is expired and thus wither a value is actually written. Thus, writer now has a new API function which returns the KV count written so far. reader: lookup/fold API hides the TStamp tuples, so only the next_node API used by the merger is exposed to these {Key, {_, TStamp}} entries. nursery: like reader, the TStamp'ed tuples are not exposed in the client API; expired values are simply not returned from fold/lookup. hanoidb: add config option {expiry_secs, N}. other modules: Make sure that config is passed all the way down through (sub) processes to be able to utilize the config option everywhere. test: update to work with new option.
2012-05-11 10:00:32 +00:00
{ok, BT1} = hanoidb_writer:open("test1", [{expiry_secs, 0}]),
Split tests to where they belong. Add simple statem test. ; Split the writer tests to the writer parts. This is not entirely correct since they both read and write, but oh, well. ; Provide a simple statem test which opens trees and puts values.
2012-01-06 21:56:23 +00:00
lists:foldl(fun(N,_) ->
Rename hanoi -> hanoidb
2012-05-07 15:22:55 +00:00
ok = hanoidb_writer:add(BT1, <<N:128>>, <<"data",N:128>>)
Split tests to where they belong. Add simple statem test. ; Split the writer tests to the writer parts. This is not entirely correct since they both read and write, but oh, well. ; Provide a simple statem test which opens trees and puts values.
2012-01-06 21:56:23 +00:00
end,
ok,
lists:seq(1,10000,2)),
Rename hanoi -> hanoidb
2012-05-07 15:22:55 +00:00
ok = hanoidb_writer:close(BT1),
Split tests to where they belong. Add simple statem test. ; Split the writer tests to the writer parts. This is not entirely correct since they both read and write, but oh, well. ; Provide a simple statem test which opens trees and puts values.
2012-01-06 21:56:23 +00:00
Refactor for expiry_secs option Tree nodes now hold entries at the form {Key, ?TOMBSTONE | BinValue | {?TOMBSTONE, TStamp} | {BinValue, TStamp}} We use the form without TStamp when expiry_secs is unset or set to 0 (i.e., values don't expire). merger/writer: Move KV count into writer, because now the writer:add determines if a value is expired and thus wither a value is actually written. Thus, writer now has a new API function which returns the KV count written so far. reader: lookup/fold API hides the TStamp tuples, so only the next_node API used by the merger is exposed to these {Key, {_, TStamp}} entries. nursery: like reader, the TStamp'ed tuples are not exposed in the client API; expired values are simply not returned from fold/lookup. hanoidb: add config option {expiry_secs, N}. other modules: Make sure that config is passed all the way down through (sub) processes to be able to utilize the config option everywhere. test: update to work with new option.
2012-05-11 10:00:32 +00:00
{ok, BT2} = hanoidb_writer:open("test2", [{expiry_secs, 0}]),
Split tests to where they belong. Add simple statem test. ; Split the writer tests to the writer parts. This is not entirely correct since they both read and write, but oh, well. ; Provide a simple statem test which opens trees and puts values.
2012-01-06 21:56:23 +00:00
lists:foldl(fun(N,_) ->
Rename hanoi -> hanoidb
2012-05-07 15:22:55 +00:00
ok = hanoidb_writer:add(BT2, <<N:128>>, <<"data",N:128>>)
Split tests to where they belong. Add simple statem test. ; Split the writer tests to the writer parts. This is not entirely correct since they both read and write, but oh, well. ; Provide a simple statem test which opens trees and puts values.
2012-01-06 21:56:23 +00:00
end,
ok,
lists:seq(2,5001,1)),
Rename hanoi -> hanoidb
2012-05-07 15:22:55 +00:00
ok = hanoidb_writer:close(BT2),
Split tests to where they belong. Add simple statem test. ; Split the writer tests to the writer parts. This is not entirely correct since they both read and write, but oh, well. ; Provide a simple statem test which opens trees and puts values.
2012-01-06 21:56:23 +00:00
Implement incremental insert This slows down insert to be log2(N), where N is the total number of objects in the store. The upside is that it also removes the terrible worst case scenarios for insert.
2012-04-19 17:57:39 +00:00
self() ! {step, {self(), none}, 2000000000},
Refactor for expiry_secs option Tree nodes now hold entries at the form {Key, ?TOMBSTONE | BinValue | {?TOMBSTONE, TStamp} | {BinValue, TStamp}} We use the form without TStamp when expiry_secs is unset or set to 0 (i.e., values don't expire). merger/writer: Move KV count into writer, because now the writer:add determines if a value is expired and thus wither a value is actually written. Thus, writer now has a new API function which returns the KV count written so far. reader: lookup/fold API hides the TStamp tuples, so only the next_node API used by the merger is exposed to these {Key, {_, TStamp}} entries. nursery: like reader, the TStamp'ed tuples are not exposed in the client API; expired values are simply not returned from fold/lookup. hanoidb: add config option {expiry_secs, N}. other modules: Make sure that config is passed all the way down through (sub) processes to be able to utilize the config option everywhere. test: update to work with new option.
2012-05-11 10:00:32 +00:00
{Time,{ok,Count}} = timer:tc(hanoidb_merger, merge, ["test1", "test2", "test3", 10000, true, [{expiry_secs, 0}]]),
Split tests to where they belong. Add simple statem test. ; Split the writer tests to the writer parts. This is not entirely correct since they both read and write, but oh, well. ; Provide a simple statem test which opens trees and puts values.
2012-01-06 21:56:23 +00:00
WIP: working toward fixing the blocking fold test
2012-06-19 14:03:42 +00:00
% error_logger:info_msg("time to merge: ~p/sec (time=~p, count=~p)~n", [1000000/(Time/Count), Time/1000000, Count]),
Split tests to where they belong. Add simple statem test. ; Split the writer tests to the writer parts. This is not entirely correct since they both read and write, but oh, well. ; Provide a simple statem test which opens trees and puts values.
2012-01-06 21:56:23 +00:00
ok.
Reference in a new issue Copy permalink