* Perfile LevelDB instance usage are changed to use single instance
per FLU server.
* machi_csum_file reference is managed with machi_flu_filename_mgr
as an aim to manage filenames with leveldb
* Not only chunk checksums, but the list of trimmed files are also
stored in LevelDB.
* Remove 1024 bytes file header; instead put any metadata into
LevelDB if needed.
* LevelDB `db_ref()` lifecycle is same as that of `machi_metadata_mgr`
* `machi_file_proxy` just uses it as it's passed at process startup
* There are several optimization space still left as it is
WIP
When there is TCP_WAIT connection whose local part has
port to be listened, listen (bind) will fail by eaddrinuse
_on Linux_ (won't on Mac OS X).
This commit also adds some logs and pattern matches.
Reference
- Ephemeral port - Wikipedia, the free encyclopedia
https://en.wikipedia.org/wiki/Ephemeral_port
"Many Linux kernels use the port range 32768 to 61000.[note 2]
FreeBSD has used the IANA port range since release 4.6. Previous
versions, including the Berkeley Software Distribution (BSD), use
ports 1024 to 5000 as ephemeral ports.[2]"
- Demostration of collision between already-closed ephemeral port
and listen port on Linux (Mac OS X allows)
https://gist.github.com/shino/36ae1e01608366d52236
This is simply a change of read_chunk() protocol, where a response of
read_chunk() becomes list of written bytes along with checksum. All
related code including repair is changed as such. This is to pass all
tests and not actually supporting partial chunks.
So, the PB style encoding of the Mpb_LL_WriteProjectionReq message
is about 35-36 times slower than using Erlang's term_to_binary()
and binary_to_term(). {sigh}
So, there's some cheating going on, because some of the parts of
the #projection_v1{} and #p_srvr{} records aren't fully specified.
Those parts are being specified as "opaque" in the field names, e.g.
optional bytes opaque_flap = 10;
optional bytes opaque_inner = 11;
required bytes opaque_dbg = 12;
required bytes opaque_dbg2 = 13;
The serialization that's being used is erlang term sexprs. That isn't
portable. So if/when we really need to deal with a non-Erlang
language, we'll have to straighten this out further.
