* Perfile LevelDB instance usage are changed to use single instance
per FLU server.
* Not only chunk checksums, but the list of trimmed files are also
stored in LevelDB.
* Remove 1024 bytes file header; instead put any metadata into
LevelDB if needed.
* LevelDB `db_ref()` lifecycle is same as that of `machi_metadata_mgr`
* `machi_file_proxy` just uses it as it's passed at process startup
* There are several optimization space still left as it is
When there is TCP_WAIT connection whose local part has
port to be listened, listen (bind) will fail by eaddrinuse
_on Linux_ (won't on Mac OS X).
This commit also adds some logs and pattern matches.
Reference
- Ephemeral port - Wikipedia, the free encyclopedia
https://en.wikipedia.org/wiki/Ephemeral_port
"Many Linux kernels use the port range 32768 to 61000.[note 2]
FreeBSD has used the IANA port range since release 4.6. Previous
versions, including the Berkeley Software Distribution (BSD), use
ports 1024 to 5000 as ephemeral ports.[2]"
- Demostration of collision between already-closed ephemeral port
and listen port on Linux (Mac OS X allows)
https://gist.github.com/shino/36ae1e01608366d52236
Was a service previously. Now contains both merklet
and the naive implementations. Put construction
timing stuff into the test.
Tests are not truly meaningful yet.
* maybe_gc/2 is triggered at machi_file_proxy, when chunk is deleted
and the file is larger than `max_file_size`
* A file is deleted if all chunks except 1024 bytes header are trimmed
* If a file is going to be deleted, file_proxy notifies metadata_mgr
to remember the filename persistently, whose filename is
`known_files_<FluName>`
* Such trimmed filenames are stored in a machi_plist file per flu
* machi_file_proxy could not be started if the filename is in the
manager's list. Consequently, any write, read and trim operations
cannot happen against deleted file.
* After the file was trimmed, any read request to the file returns
`{error, trimmed}`
* Disclaimer: no tests written yet and machi_plist does not support
any recovery from partial writes.
* Add some thoughts as comments for repairing trims.
* State diagram of every byte is as follows:
```
state\action| write/append | read_chunk | trim_chunk
------------+----------------+------------------+---------------
unwritten | -> written | fail (+repair) | -> trimmed
written | noop or repair | return content | -> trimmed
trimmed | fail | fail | noop
```
