Bump version, update Changelog and documentation for 1.2 release.
This commit is contained in:
parent
47ebd5b752
commit
16b1d9e7a3
4 changed files with 77 additions and 5 deletions
70
Changelog
70
Changelog
|
@ -1,3 +1,67 @@
|
||||||
|
== 1.2.0 Major Stable Release ==
|
||||||
|
Fix calculation of extra scratch space for Dedupe.
|
||||||
|
Add missing return value check in small buffer Delta2.
|
||||||
|
Update test failure detection in test driver.
|
||||||
|
Fix numeric parsing.
|
||||||
|
Fix dedupe bug introduced in last commit.
|
||||||
|
Reset valid flag when resetting dedupe context.
|
||||||
|
Cleanup test suites.
|
||||||
|
Do not abort test suite on failure of a test case.
|
||||||
|
Fix Delta2 handling for small buffers.
|
||||||
|
Fix LZP handling during preprocessing.
|
||||||
|
Fix type flag handling during preprocessing.
|
||||||
|
Update test cases to use configurable list of test corpus files.
|
||||||
|
Update some more int64_t datatypes to uint64_t
|
||||||
|
Update to latest XXHash version.
|
||||||
|
Fix Keccak invocation and reset default checksum back to SKEIN 256.
|
||||||
|
Improve Dedupe performance.
|
||||||
|
Fix compiler warning in allocator code.
|
||||||
|
Major enhancements to Delta2 encoding.
|
||||||
|
Fix the byteswap macros.
|
||||||
|
Start adding assertions.
|
||||||
|
Introduce strict compiler flags and fix scores of warnings/issues.
|
||||||
|
Avoid different optimization flags for Dedupe sources.
|
||||||
|
Fix liberal mixing of uint64_t and int64_t (should all be uint64_t).
|
||||||
|
Fix corner case crash when decompressing.
|
||||||
|
Fix Delta2 decode handling when not compressing.
|
||||||
|
More debug statements and other minor changes.
|
||||||
|
Avoid transposing below-threshold spans. Reduces compression ratio.
|
||||||
|
Use little-endian storage format for numbers to optimize for x86.
|
||||||
|
Improve embedded table detection.
|
||||||
|
Reduce Delta2 header sizes.
|
||||||
|
Fix preprocessing behavior when LZP does not compress but Delta2 works.
|
||||||
|
Improve Delta2 scanning speed and effectiveness.
|
||||||
|
Add destination buffer overflow check in Delta2.
|
||||||
|
Add rough speed computation.
|
||||||
|
More tweaks to Delta2 implementation.
|
||||||
|
Ability to run individual test suites from makefile.
|
||||||
|
Silence more Gcc warnings.
|
||||||
|
Update testing note.
|
||||||
|
Improve 64-bit compiler and platform checks.
|
||||||
|
Ensure core dumps are enabled during testing.
|
||||||
|
Enable building with alternate Zlib and Bzlib.
|
||||||
|
Fix correct setting of output size when using Delta2 without LZP.
|
||||||
|
README formatting.
|
||||||
|
Make Delta2 encoding independent of LZP.
|
||||||
|
Tweak Delta2 parameters.
|
||||||
|
Update README and test cases.
|
||||||
|
Update README to align with current features/behavior.
|
||||||
|
Fine tune transpose parameters.
|
||||||
|
Fix minor nits.
|
||||||
|
Change confusing structure member name.
|
||||||
|
Add Matrix Transpose of Dedupe index to compress it better.
|
||||||
|
Fix handling of Dedupe index compression failure.
|
||||||
|
Ensure intermediate file cleanup in tests.
|
||||||
|
Update to latest LZ4.
|
||||||
|
Get rid of size_t in places where 64-bitness is assumed.
|
||||||
|
Add support for 64-bit Keccak implementation.
|
||||||
|
Sanitize error message from tests.
|
||||||
|
Add more tests.
|
||||||
|
Improve platform detection in config script.
|
||||||
|
Add adaptive delta encoding test.
|
||||||
|
Implement Adaptive Delta Encoding (Delta2).
|
||||||
|
Work in progress global dedupe config setup.
|
||||||
|
|
||||||
== 1.1.0 Stable Release ==
|
== 1.1.0 Stable Release ==
|
||||||
Fix building without Libbsc support.
|
Fix building without Libbsc support.
|
||||||
Add more tests for corrupted encrypted files.
|
Add more tests for corrupted encrypted files.
|
||||||
|
@ -14,6 +78,7 @@ Fix corner case dedupe bug in error handling flow.
|
||||||
Bump archive version signature.
|
Bump archive version signature.
|
||||||
Work in progress global dedupe config loader.
|
Work in progress global dedupe config loader.
|
||||||
Use fixed rolling-hash mask for better block size approximation.
|
Use fixed rolling-hash mask for better block size approximation.
|
||||||
|
|
||||||
== 1.0.0 Stable Release ==
|
== 1.0.0 Stable Release ==
|
||||||
Fix chunk flag setup when compression fails in adaptive mode.
|
Fix chunk flag setup when compression fails in adaptive mode.
|
||||||
Prevent display of non-fatal errors during compression.
|
Prevent display of non-fatal errors during compression.
|
||||||
|
@ -27,6 +92,7 @@ Tweak chunking parameters for better block size distribution and dedupe ratio.
|
||||||
Add some more debug mode info.
|
Add some more debug mode info.
|
||||||
Minor fix for adapt mode.
|
Minor fix for adapt mode.
|
||||||
Use Libbsc for XML data in adapt2 mode.
|
Use Libbsc for XML data in adapt2 mode.
|
||||||
|
|
||||||
== 0.9.1 Minor update Release ==
|
== 0.9.1 Minor update Release ==
|
||||||
Portability to Debian based distros.
|
Portability to Debian based distros.
|
||||||
Enable SSE4/AVX detection for AMD platforms (Bulldozer has both).
|
Enable SSE4/AVX detection for AMD platforms (Bulldozer has both).
|
||||||
|
@ -49,6 +115,7 @@ Fix polynomial computation.
|
||||||
Fix incorrect block length when doing fixed-block dedupe.
|
Fix incorrect block length when doing fixed-block dedupe.
|
||||||
Remove unused structure member.
|
Remove unused structure member.
|
||||||
Switch to multiplicative rolling hash for good
|
Switch to multiplicative rolling hash for good
|
||||||
|
|
||||||
== 0.8.5 Update Release ==
|
== 0.8.5 Update Release ==
|
||||||
Update adaptive mode heuristic based on algorithms.
|
Update adaptive mode heuristic based on algorithms.
|
||||||
Remove incorrect check in PPMd decompression code.
|
Remove incorrect check in PPMd decompression code.
|
||||||
|
@ -66,6 +133,7 @@ Speed up Hash computation for dedupe blocks.
|
||||||
Add support for Fixed-Block deduplication.
|
Add support for Fixed-Block deduplication.
|
||||||
Reduce dedupe loop checks for slight speed edge.
|
Reduce dedupe loop checks for slight speed edge.
|
||||||
Implement K-min-values Sketch for Similarity detection.
|
Implement K-min-values Sketch for Similarity detection.
|
||||||
|
|
||||||
== 0.8.1 Bugfix Release ==
|
== 0.8.1 Bugfix Release ==
|
||||||
Fix return code handling in LZP pre-compression, crashed adaptive modes.
|
Fix return code handling in LZP pre-compression, crashed adaptive modes.
|
||||||
Allow user-specified minimum Dedupe block size.
|
Allow user-specified minimum Dedupe block size.
|
||||||
|
@ -81,6 +149,7 @@ Import Skein code from NIST CD submission
|
||||||
Make checksum algorithms pluggable
|
Make checksum algorithms pluggable
|
||||||
Fix handling of huge buffers (>2GB) in LZP
|
Fix handling of huge buffers (>2GB) in LZP
|
||||||
Cleanup of some buffer sizing code
|
Cleanup of some buffer sizing code
|
||||||
|
|
||||||
== 0.8 Beta release ==
|
== 0.8 Beta release ==
|
||||||
Add support for libbsc a high-performance block sorting compressor.
|
Add support for libbsc a high-performance block sorting compressor.
|
||||||
Enable external algorithm threading for single chunk compressed files.
|
Enable external algorithm threading for single chunk compressed files.
|
||||||
|
@ -109,6 +178,7 @@ Slight improvement to similarity computation
|
||||||
A simple mechanism to include DEBUG mode stats
|
A simple mechanism to include DEBUG mode stats
|
||||||
Implement secondary sketch based on character counts to refine similarity checksum.
|
Implement secondary sketch based on character counts to refine similarity checksum.
|
||||||
Proper checksum update for last block.
|
Proper checksum update for last block.
|
||||||
|
|
||||||
== 0.7 Beta release ==
|
== 0.7 Beta release ==
|
||||||
Bump version to 0.7
|
Bump version to 0.7
|
||||||
Fix handling of compression flags in adaptive mode
|
Fix handling of compression flags in adaptive mode
|
||||||
|
|
|
@ -106,8 +106,8 @@ NOTE: The option "libbsc" uses Ilya Grebnov's block sorting compression library
|
||||||
for data containing tables of numerical values especially if those are
|
for data containing tables of numerical values especially if those are
|
||||||
in an arithmetic series. In this implementation basic Delta Encoding is
|
in an arithmetic series. In this implementation basic Delta Encoding is
|
||||||
combined with Run-Length encoding and Matrix transpose
|
combined with Run-Length encoding and Matrix transpose
|
||||||
NOTE - If data has mixed textual and numeric table components then both -L and
|
NOTE - Both -L and -P can be used together to give maximum benefit on most
|
||||||
-P can be used together.
|
datasets.
|
||||||
|
|
||||||
'-S' <cksum>
|
'-S' <cksum>
|
||||||
- Specify chunk checksum to use: CRC64, SKEIN256, SKEIN512, SHA256 and
|
- Specify chunk checksum to use: CRC64, SKEIN256, SKEIN512, SHA256 and
|
||||||
|
@ -125,6 +125,8 @@ NOTE: The option "libbsc" uses Ilya Grebnov's block sorting compression library
|
||||||
gives lower dedupe ratio than content-aware dedupe (-D) and does not
|
gives lower dedupe ratio than content-aware dedupe (-D) and does not
|
||||||
support delta compression.
|
support delta compression.
|
||||||
|
|
||||||
|
'-B' <1..5>
|
||||||
|
- Specify an average Dedupe block size. 1 - 4K, 2 - 8K ... 5 - 64K.
|
||||||
'-M' - Display memory allocator statistics
|
'-M' - Display memory allocator statistics
|
||||||
'-C' - Display compression statistics
|
'-C' - Display compression statistics
|
||||||
|
|
||||||
|
|
4
main.c
4
main.c
|
@ -155,8 +155,8 @@ usage(void)
|
||||||
" '-P' - Enable Adaptive Delta Encoding. It can improve compresion ratio for\n"
|
" '-P' - Enable Adaptive Delta Encoding. It can improve compresion ratio for\n"
|
||||||
" data containing tables of numerical values especially if those are in\n"
|
" data containing tables of numerical values especially if those are in\n"
|
||||||
" an arithmetic series.\n"
|
" an arithmetic series.\n"
|
||||||
" NOTE - If data has mixed textual and numeric table components then both -L and\n"
|
" NOTE - Both -L and -P can be used together to give maximum benefit on most.\n"
|
||||||
" -P can be used together.\n"
|
" datasets.\n"
|
||||||
" '-S' <cksum>\n"
|
" '-S' <cksum>\n"
|
||||||
" - Specify chunk checksum to use: CRC64, SKEIN256, SKEIN512, SHA256 and\n"
|
" - Specify chunk checksum to use: CRC64, SKEIN256, SKEIN512, SHA256 and\n"
|
||||||
" SHA512. Default one is SKEIN256.\n"
|
" SHA512. Default one is SKEIN256.\n"
|
||||||
|
|
|
@ -42,7 +42,7 @@ extern "C" {
|
||||||
#define FLAG_DEDUP 1
|
#define FLAG_DEDUP 1
|
||||||
#define FLAG_DEDUP_FIXED 2
|
#define FLAG_DEDUP_FIXED 2
|
||||||
#define FLAG_SINGLE_CHUNK 4
|
#define FLAG_SINGLE_CHUNK 4
|
||||||
#define UTILITY_VERSION "1.1.0"
|
#define UTILITY_VERSION "1.2.0"
|
||||||
#define MASK_CRYPTO_ALG 0x30
|
#define MASK_CRYPTO_ALG 0x30
|
||||||
#define MAX_LEVEL 14
|
#define MAX_LEVEL 14
|
||||||
|
|
||||||
|
|
Loading…
Reference in a new issue