Moinak Ghosh
26a4f42506
Introduce strict compiler flags and fix scores of warnings/issues.
...
Avoid different optimization flags for Dedupe sources.
Fix liberal mixing of uint64_t and int64_t (should all be uint64_t).
Fix corner case crash when decompressing.
2012-12-27 23:06:48 +05:30
Moinak Ghosh
d597f0f05c
Major enhancements to Delta2 encoding.
...
Avoid transposing below-threshold spans. Reduces compression ratio.
Use little-endian storage format for numbers to optimize for x86.
Improve embedded table detection.
Reduce header sizes.
Get rid of Gcc's LTO flag. Causes a performance drop.
Fix preprocessing behavior when LZP does not compress but Delta2 works.
2012-12-23 23:50:45 +05:30
Moinak Ghosh
1fe5157eec
Ability to run individual test suites from makefile.
2012-12-17 20:59:27 +05:30
Moinak Ghosh
25bb54e3bc
Improve 64-bit compiler and platform checks.
...
Ensure core dumps are enabled during testing.
2012-12-17 18:37:27 +05:30
Moinak Ghosh
fb30b5c295
Enable building with alternate Zlib and Bzlib.
...
Update README and comments.
Fix correct setting of output size when using Delta2 without LZP.
2012-12-16 23:17:04 +05:30
Moinak Ghosh
375ebefa0d
Add Matrix Transpose of Dedupe index to compress it better.
...
Fix handling of Dedupe index compression failure.
2012-12-13 00:00:47 +05:30
Moinak Ghosh
c7b960e72c
Enable use of Link Time Optimizations in GCC.
2012-12-11 17:58:23 +05:30
Moinak Ghosh
75c81b5e9c
Add support for 64-bit Keccak implementation.
...
Sanitize error message from tests.
Add more tests.
Improve platform detection in config script.
2012-12-08 14:19:01 +05:30
Moinak Ghosh
970d8163a2
Add adaptive delta encoding test.
2012-12-05 22:03:09 +05:30
Moinak Ghosh
29b0d8fd7b
Implement Adaptive Delta Encoding.
2012-12-05 00:09:47 +05:30
Moinak Ghosh
6c3173f929
Work in progress global dedupe config setup.
2012-11-29 22:28:50 +05:30
Moinak Ghosh
6e9ca97bdf
Add CRC32 header checksums in non-cryptographic mode.
2012-11-23 00:00:39 +05:30
Moinak Ghosh
7debc60dd5
Add a functionality test suite.
2012-11-08 22:22:37 +05:30
Moinak Ghosh
c5ebe1f30a
Portability to Debian based distros.
...
Enable SSE4/AVX detection for AMD platforms (Bulldozer has both).
Portable long long int print formatting to silence gcc 4.6 warnings.
2012-10-21 21:03:07 +05:30
Moinak Ghosh
d13aad830e
Implement HMAC Functionality.
...
Refactor crypto utility functions in a separate file.
Add HMAC verification for file header.
Add a few extra input validation checks.
Include space in chunk header for HMAC.
2012-10-17 23:32:35 +05:30
Moinak Ghosh
f2d7bea902
Support for chunk-level encryption using AES and Scrypt based PBE.
...
Couple of minor fixes.
2012-10-15 12:10:00 +05:30
Moinak Ghosh
21cbef6d60
Incorporate SSE/AVX optimized Intel SHA-256 implementation.
...
Add support for runtime cpuid detection.
2012-10-06 20:51:23 +05:30
Moinak Ghosh
2578676b1b
Add support for SHA256 and SHA512 digests from OpenSSL library.
2012-10-03 22:43:02 +05:30
Moinak Ghosh
f3f472b860
Implement K-min-values Sketch for Similarity detection.
2012-09-11 20:26:36 +05:30
Moinak Ghosh
262566b59a
Add xxHash for Rabin block checksums, slightly faster than CRC64.
...
Fix missing initialization of character counts table.
Some file reorganization.
2012-09-02 20:40:32 +05:30
Moinak Ghosh
4ba840b255
Add ASM version of Skein for x64 platforms with auto-detection
...
Error checking for checksum flag when decompressing
Update comments and READMEs
2012-09-01 14:40:15 +05:30
Moinak Ghosh
eda312ce1e
Add support for Skein512 and Skein256 checksums
...
Import Skein code from NIST CD submission
Make checksum algorithms pluggable
Fix handling of huge buffers (>2GB) in LZP
Cleanup of some buffer sizing code
Speed up CRC64 calculation in dedupe chunking
2012-08-31 22:36:06 +05:30
Moinak Ghosh
d75535bc7e
Add support for libbsc a high-performance block sorting compressor.
...
Enable external algorithm threading for single chunk compressed files.
Update docs.
2012-08-27 21:51:55 +05:30
Moinak Ghosh
5f41057f9c
Add config script to generate Makefile based on flags.
...
Add install target and installation readme file.
A few comment changes.
2012-08-26 14:09:24 +05:30
