greg/pcompress
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity
151 commits 3 branches 21 tags 7.6 MiB
Commit graph

151 commits

This branch
This branch
All branches
Author SHA1 Message Date
Moinak Ghosh
d597f0f05c Major enhancements to Delta2 encoding. 
Avoid transposing below-threshold spans. Reduces compression ratio.
Use little-endian storage format for numbers to optimize for x86.
Improve embedded table detection.
Reduce header sizes.
Get rid of Gcc's LTO flag. Causes a performance drop.
Fix preprocessing behavior when LZP does not compress but Delta2 works.
 2012-12-23 23:50:45 +05:30
Moinak Ghosh
a43fdd7d2c Improve Delta2 scanning speed and effectiveness. 
Add destination buffer overflow check in Delta2.
Add rough speed computation.
 2012-12-23 00:44:56 +05:30
Moinak Ghosh
765b014018 More tweaks to Delta2 implementation. 2012-12-21 22:58:30 +05:30
Moinak Ghosh
9bd8146e88 Improve Delta2 performance. 2012-12-20 23:49:44 +05:30
Moinak Ghosh
e9c3b34ff5 Add debug build verbose messages to delta code. 2012-12-19 22:42:55 +05:30
Moinak Ghosh
c2df666daf Major improvement to Delta2 encoding algorithm. 2012-12-18 23:32:55 +05:30
Moinak Ghosh
1fe5157eec Ability to run individual test suites from makefile. 2012-12-17 20:59:27 +05:30
Moinak Ghosh
00c8c534d0 Silence more Gcc warnings. 
Update testing note.
 2012-12-17 20:23:30 +05:30
Moinak Ghosh
25bb54e3bc Improve 64-bit compiler and platform checks. 
Ensure core dumps are enabled during testing.
 2012-12-17 18:37:27 +05:30
Moinak Ghosh
fb30b5c295 Enable building with alternate Zlib and Bzlib. 
Update README and comments.
Fix correct setting of output size when using Delta2 without LZP.
 2012-12-16 23:17:04 +05:30
Moinak Ghosh
5ac47db6d5 README formatting. 2012-12-15 22:09:07 +05:30
Moinak Ghosh
ef0191729e Make Delta2 encoding independent of LZP. 
Tweak Delta2 parameters.
Update README and test cases.
 2012-12-15 22:03:23 +05:30
Moinak Ghosh
b01d255f6c Update README to align with current features/behavior. 2012-12-15 12:42:46 +05:30
Moinak Ghosh
a98778d62f Fine tune transpose parameters. 
Fix minor nits.
 2012-12-14 19:12:48 +05:30
Moinak Ghosh
b0f41c2888 Add matrix transpose to Delta2 encoding. 
Change confusing structure member name.
 2012-12-13 21:18:16 +05:30
Moinak Ghosh
375ebefa0d Add Matrix Transpose of Dedupe index to compress it better. 
Fix handling of Dedupe index compression failure.
 2012-12-13 00:00:47 +05:30
Moinak Ghosh
c7b960e72c Enable use of Link Time Optimizations in GCC. 2012-12-11 17:58:23 +05:30
Moinak Ghosh
42858c10a2 Ensure intermediate file cleanup in tests. 2012-12-11 16:23:35 +05:30
Moinak Ghosh
03840b31c5 Update to latest LZ4. 
Update a couple of comments.
 2012-12-11 11:38:42 +05:30
Moinak Ghosh
224fb529e9 Get rid of size_t in places where 64-bitness is assumed. 2012-12-09 10:15:06 +05:30
Moinak Ghosh
75c81b5e9c Add support for 64-bit Keccak implementation. 
Sanitize error message from tests.
Add more tests.
Improve platform detection in config script.
 2012-12-08 14:19:01 +05:30
Moinak Ghosh
970d8163a2 Add adaptive delta encoding test. 2012-12-05 22:03:09 +05:30
Moinak Ghosh
24d5368f7f Update README. 2012-12-05 00:12:29 +05:30
Moinak Ghosh
29b0d8fd7b Implement Adaptive Delta Encoding. 2012-12-05 00:09:47 +05:30
Moinak Ghosh
51249c858d Work in progress Adaptive Delta Encoding. 2012-12-03 23:45:41 +05:30
Moinak Ghosh
6c3173f929 Work in progress global dedupe config setup. 2012-11-29 22:28:50 +05:30
Moinak Ghosh
1f0c237495 Update Change and bump version for 1.1 release. 2012-11-26 20:27:09 +05:30
Moinak Ghosh
33c727e6e7 Fix building without Libbsc support. 
Add more tests for corrupted encrypted files.
 2012-11-26 20:21:03 +05:30
Moinak Ghosh
deecbadac4 Fix thread error reporting. 
Update error condition tests to not truncate archive.
 2012-11-25 15:12:45 +05:30
Moinak Ghosh
d250322490 Fix issues with error handling. 
Add new tests for out of range values and corrupted file.
 2012-11-24 23:53:07 +05:30
Moinak Ghosh
bdf16c4cb9 Avoid CRC32 when decompressing older version archives. 2012-11-24 00:00:05 +05:30
Moinak Ghosh
817e16cce6 Use HMAC on header and encrypted data, avoid regular digest when encrypting. 2012-11-23 22:27:14 +05:30
Moinak Ghosh
6e9ca97bdf Add CRC32 header checksums in non-cryptographic mode. 2012-11-23 00:00:39 +05:30
Moinak Ghosh
d054e0f713 Zlib optimizations. Use raw deflate streams to avoid unnecessary adler32. 
Change some function signatures to improve algo init function behavior.
Fix corner case dedupe bug in error handling flow.
Bump archive version signature.
 2012-11-22 21:02:50 +05:30
Moinak Ghosh
3b1d6b55fe Work in progress global dedupe config loader. 2012-11-19 21:41:56 +05:30
Moinak Ghosh
2909a3abff Update README. 2012-11-18 23:19:22 +05:30
Moinak Ghosh
393ced991a A couple of minor cleanups. 2012-11-18 20:20:16 +05:30
Moinak Ghosh
b2cbf0699e Use fixed rolling-hash mask for better block size approximation. 2012-11-11 14:58:39 +05:30
Moinak Ghosh
4d2e8ba644 Update Changelog for 1.0 release. 
Update version string and install doc.
Add Fixed-Block Deduplication tests.
 2012-11-09 21:42:56 +05:30
Moinak Ghosh
f9a15972d6 Add some more crypto test cases. 2012-11-09 21:17:06 +05:30
Moinak Ghosh
5b1a13480b Fix chunk flag setup when compression fails in adaptive mode. 
Prevent display of non-fatal errors during compression.
 2012-11-09 21:07:23 +05:30
Moinak Ghosh
77db54e712 Add buffer overflow check in Ppmd compression routines. 
Fix pipe mode encryption check.
Change file difference check in tests.
Add more tests.
 2012-11-09 20:05:13 +05:30
Moinak Ghosh
da9083ae12 Fix encryption with adaptive modes. 
Add missing zero-out of algorithm data field.
 2012-11-09 18:53:48 +05:30
Moinak Ghosh
7debc60dd5 Add a functionality test suite. 2012-11-08 22:22:37 +05:30
Moinak Ghosh
eacbf207aa Tweak chunking parameters for better block size distribution and dedupe ratio. 2012-11-08 19:41:33 +05:30
Moinak Ghosh
e437390e53 Add some more debug mode info. 2012-11-07 22:55:14 +05:30
Moinak Ghosh
3aa33f5b94 Minor fix for adapt mode. 2012-11-04 21:46:04 +05:30
Moinak Ghosh
f0c7ba87a3 Use Libbsc for XML data in adapt2 mode. 2012-11-04 21:13:26 +05:30
Moinak Ghosh
ddda9e5320 Bump version to 0.9.1 
Update Changelog.
 2012-10-24 18:46:27 +05:30
Moinak Ghosh
c5ebe1f30a Portability to Debian based distros. 
Enable SSE4/AVX detection for AMD platforms (Bulldozer has both).
Portable long long int print formatting to silence gcc 4.6 warnings.
 2012-10-21 21:03:07 +05:30