Moinak Ghosh
262566b59a
Add xxHash for Rabin block checksums, slightly faster than CRC64.
...
Fix missing initialization of character counts table.
Some file reorganization.
2012-09-02 20:40:32 +05:30
Moinak Ghosh
4ba840b255
Add ASM version of Skein for x64 platforms with auto-detection
...
Error checking for checksum flag when decompressing
Update comments and READMEs
2012-09-01 14:40:15 +05:30
Moinak Ghosh
eda312ce1e
Add support for Skein512 and Skein256 checksums
...
Import Skein code from NIST CD submission
Make checksum algorithms pluggable
Fix handling of huge buffers (>2GB) in LZP
Cleanup of some buffer sizing code
Speed up CRC64 calculation in dedupe chunking
2012-08-31 22:36:06 +05:30
Moinak Ghosh
d75535bc7e
Add support for libbsc a high-performance block sorting compressor.
...
Enable external algorithm threading for single chunk compressed files.
Update docs.
2012-08-27 21:51:55 +05:30
Moinak Ghosh
9eac774eb1
Add multithreaded LZMA port from p7zip
...
Compute balanced thread count between chunk threads and algo threads
Generic way to handle querying algorithm parameters
Clean up unnecessary includes
2012-08-18 10:20:52 +05:30
Moinak Ghosh
2dadf411fa
Reduce memory consumption and improve performance in dedupe
...
Re-introduce crc64 for dedup blocks to avoid wasted memcpy-s
Restructe block array to be an array of pointers allocated on demand
Fix a corner case issue when splitting chunks at a dedup boundary
2012-08-17 11:03:02 +05:30
Moinak Ghosh
7ff2cb74c4
Increase the small size slabs a bit.
...
Move 64Bit integer hashing function to common file for use in other places.
2012-07-29 15:02:51 +05:30
Moinak Ghosh
94563a7ecd
Fix buffer size computation when allocating Rabin block array.
...
Reduce memory usage of Rabin block array.
Add an SSE optimization for bsdiff.
Move integer hashing function to utils file.
More updates to README.
2012-07-28 23:55:24 +05:30
Moinak Ghosh
0091a0da02
Remove debug messages.
...
Fix last segment detection.
2012-07-10 21:11:31 +05:30
Moinak Ghosh
010f49f412
Implement ability to partition chunks at the last rabin boundary instead of fixed size.
2012-07-08 21:44:08 +05:30
Moinak Ghosh
d3f5287ee5
Update License info to LGPLv3.
2012-07-07 22:18:29 +05:30
Moinak Ghosh
cbf9728278
Implement Deduplication based on Rabin Fingerprinting: work in progress.
...
Fix bug that prevented pipe mode from being used.
Allow building without specialized allocator.
Use basic optimize flag in debuig build.
2012-06-29 18:23:55 +05:30
Moinak Ghosh
733923cbf2
Add ability to adjust chunk boundary based on Rabin Fingerprinting to improve compression.
...
Remove unnecessary checks in compression loop.
2012-06-21 20:27:05 +05:30
Moinak Ghosh
e46d3d10b3
Add basic compression statistics and new option to display them.
2012-05-31 21:36:33 +05:30
Moinak Ghosh
07dfed7769
Initial Commit.
2012-05-28 20:19:29 +05:30
