Commit graph

5 commits

Author SHA1 Message Date
Moinak Ghosh
c0b4aa0116 Many optimizations and changes to Segmented Global Dedupe.
Use chunk hash based similarity matching rather than content based.
Use sorting to order hash buffer rather than min-heap for better accuracy.
Use fast CRC64 for similarity hash for speed and lower memory requirements.
2013-04-21 18:11:16 +05:30
Moinak Ghosh
f8f23e5200 Major License text cleanup. 2013-03-07 20:26:48 +05:30
Moinak Ghosh
72b23dac1a Add AES-NI optimized code derived from latest OpenSSL upstream.
Add AES instruction set detection.
Add missing license headers to a few files.
2013-02-25 19:23:51 +05:30
Moinak Ghosh
d49a088eea Fixes and performance improvements for Dedupe Delta Compression
Avoid using fingerprints in minhash computation and fix write amplification
Modify min-heap to use 64bit values
Improve bsdiff performance
Fix pointer comparison in bsdiff
Use 32bit offsets in bsdiff to reduce memory usage
Improve Zero RLE Encoder performance
Add more buffer overflow checks in Zero RLE Decoder
2013-01-13 22:04:59 +05:30
Moinak Ghosh
f3f472b860 Implement K-min-values Sketch for Similarity detection. 2012-09-11 20:26:36 +05:30