Commit graph

55 commits

Author SHA1 Message Date
Moinak Ghosh
8386e72566 Rewrite core dedupe logic to simplify code and improve performance.
Hashtable based chunk-level deduplication instead of Quicksort.
Fix a corner case bug in Dedupe decompression.
2012-09-23 14:57:09 +05:30
Moinak Ghosh
e3befd9e16 Add support for Fixed-Block deduplication.
More refactoring of symbol names.
2012-09-16 11:12:58 +05:30
Moinak Ghosh
a6b3719d89 Fix conditional in heapq function. 2012-09-11 21:59:08 +05:30
Moinak Ghosh
f3f472b860 Implement K-min-values Sketch for Similarity detection. 2012-09-11 20:26:36 +05:30
Moinak Ghosh
262566b59a Add xxHash for Rabin block checksums, slightly faster than CRC64.
Fix missing initialization of character counts table.
Some file reorganization.
2012-09-02 20:40:32 +05:30