Moinak Ghosh
5bb028fe03
Change Segmented Dedupe flow to improve parallelism.
...
Periodically sync writes to segcache file.
Use simple insertion sort for small numbers of elements.
2013-04-25 23:42:32 +05:30
Moinak Ghosh
c0b4aa0116
Many optimizations and changes to Segmented Global Dedupe.
...
Use chunk hash based similarity matching rather than content based.
Use sorting to order hash buffer rather than min-heap for better accuracy.
Use fast CRC64 for similarity hash for speed and lower memory requirements.
2013-04-21 18:11:16 +05:30
Moinak Ghosh
3b8a5813fd
Many optimizations to segmented global dedupe.
...
Use chunk hash based cumulative similarity matching instead of chunk content.
2013-04-19 22:51:51 +05:30
Moinak Ghosh
8ae571124d
Complete implementation for Segmented Global Deduplication.
2013-04-18 21:26:24 +05:30
Moinak Ghosh
a22b52cf08
Work in progress changes for Segmented Global Deduplication.
2013-04-14 23:51:54 +05:30
Moinak Ghosh
50251107de
Work in progress changes for Segmented Global Deduplication.
2013-04-09 22:23:51 +05:30
