Commit graph

8 commits

Author SHA1 Message Date
Moinak Ghosh
0433452b37 Miscellaneous refactoring.
Add some headers for OSX.
2014-05-24 23:52:30 +05:30
Moinak Ghosh
b23b5789fb Fix bugs and improve accuracy in Segmented Dedupe.
Fix segment hashlist size computation.
Remove unnecessary sync of segment hashlist file writes.
Pass correct number of threads to index creation routine.
Add more error checks.
Handle correct positioning of segment hashlist file offset on write error.
Add missing semaphore signaling at dedupe abort points with global dedupe.
Use closer min-values sampling for improved segmented dedupe accuracy.
Update proper checksum info in README.
2013-04-30 19:35:18 +05:30
Moinak Ghosh
5bb028fe03 Change Segmented Dedupe flow to improve parallelism.
Periodically sync writes to segcache file.
Use simple insertion sort for small numbers of elements.
2013-04-25 23:42:32 +05:30
Moinak Ghosh
c0b4aa0116 Many optimizations and changes to Segmented Global Dedupe.
Use chunk hash based similarity matching rather than content based.
Use sorting to order hash buffer rather than min-heap for better accuracy.
Use fast CRC64 for similarity hash for speed and lower memory requirements.
2013-04-21 18:11:16 +05:30
Moinak Ghosh
3b8a5813fd Many optimizations to segmented global dedupe.
Use chunk hash based cumulative similarity matching instead of chunk content.
2013-04-19 22:51:51 +05:30
Moinak Ghosh
8ae571124d Complete implementation for Segmented Global Deduplication. 2013-04-18 21:26:24 +05:30
Moinak Ghosh
a22b52cf08 Work in progress changes for Segmented Global Deduplication. 2013-04-14 23:51:54 +05:30
Moinak Ghosh
50251107de Work in progress changes for Segmented Global Deduplication. 2013-04-09 22:23:51 +05:30
Renamed from rabin/global/db.h (Browse further)