greg/pcompress
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity
410 commits 3 branches 21 tags 7.6 MiB
Commit graph

410 commits

This branch
This branch
All branches
Author SHA1 Message Date
Moinak Ghosh
935717373b Capability to list offset and length of each block when deduplication for external use. 2014-03-30 17:35:21 +05:30
Moinak Ghosh
2f11e11717 Bump Version and update Changelog for 3.1 Release. 2014-03-02 18:46:03 +05:30
Moinak Ghosh
c15957b990 Avoid auto-selection variable chunking for buffer sizes below threshold. 2014-03-02 17:13:31 +05:30
Moinak Ghosh
9d40f3c2fb Do not auto-select Global Dedupe for below threshold buffers. 2014-02-22 23:34:26 +05:30
Moinak Ghosh
8a1f47917f Fix issue #18. 
Do not try to generate a target filename in pipe mode.
 2014-02-05 23:43:07 +05:30
Moinak Ghosh
2702544d3f Scale default compression buffer size for levels > 8. 2014-01-29 23:27:53 +05:30
Moinak Ghosh
33281a2257 Fix issue #17. 
Use LZ4 and Libbsc extra padding space for the compression buffer in adaptive modes.
 2014-01-29 00:12:04 +05:30
Moinak Ghosh
518ecf23a7 Fix issue #16. 2014-01-28 20:44:45 +05:30
Moinak Ghosh
f8d3ddfe39 Fix issue #15. 2014-01-15 22:42:18 +05:30
Moinak Ghosh
31b1f56902 Fix issue #14 
Updated fix for issue #12
Add check for correct Libarchive version.
Update header checks to abort if not found.
 2014-01-14 21:59:32 +05:30
Moinak Ghosh
ba1de93514 Update installation doc. 
The '--help' option to config should not generate a Makefile.
 2014-01-12 21:52:27 +05:30
Moinak Ghosh
16a8d2fcf4 Update Changelog for new release. 2014-01-12 21:03:25 +05:30
Moinak Ghosh
62568e9066 Basic capability to list contents of an archive without extracting to disk. 2014-01-12 20:38:20 +05:30
Moinak Ghosh
3ddaf6d45f Bump version and update command help text. 2014-01-04 21:45:23 +05:30
Moinak Ghosh
16da0b0339 Fix handling of some options. 
Update README with additional option details.
 2014-01-03 22:51:02 +05:30
Moinak Ghosh
aef48f715f Change nftw() to depth-first scan to handle restoring directory permissions correctly. 
When sorting cause directories to be sorted after files and in descending order of nesting level.
Take out stray printf().
 2014-01-01 21:38:17 +05:30
Moinak Ghosh
683c3e48b5 Detect some DICOM formats and use BSC for DICOM data. 2014-01-01 19:44:58 +05:30
Moinak Ghosh
ea345a902a Overhaul documentation part #1 
Detect and handle uncompressed PDF files using libbsc.
Force binary/text data detection for tar archives.
Get rid of unnecessary CLI option.
Add full pipeline mode check when archiving.
 2013-12-30 23:24:37 +05:30
Moinak Ghosh
35043a74b0 Update to PackJPG 2.5i. 
Fixes crash with malformed Jpeg.
 2013-12-28 21:56:03 +05:30
Moinak Ghosh
f22a80f5e2 Really fix issue #12. 2013-12-28 21:49:55 +05:30
Moinak Ghosh
4c75a2da48 Fix issue #12. 
Fix issue #13.
Create output directory with correct mode.
Fix the flow where pathname list is not sorted.
Fix ppmd decompression bug introduced in previous commit.
Reduce compression level for automatic pathname sorting.
Change to extraction directory only after opening archive.
 2013-12-27 23:49:47 +05:30
Moinak Ghosh
a022a958c3 Free PPMD buffer after compression, rather than caching. 
Introduce new API in allocator to release buffer to OS.
Release LZMA buffers after use.
 2013-12-21 23:32:27 +05:30
Moinak Ghosh
271414535e Drastically reduce memory consumption of PPMD8 in adaptive mode (Use lower max model order). 2013-12-21 20:42:38 +05:30
Moinak Ghosh
5521955a94 Detect AR archives and set the type. 
Re-use a less common type code for AR.
Use Dispack generically for all executables and AR archives.
 2013-12-18 23:00:39 +05:30
Moinak Ghosh
a741f34f78 Move MSDOS COM single-byte magic number checks to last in the list. 
Move advanced options flag into context structure.
Include dtd files as text type.
 2013-12-18 00:09:32 +05:30
Moinak Ghosh
dcc64d9e47 Update PackJPG to version 2.5h. 
Fix missing bounds checking in Delta2.
 2013-12-14 20:58:59 +05:30
Moinak Ghosh
a851bac247 Check harder with more strides in Delta2 for extreme compression levels. 2013-12-13 19:53:14 +05:30
Moinak Ghosh
bb08b24989 Make LibArchive filter process buffer more generic. 
Include explicit CLI flags for PackJPG and Dispack.
Avoid auto-selection of filters if advanced options are specified.
 2013-12-12 00:22:15 +05:30
Moinak Ghosh
393fd790b0 Add more robust checks for Jpeg and packJPG format files in filter routine. 
Use case-insensitive checks for extension names.
Enable more features based on compression level, when archiving.
 2013-12-08 23:24:06 +05:30
Moinak Ghosh
733e6f8245 Do not use Libbsc for TIFFs. Not all TIFFs compress well with Libbsc. 
Fix DEBUG-STATS build for Dispack.
 2013-12-06 22:53:41 +05:30
Moinak Ghosh
36ed5d5a78 Use adapt2 as default compression in archive mode. 
Add more filter auto-selection by compression level in archive mode.
Replace odd stride lengths in Delta2 with standard numeric type lengths and improve performance.
 2013-12-05 22:20:01 +05:30
Moinak Ghosh
316d5aa4a8 Remove fast path exit to allow compressing headers and zero paddings via LZ4. 2013-12-04 23:05:18 +05:30
Moinak Ghosh
5e484f0694 Use libbsc for AVI and MP4 files. 2013-12-04 20:07:52 +05:30
Moinak Ghosh
3f62cdf7d5 Use Libbsc for MP4 and FLAC files. 
Change some rare file type codes to indicate some common types.
 2013-12-03 21:56:07 +05:30
Moinak Ghosh
958bdf7edc Use Libbsc for TIFF images. 
Workaround for packJPG limitation.
 2013-12-02 21:50:19 +05:30
Moinak Ghosh
5a49252bb9 Remove external Libbsc option. 2013-11-30 22:43:31 +05:30
Moinak Ghosh
fb25e53b4f Add forked and optimized copy of LGPL version of Libbsc. 
Strip out Sort Transform from Libbsc copy.
Reduce Libbsc memory use.
Avoid redundant adler32 of data block in Libbsc.
 2013-11-30 22:13:33 +05:30
Moinak Ghosh
c4c4b47138 Use Libbsc for DNA Sequence data instead of PPMD. Faster, better compression. 
Fix pz extension handling for real.
 2013-11-30 09:58:21 +05:30
Moinak Ghosh
dfeea8c19b Avoid Delta2,LZP for TIFF files. Negatively impacts compression. 2013-11-29 19:47:57 +05:30
Moinak Ghosh
306f145f22 Use libbsc/ppmd for BMP files. 
Fix extension based hashing.
Do not append .pz extension to filenames already having it.
Some code formatting changes.
 2013-11-28 22:42:51 +05:30
Moinak Ghosh
bd530e3393 Get rid of nagging warning. 2013-11-26 21:43:17 +05:30
Moinak Ghosh
7bf967b572 Fix PackJPG library usage. PackJPG interface doc is incomplete, ugh! 
Handle the case where PackJPG expands the file rather than compressing.
 2013-11-26 21:24:01 +05:30
Moinak Ghosh
4923551570 Fix Dispack decoding. 2013-11-24 22:44:35 +05:30
Moinak Ghosh
0192790c02 Add Dispack filter with auto-detection of x86 executables in archive mode. 
More elaborate magic header based detection of 32-bit and 64-bit x86 binaries.
Always use fast-mode LZ4 in Adaptive modes.
 2013-11-24 19:45:58 +05:30
Moinak Ghosh
1e2c3e479a Optimize preprocessed compression and avoid a bunch of memory copies. 
Fix a crash.
Add a few more file types.
More comments.
 2013-11-22 20:44:26 +05:30
Moinak Ghosh
664c8ef75b Fix fd leak. 2013-11-15 23:06:31 +05:30
Moinak Ghosh
c09a2b7b81 Fix issues when handling Jpegs where packJPG borks. 2013-11-15 23:02:09 +05:30
Moinak Ghosh
11584cab52 Add fast handling of totally incompressible data (like Jpegs) in adaptive modes. 
Add function to indicate totally incompressible data when archiving.
Reformat if statements in some places to reduce branching.
 2013-11-15 21:06:23 +05:30
Moinak Ghosh
c567a1d2f5 Enable auto-filtering of archive entries based on compression level. 
Miscellaneous fixes.
 2013-11-14 21:54:46 +05:30
Moinak Ghosh
e90c52e516 Work in progress changes for packJPG encoding and decoding. 
Enhance custom LibArchive filter functionlity.
 2013-11-13 23:28:01 +05:30