greg/pcompress
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

Commit graph

  • c6e779c400 Merge pull request #46 from xmikus01/patch-1 master Moinak Ghosh 2015-10-03 10:12:11 +0530
  • c6d91cb32e Remove bashisms in config Petr Mikusek 2015-09-23 13:51:54 +0200
  • 8ff1647cf5 Merge pull request #44 from v50110/patch-2 Moinak Ghosh 2015-08-16 21:30:43 +0530
  • 1bac03c136 Merge pull request #43 from v50110/patch-1 Moinak Ghosh 2015-08-16 21:29:16 +0530
  • 9f69afaa2f Adds missing check for pctx->main_cancel vales 2015-07-23 11:36:55 +0500
  • 0bca8f75bb Make ./config GCC 5.x aware vales 2015-07-23 11:19:31 +0500
  • c52306fe6d Merge pull request #38 from szepeviktor/patch-1 Moinak Ghosh 2015-06-14 16:13:00 +0530
  • 3459026b36 Updating options Viktor Szépe 2015-04-29 13:19:24 +0200
  • 6a757ddb2c Multitue of tweaks and improvements. * Use BSC for PNM type and Markup containing binary data. * Change thresholds in analyzer. * Properly use double precision in analyzer for accuracy. * Indicate BSC processing of packPNM output * Bring back raw-block Dispack for file not processed by Dispack filter. Moinak Ghosh 2015-03-22 23:36:04 +0530
  • 4360c5581f Fix regression in handling of incompressible chunks. Attempt to properly signal exit on decompression error (not working yet). Moinak Ghosh 2015-02-09 23:00:06 -0800
  • 7ef20ec5be Specialized dictionary encoding for FASTA files. Moinak Ghosh 2015-02-01 16:20:03 -0800
  • 30dee9a1a9 Improve check in E8E9 filter to avoid unnecessary encodes. Allow small increase in output chunk size for transform preprocessing (E8E9). Ensure chunk allocations always include oerhead size. Moinak Ghosh 2015-01-25 23:17:23 -0800
  • 678a6a2da4 A few small fixes. Effect same compression algo for Jpeg and PackJPG output. Fix compiler warning in PackPNM. Allow unknown type (0) to be specified for Dispack output (for analyzer). Moinak Ghosh 2015-01-17 20:03:06 +0530
  • d5e1d2cdef Some fixes in the Dictionary preprocessor. Fix checking of data type flags. Allow file-level filters to change output data type. Tweak analyzer threshold for markup type. Moinak Ghosh 2015-01-13 19:59:09 +0530
  • 077da83d5d A bunch of small fixes in Dict. Improve text analysis for markup tags. Use Libbsc for plain text and PPMd for markup mixed text. Change thresholds. Moinak Ghosh 2015-01-11 17:36:46 +0530
  • 66a482c968 A new Dictionary preprocessor for text files. Moinak Ghosh 2015-01-09 22:13:24 +0530
  • 73307c3996 Multiple checks and balances in Dispack to avoid buffer overlfow. Allow filter variants to omit the standard header. Use E8E9 in Dispack filter as a fallback. Fix integer overflow for type value in thread data struct. Do not inline functions in DEBUG build. Moinak Ghosh 2014-12-21 14:13:58 +0530
  • 1db822d866 Add Dispack file-level filter in the libarchive chain. Add new file type for Win32-PE executables (Dispack). Reset file type flag after filter processing for better compression. Fix array index handling for file type list. Moinak Ghosh 2014-12-20 11:24:09 +0530
  • 9a5361f010 Misclleanous cleanups. Fix some formatting. Increase metadata chunk size slightly. Moinak Ghosh 2014-12-19 19:15:39 +0530
  • 2cd41ec257 Revamp Filter handling code. 1) Really avoid adding filter xattr for non-processed files. 2) Clean up filter error handling. 3) Avoid libarchive data writes in filter callbacks. 4) Have libarchive data writes in a single place. 5) Properly handle skipping filter processing for a file. 6) Fix temporary file pathname handling. Moinak Ghosh 2014-12-14 23:37:40 +0530
  • dfe18ef48f Fix missed archive entry record. Fix enabling of metadata stream feature. Fix log message text. Use macro for path separator. Moinak Ghosh 2014-12-11 23:16:26 +0530
  • f970b41e34 A bunch of improvements and fixes. - Fix heap corruption in DICT Filter. - Make default Dedup block size as 8KB. - Revamp executable file handling: Part#1. - Developed new E8E9 filter that works better than Dispack on raw data blocks. - Remove block-based Dispack encoding. File-specific Dispack filter to be added. - Improve file header based executable file detection. - Introduce new sorting algorithm for filenames without extension. Moinak Ghosh 2014-12-11 19:15:36 +0530
  • 4c62e4db60 Fix buffer size handling in metadata chunks. Moinak Ghosh 2014-11-29 13:18:01 +0530
  • 753360e479 Tweak some data type settings. Moinak Ghosh 2014-11-19 20:19:16 +0530
  • b257c83f33 Detect a few mozilla file signatures. Add missing option to suppress pathname sorting. Fix chunk sizing to properly auto-enable deduplication. Fix default dedupe block size to 8KB. Moinak Ghosh 2014-11-16 22:57:47 +0530
  • 29b5efc988 Add couple of mozilla file extensions. Check for files > INT64_T when sorting. Makefile targets to help development. Moinak Ghosh 2014-11-15 19:17:33 +0530
  • cfe2cea544 Add a couple of prefetch in LZMA for a 4%-18% improvement. Cleanup some whitespace. Moinak Ghosh 2014-11-09 22:04:34 +0530
  • 507e7c75d3 Centralise data analysis routine for optimum performance and leverage. Utilise buffer data analysis for preprocessing filters. Moinak Ghosh 2014-11-06 22:23:33 +0530
  • 848010fbb5 Tweak LZP and Dict to mostly avoid non-text files. Moinak Ghosh 2014-11-05 22:05:19 +0530
  • 62c7590f26 Detiled listing of archive members (-i). Moinak Ghosh 2014-11-04 00:36:18 +0530
  • 5a742b94f0 Add checksum verification of metadata chunks. Moinak Ghosh 2014-11-03 23:32:12 +0530
  • d63f848a24 Fix encryption of metadata chunks. Moinak Ghosh 2014-11-03 23:22:22 +0530
  • ffa7f9cff8 Evict libiconv for Linux plaforms. Moinak Ghosh 2014-11-03 21:14:23 +0530
  • b2ad225fbb iImplement fast TOC listing for metadata streams. Fix help text. Removed redundant allocator code. Actually free memory on exit. Moinak Ghosh 2014-11-03 20:20:05 +0530
  • 3259c7ced0 Fix sub-type count. Moinak Ghosh 2014-10-27 21:23:03 +0530
  • b7804a0caa Improve file sorting algorithm. Add more file extension names. Fix data type mask size. Moinak Ghosh 2014-10-27 19:23:03 +0530
  • cc68550670 Add metadata stream flag for archive. Change flag bit to not collide with checksum id. Handle '-T' option properly. Moinak Ghosh 2014-10-25 22:57:31 +0530
  • e7081eb5a3 Git commit - rehash. Incorrect earlier commit. Implement Separate metadata stream. Fix blatant wrong check in Bzip2 compressor. Implement E8E9 filter fallback in Dispack. Improve dict buffer size checks. Reduce thread count to control memory usage in archive mode. Moinak Ghosh 2014-10-24 23:30:40 +0530
  • 1a00613457 Changes for future feature (disabled now): Separate metadata stream. Fix blatant wrong check in Bzip2 compressor. Implement E8E9 filter fallback in Dispack. Improve dict buffer size checks. Reduce thread count to control memory usage in archive mode. Moinak Ghosh 2014-10-24 21:12:48 +0530
  • e3c32ed6d6 Remove unneeded archive writing function. Improve filter scratch buffer handling. Improve memory accounting. Remove delayed allocation when compressing. Allows better memory estimation. Some cstyle fixes. Moinak Ghosh 2014-09-24 21:54:36 +0530
  • 6ba9c4b721 Change EXE code detection process to CSC's Analyzer. Moinak Ghosh 2014-09-23 21:19:10 +0530
  • 4c3f3d5ee2 Libarchive tweaks to indicate data vs metadata reads and writes. Moinak Ghosh 2014-09-22 21:21:52 +0530
  • 3debf1340c Fix missing newline in error message. Moinak Ghosh 2014-09-21 22:08:10 +0530
  • 50643b1f93 Fix libarchive configure and makefiles for Linux build. Moinak Ghosh 2014-09-21 19:57:28 +0530
  • 3fedef7f7f Add missing file for clean libarchive build. Moinak Ghosh 2014-09-20 22:06:36 +0530
  • 2e5f2d8aab Make DICT filter useful. Improve data analysis in adaptive_compress. Moinak Ghosh 2014-09-20 21:49:06 +0530
  • 071a9e2b26 Update,simplify analyzer function to indicate text data for Dict filter. Fix archive header writing bug. Strip ^M chars from dict filter files. Include DICT preprocessing type. Fix a bunch of bugs found by Xcode. Moinak Ghosh 2014-09-20 12:49:00 +0530
  • 4fedebc607 Dict filter work in progress. Moinak Ghosh 2014-09-18 22:51:25 +0530
  • f34962f8cc Set Wavpack compression mode based on compression level. Moinak Ghosh 2014-09-17 21:43:00 +0530
  • af39994a59 Working Wavpack filter for compressing WAV filies. Improved error handling of filter routines. Improved verbose logging. Moinak Ghosh 2014-09-17 20:34:38 +0530
  • fd087a8949 Step 0 of adding WavPack filter - does not work yet. WAV file detection. Rename libarchive dir to be generic. Moinak Ghosh 2014-09-14 23:56:38 +0530
  • 5a875f3174 Regenerate extensions hash. Moinak Ghosh 2014-09-12 17:00:36 +0530
  • 3e9a46a602 Add tagging of filter-processed entries with custom XATTR. Add magic number based detection of JPEG and PNM formats. Moinak Ghosh 2014-09-11 20:29:53 +0530
  • 08a30d270d Add caller-defined xattr handling features to private libarchive. Moinak Ghosh 2014-09-11 19:17:02 +0530
  • 9ecbbbafd0 Pull in private copy of libarchive to add pcmpress-specific functionality. First step to add packPNM support. Moinak Ghosh 2014-09-11 18:34:43 +0530
  • 9feee0a36d Add MPLv2 license forking script. Moinak Ghosh 2014-08-28 23:06:02 +0530
  • 376a56622b Several fixes for issue #21. Moinak Ghosh 2014-08-28 22:48:36 +0530
  • d5ceda559e Update Licensing notes and build notes. More whitespace fixes. Moinak Ghosh 2014-07-26 15:28:40 +0530
  • c1411a6af6 More whitespace cleanup and MPLv2 licensing support. Moinak Ghosh 2014-07-24 23:48:42 +0530
  • 10f40e1c6f Part 1 changes to allow dual licensing to MPLV2. Make external LGPL code/features disabled in MPLV2 variant. Nuke some unwanted whitespace (cstyle). Moinak Ghosh 2014-07-24 22:20:30 +0530
  • 0433452b37 Miscellaneous refactoring. Add some headers for OSX. Moinak Ghosh 2014-05-24 23:52:30 +0530
  • a6676698fa Configurable optimization level for libbsc. Moinak Ghosh 2014-05-09 22:58:51 +0530
  • 63509327e4 Tweak optimization flags for libbsc. Reduce max opt level on OS X to avoid crashes due to Gcc code generation quirks. Moinak Ghosh 2014-05-09 22:53:47 +0530
  • a62e1aa5f7 Config script option to disable AVX. Fix compiler (Gcc 4.9) warnings. Moinak Ghosh 2014-05-05 19:40:53 +0530
  • 125828b6b3 Update CPUID detection for AVX2. Moinak Ghosh 2014-05-04 22:35:12 +0530
  • 7055a8fc71 Add AVX detection and usage of -mavx[2] flags. Force GCC to use Clang assembler always on MAC OS X for AVX support. Moinak Ghosh 2014-05-04 22:03:40 +0530
  • 63bef473cc Working MAC OS X port. Compatibility layer for semaphore handling. Moinak Ghosh 2014-05-04 21:11:31 +0530
  • 7830473c65 Avoid cleaning buildtmp for make clean. Causes subsequent make to fail. Moinak Ghosh 2014-05-04 19:55:01 +0530
  • 9cbdd7ce8c Fix Libbsc and dynamic lib builds. Moinak Ghosh 2014-05-04 14:45:07 +0530
  • 4a9cd8c48e More portability tweaks. Fix compiler warnings. Moinak Ghosh 2014-05-04 13:32:11 +0530
  • 4b037f0ed7 More portability tweaks. Moinak Ghosh 2014-05-04 11:54:33 +0530
  • f2da433188 More portability tweaks. Handle platform specific yasm parameters. Resolve namespace conflict on OSX. Do not build Skein ASM code on OSX. Moinak Ghosh 2014-04-30 22:46:24 +0530
  • 6fba8aa8ac More OSX compatibility code. Fix new warnings with Gcc 4.8. Moinak Ghosh 2014-04-28 00:12:51 +0530
  • 8b52f7ccb8 Some initial tidbits for the start of an OSX/Windows port. Moinak Ghosh 2014-04-26 23:05:14 +0530
  • 935717373b Capability to list offset and length of each block when deduplication for external use. Moinak Ghosh 2014-03-30 17:35:21 +0530
  • 2f11e11717 Bump Version and update Changelog for 3.1 Release. 3.1Bugfix Moinak Ghosh 2014-03-02 18:46:03 +0530
  • c15957b990 Avoid auto-selection variable chunking for buffer sizes below threshold. Moinak Ghosh 2014-03-02 17:13:31 +0530
  • 9d40f3c2fb Do not auto-select Global Dedupe for below threshold buffers. Moinak Ghosh 2014-02-22 23:34:26 +0530
  • 8a1f47917f Fix issue #18. Do not try to generate a target filename in pipe mode. Moinak Ghosh 2014-02-05 23:43:07 +0530
  • 2702544d3f Scale default compression buffer size for levels > 8. Moinak Ghosh 2014-01-29 23:27:53 +0530
  • 33281a2257 Fix issue #17. Use LZ4 and Libbsc extra padding space for the compression buffer in adaptive modes. Moinak Ghosh 2014-01-29 00:12:04 +0530
  • 518ecf23a7 Fix issue #16. Moinak Ghosh 2014-01-28 20:44:45 +0530
  • f8d3ddfe39 Fix issue #15. Moinak Ghosh 2014-01-15 22:42:18 +0530
  • 31b1f56902 Fix issue #14 Updated fix for issue #12 Add check for correct Libarchive version. Update header checks to abort if not found. Moinak Ghosh 2014-01-14 21:59:32 +0530
  • ba1de93514 Update installation doc. The '--help' option to config should not generate a Makefile. 3.0Beta Moinak Ghosh 2014-01-12 21:52:27 +0530
  • 16a8d2fcf4 Update Changelog for new release. Moinak Ghosh 2014-01-12 21:03:25 +0530
  • 62568e9066 Basic capability to list contents of an archive without extracting to disk. Moinak Ghosh 2014-01-12 20:38:20 +0530
  • 3ddaf6d45f Bump version and update command help text. Moinak Ghosh 2014-01-04 21:45:23 +0530
  • 16da0b0339 Fix handling of some options. Update README with additional option details. Moinak Ghosh 2014-01-03 22:51:02 +0530
  • aef48f715f Change nftw() to depth-first scan to handle restoring directory permissions correctly. When sorting cause directories to be sorted after files and in descending order of nesting level. Take out stray printf(). Moinak Ghosh 2014-01-01 21:38:17 +0530
  • 683c3e48b5 Detect some DICOM formats and use BSC for DICOM data. Moinak Ghosh 2014-01-01 19:44:58 +0530
  • ea345a902a Overhaul documentation part #1 Detect and handle uncompressed PDF files using libbsc. Force binary/text data detection for tar archives. Get rid of unnecessary CLI option. Add full pipeline mode check when archiving. Moinak Ghosh 2013-12-30 23:24:37 +0530
  • 35043a74b0 Update to PackJPG 2.5i. Fixes crash with malformed Jpeg. Moinak Ghosh 2013-12-28 21:56:03 +0530
  • f22a80f5e2 Really fix issue #12. Moinak Ghosh 2013-12-28 21:49:55 +0530
  • 4c75a2da48 Fix issue #12. Fix issue #13. Create output directory with correct mode. Fix the flow where pathname list is not sorted. Fix ppmd decompression bug introduced in previous commit. Reduce compression level for automatic pathname sorting. Change to extraction directory only after opening archive. Moinak Ghosh 2013-12-27 23:49:47 +0530
  • a022a958c3 Free PPMD buffer after compression, rather than caching. Introduce new API in allocator to release buffer to OS. Release LZMA buffers after use. Moinak Ghosh 2013-12-21 23:32:27 +0530
  • 271414535e Drastically reduce memory consumption of PPMD8 in adaptive mode (Use lower max model order). Moinak Ghosh 2013-12-21 20:42:38 +0530
  • 5521955a94 Detect AR archives and set the type. Re-use a less common type code for AR. Use Dispack generically for all executables and AR archives. Moinak Ghosh 2013-12-18 23:00:39 +0530
  • a741f34f78 Move MSDOS COM single-byte magic number checks to last in the list. Move advanced options flag into context structure. Include dtd files as text type. Moinak Ghosh 2013-12-18 00:09:32 +0530
  • dcc64d9e47 Update PackJPG to version 2.5h. Fix missing bounds checking in Delta2. Moinak Ghosh 2013-12-14 20:58:59 +0530