Update Licensing notes and build notes.
More whitespace fixes.
This commit is contained in:
parent
c1411a6af6
commit
d5ceda559e
20 changed files with 55 additions and 29 deletions
8
INSTALL
8
INSTALL
|
@ -110,6 +110,14 @@ is not the usual GNU Autoconf script.
|
||||||
support is used. So a local static libarchive build with
|
support is used. So a local static libarchive build with
|
||||||
most features disabled can be used with this option.
|
most features disabled can be used with this option.
|
||||||
|
|
||||||
|
--with-external-libbsc=<path to libbsc source tree>
|
||||||
|
Enable building with exernal libbsc sources. In the LGPL licensed
|
||||||
|
sources a fork of the previous LGPL licensed libbsc source tree is
|
||||||
|
already included in the Pcompress repo. The same is not true for the
|
||||||
|
MPLv2 licensed sources. So libbsc has to be downloaded separately
|
||||||
|
from: http://libbsc.com/ and linked into Pcompress using this option.
|
||||||
|
The current libbsc sources are licensed with ASLv2.
|
||||||
|
|
||||||
--no-sse-detect Do not try to detect the CPU's SSE capability. This
|
--no-sse-detect Do not try to detect the CPU's SSE capability. This
|
||||||
mode will simply use SSE2 as the fallback default.
|
mode will simply use SSE2 as the fallback default.
|
||||||
Using SSE4 and later improves performance significantly.
|
Using SSE4 and later improves performance significantly.
|
||||||
|
|
18
README.md
18
README.md
|
@ -33,6 +33,16 @@ Pcompress also supports encryption via AES, Salsa20 and uses Scrypt from
|
||||||
Tarsnap for Password Based Key generation. A unique key is generated per
|
Tarsnap for Password Based Key generation. A unique key is generated per
|
||||||
session even if the same password is used and HMAC is used to do authentication.
|
session even if the same password is used and HMAC is used to do authentication.
|
||||||
|
|
||||||
|
LICENSING
|
||||||
|
=========
|
||||||
|
|
||||||
|
Pcompress is dual licensed with LGPLv3 and MPLv2 sources. The main git
|
||||||
|
repository is LGPL licensed. A separate tarball of the sources with
|
||||||
|
MPLv2 license is made available as a download. This is updated
|
||||||
|
periodically. Since Pcompress also integrates a bunch of third-party
|
||||||
|
software a few features may be missing in the MPLv2 licensed version
|
||||||
|
because of the upstream software being LGPL licensed originally.
|
||||||
|
|
||||||
Links of Interest
|
Links of Interest
|
||||||
=================
|
=================
|
||||||
|
|
||||||
|
@ -198,6 +208,12 @@ Compression Algorithms
|
||||||
However in a few cases this can produce slightly lesser compression
|
However in a few cases this can produce slightly lesser compression
|
||||||
gain.
|
gain.
|
||||||
|
|
||||||
|
libbsc - This is a new block-sorting compressor having much better effectiveness
|
||||||
|
and performance over a variety of data types as compared to Bzip2.
|
||||||
|
NOTE: In the LGPL licensed version libbsc is an integral part of Pcompress.
|
||||||
|
When building MPLv2 licensed sources, the libbsc sources must be
|
||||||
|
downloaded separately and linked in. This is described in the INSTALL file.
|
||||||
|
|
||||||
PPMD - Slow. Extreme compression for Text, average compression for binary.
|
PPMD - Slow. Extreme compression for Text, average compression for binary.
|
||||||
In addition PPMD decompression time is also high for large chunks.
|
In addition PPMD decompression time is also high for large chunks.
|
||||||
This requires lots of RAM similar to LZMA. PPMd requires
|
This requires lots of RAM similar to LZMA. PPMd requires
|
||||||
|
@ -232,6 +248,8 @@ Enabled features based on Compression Level
|
||||||
10 - 14 - Global Dedup block size 2KB, Adaptive Delta Encoding with extra rounds, Dispack,
|
10 - 14 - Global Dedup block size 2KB, Adaptive Delta Encoding with extra rounds, Dispack,
|
||||||
LZP Preprocessing, PackJPG filter for Jpegs.
|
LZP Preprocessing, PackJPG filter for Jpegs.
|
||||||
|
|
||||||
|
NOTE: - LZP Preprocessing and PackJPG are not available in the MPLv2 licensed version.
|
||||||
|
|
||||||
Encryption
|
Encryption
|
||||||
==========
|
==========
|
||||||
Pcompress supports encryption and authentication in both archive and single-file
|
Pcompress supports encryption and authentication in both archive and single-file
|
||||||
|
|
|
@ -20,7 +20,7 @@
|
||||||
* If not, see <http://www.gnu.org/licenses/>.
|
* If not, see <http://www.gnu.org/licenses/>.
|
||||||
*
|
*
|
||||||
* moinakg@belenix.org, http://moinakg.wordpress.com/
|
* moinakg@belenix.org, http://moinakg.wordpress.com/
|
||||||
*
|
*
|
||||||
*/
|
*/
|
||||||
|
|
||||||
#ifndef __ALLOCATOR_H__
|
#ifndef __ALLOCATOR_H__
|
||||||
|
|
|
@ -20,7 +20,7 @@
|
||||||
* If not, see <http://www.gnu.org/licenses/>.
|
* If not, see <http://www.gnu.org/licenses/>.
|
||||||
*
|
*
|
||||||
* moinakg@belenix.org, http://moinakg.wordpress.com/
|
* moinakg@belenix.org, http://moinakg.wordpress.com/
|
||||||
*
|
*
|
||||||
*/
|
*/
|
||||||
|
|
||||||
#include <sys/types.h>
|
#include <sys/types.h>
|
||||||
|
|
|
@ -20,7 +20,7 @@
|
||||||
* If not, see <http://www.gnu.org/licenses/>.
|
* If not, see <http://www.gnu.org/licenses/>.
|
||||||
*
|
*
|
||||||
* moinakg@belenix.org, http://moinakg.wordpress.com/
|
* moinakg@belenix.org, http://moinakg.wordpress.com/
|
||||||
*
|
*
|
||||||
*/
|
*/
|
||||||
|
|
||||||
#ifndef _CRYPTO_UTILS_H
|
#ifndef _CRYPTO_UTILS_H
|
||||||
|
@ -113,4 +113,4 @@ int hmac_cleanup(mac_ctx_t *mctx);
|
||||||
}
|
}
|
||||||
#endif
|
#endif
|
||||||
|
|
||||||
#endif
|
#endif
|
||||||
|
|
|
@ -20,7 +20,7 @@
|
||||||
* If not, see <http://www.gnu.org/licenses/>.
|
* If not, see <http://www.gnu.org/licenses/>.
|
||||||
*
|
*
|
||||||
* moinakg@belenix.org, http://moinakg.wordpress.com/
|
* moinakg@belenix.org, http://moinakg.wordpress.com/
|
||||||
*
|
*
|
||||||
*/
|
*/
|
||||||
|
|
||||||
#ifndef _SHA2_UTILS_H_
|
#ifndef _SHA2_UTILS_H_
|
||||||
|
|
|
@ -40,12 +40,12 @@
|
||||||
* Helper functions for single-call SHA3 (Keccak) hashing. Both serial
|
* Helper functions for single-call SHA3 (Keccak) hashing. Both serial
|
||||||
* and parallel versions are provided. Parallel versions use 2-stage
|
* and parallel versions are provided. Parallel versions use 2-stage
|
||||||
* Merkle Tree hashing.
|
* Merkle Tree hashing.
|
||||||
*
|
*
|
||||||
* At the leaf level data is split into BLKSZ blocks and 4 threads
|
* At the leaf level data is split into BLKSZ blocks and 4 threads
|
||||||
* compute 4 hashes of interleaved block streams. At 2nd level two
|
* compute 4 hashes of interleaved block streams. At 2nd level two
|
||||||
* new hashes are generated from hashing the 2 pairs of hash values.
|
* new hashes are generated from hashing the 2 pairs of hash values.
|
||||||
* In the final stage the 2 hash values are hashed to the final digest.
|
* In the final stage the 2 hash values are hashed to the final digest.
|
||||||
*
|
*
|
||||||
* References:
|
* References:
|
||||||
* http://eprint.iacr.org/2012/476.pdf
|
* http://eprint.iacr.org/2012/476.pdf
|
||||||
* http://gva.noekeon.org/papers/bdpv09tree.html
|
* http://gva.noekeon.org/papers/bdpv09tree.html
|
||||||
|
|
|
@ -20,7 +20,7 @@
|
||||||
* If not, see <http://www.gnu.org/licenses/>.
|
* If not, see <http://www.gnu.org/licenses/>.
|
||||||
*
|
*
|
||||||
* moinakg@belenix.org, http://moinakg.wordpress.com/
|
* moinakg@belenix.org, http://moinakg.wordpress.com/
|
||||||
*
|
*
|
||||||
*/
|
*/
|
||||||
|
|
||||||
#ifndef _SHA3_UTILS_H_
|
#ifndef _SHA3_UTILS_H_
|
||||||
|
|
|
@ -47,7 +47,7 @@
|
||||||
* 64-bit encoded value is of the following format
|
* 64-bit encoded value is of the following format
|
||||||
* Most Significant Byte = Stride length
|
* Most Significant Byte = Stride length
|
||||||
* Remaining Bytes = Number of bytes in the span
|
* Remaining Bytes = Number of bytes in the span
|
||||||
*
|
*
|
||||||
* We optimize for little-endian, so values are stored and interpreted
|
* We optimize for little-endian, so values are stored and interpreted
|
||||||
* in little-endian order.
|
* in little-endian order.
|
||||||
*/
|
*/
|
||||||
|
@ -142,7 +142,7 @@ delta2_encode(uchar_t *src, uint64_t srclen, uchar_t *dst, uint64_t *dstlen, int
|
||||||
uint64_t slen, sz, dsz, pending;
|
uint64_t slen, sz, dsz, pending;
|
||||||
int rem, lenc, hdr_ovr;
|
int rem, lenc, hdr_ovr;
|
||||||
DEBUG_STAT_EN(double strt, en);
|
DEBUG_STAT_EN(double strt, en);
|
||||||
|
|
||||||
srcpos = src;
|
srcpos = src;
|
||||||
dstpos = dst;
|
dstpos = dst;
|
||||||
dstend = dst + *dstlen;
|
dstend = dst + *dstlen;
|
||||||
|
|
|
@ -48,4 +48,4 @@ int delta2_decode(uchar_t *src, uint64_t srclen, uchar_t *dst, uint64_t *dstlen)
|
||||||
}
|
}
|
||||||
#endif
|
#endif
|
||||||
|
|
||||||
#endif
|
#endif
|
||||||
|
|
|
@ -155,7 +155,7 @@ using namespace std;
|
||||||
#define NORMAL_HDR (1 + 2)
|
#define NORMAL_HDR (1 + 2)
|
||||||
#define EXTENDED_HDR (1 + 2 + 2)
|
#define EXTENDED_HDR (1 + 2 + 2)
|
||||||
// Dispack min reduction should be 8%, otherwise we abort
|
// Dispack min reduction should be 8%, otherwise we abort
|
||||||
#define DIS_MIN_REDUCE (2622)
|
#define DIS_MIN_REDUCE (2622)
|
||||||
|
|
||||||
#define MAXINSTR 15 // maximum size of a single instruction in bytes (actually, decodeable ones are shorter)
|
#define MAXINSTR 15 // maximum size of a single instruction in bytes (actually, decodeable ones are shorter)
|
||||||
|
|
||||||
|
@ -223,7 +223,7 @@ enum Streams
|
||||||
ST_JUMP32, // 32-bit jump target
|
ST_JUMP32, // 32-bit jump target
|
||||||
|
|
||||||
ST_MAX,
|
ST_MAX,
|
||||||
|
|
||||||
// these components of the instruction stream are also identified
|
// these components of the instruction stream are also identified
|
||||||
// seperately, but stored together with another stream since there's
|
// seperately, but stored together with another stream since there's
|
||||||
// high correlation between them (or just because one streams provides
|
// high correlation between them (or just because one streams provides
|
||||||
|
@ -641,7 +641,7 @@ static sU8 *
|
||||||
DisFilter(DisFilterCtx &ctx, sU8 *src, sU32 size, sU32 origin, sU8 *dst, sU32 &outputSize)
|
DisFilter(DisFilterCtx &ctx, sU8 *src, sU32 size, sU32 origin, sU8 *dst, sU32 &outputSize)
|
||||||
{
|
{
|
||||||
// DisFilterCtx ctx(origin,origin+size);
|
// DisFilterCtx ctx(origin,origin+size);
|
||||||
|
|
||||||
// main loop: handle everything but the last few bytes
|
// main loop: handle everything but the last few bytes
|
||||||
sU32 pos = 0;
|
sU32 pos = 0;
|
||||||
while(pos < size - MAXINSTR)
|
while(pos < size - MAXINSTR)
|
||||||
|
@ -713,7 +713,7 @@ DisUnFilter(sU8 *source,sU32 sourceSize,sU8 *dest,sU32 destSize,sU32 memStart)
|
||||||
sU8 *stream[ST_MAX];
|
sU8 *stream[ST_MAX];
|
||||||
sU8 *streamEnd[ST_MAX];
|
sU8 *streamEnd[ST_MAX];
|
||||||
sU32 funcTable[256];
|
sU32 funcTable[256];
|
||||||
|
|
||||||
// read header (list of stream sizes)
|
// read header (list of stream sizes)
|
||||||
if(sourceSize < ST_MAX*4)
|
if(sourceSize < ST_MAX*4)
|
||||||
return sFALSE;
|
return sFALSE;
|
||||||
|
@ -735,7 +735,7 @@ DisUnFilter(sU8 *source,sU32 sourceSize,sU8 *dest,sU32 destSize,sU32 memStart)
|
||||||
funcTable[i] = 0;
|
funcTable[i] = 0;
|
||||||
|
|
||||||
sBool nextIsFunc = sTRUE;
|
sBool nextIsFunc = sTRUE;
|
||||||
|
|
||||||
sU8 *destStart = dest;
|
sU8 *destStart = dest;
|
||||||
sU8 *destEnd = destStart + destSize;
|
sU8 *destEnd = destStart + destSize;
|
||||||
|
|
||||||
|
@ -749,7 +749,7 @@ DisUnFilter(sU8 *source,sU32 sourceSize,sU8 *dest,sU32 destSize,sU32 memStart)
|
||||||
{
|
{
|
||||||
CheckSrc(ST_JUMPTBL_COUNT,1);
|
CheckSrc(ST_JUMPTBL_COUNT,1);
|
||||||
sInt count = Fetch8(stream[ST_JUMPTBL_COUNT]) + 1;
|
sInt count = Fetch8(stream[ST_JUMPTBL_COUNT]) + 1;
|
||||||
|
|
||||||
for(sInt i=0;i<count;i++)
|
for(sInt i=0;i<count;i++)
|
||||||
{
|
{
|
||||||
sU32 target;
|
sU32 target;
|
||||||
|
|
|
@ -45,4 +45,4 @@ void transpose(unsigned char *from, unsigned char *to, uint64_t buflen,
|
||||||
}
|
}
|
||||||
#endif
|
#endif
|
||||||
|
|
||||||
#endif
|
#endif
|
||||||
|
|
|
@ -20,7 +20,7 @@
|
||||||
* If not, see <http://www.gnu.org/licenses/>.
|
* If not, see <http://www.gnu.org/licenses/>.
|
||||||
*
|
*
|
||||||
* moinakg@belenix.org, http://moinakg.wordpress.com/
|
* moinakg@belenix.org, http://moinakg.wordpress.com/
|
||||||
*
|
*
|
||||||
*/
|
*/
|
||||||
|
|
||||||
#include <sys/types.h>
|
#include <sys/types.h>
|
||||||
|
|
|
@ -20,7 +20,7 @@
|
||||||
* If not, see <http://www.gnu.org/licenses/>.
|
* If not, see <http://www.gnu.org/licenses/>.
|
||||||
*
|
*
|
||||||
* moinakg@belenix.org, http://moinakg.wordpress.com/
|
* moinakg@belenix.org, http://moinakg.wordpress.com/
|
||||||
*
|
*
|
||||||
*/
|
*/
|
||||||
|
|
||||||
#include <sys/types.h>
|
#include <sys/types.h>
|
||||||
|
|
2
main.c
2
main.c
|
@ -20,7 +20,7 @@
|
||||||
* If not, see <http://www.gnu.org/licenses/>.
|
* If not, see <http://www.gnu.org/licenses/>.
|
||||||
*
|
*
|
||||||
* moinakg@belenix.org, http://moinakg.wordpress.com/
|
* moinakg@belenix.org, http://moinakg.wordpress.com/
|
||||||
*
|
*
|
||||||
*/
|
*/
|
||||||
|
|
||||||
/*
|
/*
|
||||||
|
|
|
@ -20,7 +20,7 @@
|
||||||
* If not, see <http://www.gnu.org/licenses/>.
|
* If not, see <http://www.gnu.org/licenses/>.
|
||||||
*
|
*
|
||||||
* moinakg@belenix.org, http://moinakg.wordpress.com/
|
* moinakg@belenix.org, http://moinakg.wordpress.com/
|
||||||
*
|
*
|
||||||
*/
|
*/
|
||||||
|
|
||||||
#include <sys/types.h>
|
#include <sys/types.h>
|
||||||
|
|
|
@ -20,7 +20,7 @@
|
||||||
* If not, see <http://www.gnu.org/licenses/>.
|
* If not, see <http://www.gnu.org/licenses/>.
|
||||||
*
|
*
|
||||||
* moinakg@belenix.org, http://moinakg.wordpress.com/
|
* moinakg@belenix.org, http://moinakg.wordpress.com/
|
||||||
*
|
*
|
||||||
*/
|
*/
|
||||||
|
|
||||||
/*
|
/*
|
||||||
|
|
|
@ -20,7 +20,7 @@
|
||||||
* If not, see <http://www.gnu.org/licenses/>.
|
* If not, see <http://www.gnu.org/licenses/>.
|
||||||
*
|
*
|
||||||
* moinakg@belenix.org, http://moinakg.wordpress.com/
|
* moinakg@belenix.org, http://moinakg.wordpress.com/
|
||||||
*
|
*
|
||||||
*/
|
*/
|
||||||
|
|
||||||
#include <stdio.h>
|
#include <stdio.h>
|
||||||
|
|
|
@ -72,16 +72,16 @@ exec_cpuid(uint32_t *regs)
|
||||||
" push %%rcx\n"
|
" push %%rcx\n"
|
||||||
" push %%rdx\n"
|
" push %%rdx\n"
|
||||||
" push %%rdi\n"
|
" push %%rdi\n"
|
||||||
|
|
||||||
" mov %0, %%rdi\n"
|
" mov %0, %%rdi\n"
|
||||||
|
|
||||||
" mov (%%rdi), %%eax\n"
|
" mov (%%rdi), %%eax\n"
|
||||||
" mov 4(%%rdi), %%ebx\n"
|
" mov 4(%%rdi), %%ebx\n"
|
||||||
" mov 8(%%rdi), %%ecx\n"
|
" mov 8(%%rdi), %%ecx\n"
|
||||||
" mov 12(%%rdi), %%edx\n"
|
" mov 12(%%rdi), %%edx\n"
|
||||||
|
|
||||||
" cpuid\n"
|
" cpuid\n"
|
||||||
|
|
||||||
" movl %%eax, (%%rdi)\n"
|
" movl %%eax, (%%rdi)\n"
|
||||||
" movl %%ebx, 4(%%rdi)\n"
|
" movl %%ebx, 4(%%rdi)\n"
|
||||||
" movl %%ecx, 8(%%rdi)\n"
|
" movl %%ecx, 8(%%rdi)\n"
|
||||||
|
|
|
@ -20,7 +20,7 @@
|
||||||
* If not, see <http://www.gnu.org/licenses/>.
|
* If not, see <http://www.gnu.org/licenses/>.
|
||||||
*
|
*
|
||||||
* moinakg@belenix.org, http://moinakg.wordpress.com/
|
* moinakg@belenix.org, http://moinakg.wordpress.com/
|
||||||
*
|
*
|
||||||
*/
|
*/
|
||||||
|
|
||||||
#include <stdio.h>
|
#include <stdio.h>
|
||||||
|
|
Loading…
Reference in a new issue