Software‎ > ‎

ZidupCompress

Description

Deduplication-friendly gzip compression.
Compress a given file or directory with gzip while restarting every block. The compression uses deflate algorithm. Output is written to output file(s). Does not process gz/zip/rar files. For more details about the compression-restart positions see PACK project and the related publication.

Download

File Version Date Comments
ZidupCompress.jar 1.0 23-Jan-2014
ZidupCompress-source.zip 1.0 23-Jan-2014

Running

UsageZidupCompress.jar <filename/dir> <chunk-bits (6-15)>
   <filename/dir> - Existing input file name or a directory to process, full path or relative.
   <chunk-bits> - can be a range like "8-9".

Example:

java -jar ZidupCompress.jar c:\Temp\yahoo\Yahoo.ver1.htm 6-7

size-in        size-out bits     chunks avg_chunk avg_compr  ratio name
366,680         134,008    6      5,348        68        25 36.55% c:\Temp\yahoo\Yahoo.ver1.htm.pack-6bits.gz
366,680         111,733    7      2,691       136        41 30.47% c:\Temp\yahoo\Yahoo.ver1.htm.pack-7bits.gz


Technical
The file is loaded to memory first, and then it is being processed as a whole.
Therefore, very large files may require using the -Xmx flag to enlarge maximal memory allocation by the JVM. For example, java -Xmx4000m -jar ...
ċ
ZidupCompress-source.zip
(50k)
Eyal Zohar,
Mar 24, 2014, 3:39 AM
ċ
ZidupCompress.jar
(79k)
Eyal Zohar,
Jan 22, 2014, 10:41 PM
Comments