Projects‎ > ‎

Elastic Compression

Elastic compression is a tool for web-servers, reverse-proxy and load-balancers. It offers an automatic setting of compression effort, according to the amount of available CPU. 
The solutions offered by conventional web-servers require a web-admin to guess how much effort its compression takes. In addition, for a new level to take effect, a restart is always needed.

Apache and PHP Solution

Monitor and set

This is a background script that monitors the CPU and determines which compression level (or a mixture) to use next.
Download: the source code here: cpu_utilization3.sh

Server-side PHP plugin

The script below is a fully-functional demo that contains the code to be copied into an existing PHP code.
Download: the source code here: getfile-ver11.php

Apache Standalone (mod_deflate)

Supporting Tools

Several tools that helped us to develop the provided solution.

DeflateRunner

Usage: DeflateRunner.jar file [levels-analysis-requests-step]
Download: the source code here: DeflateRunner.jar

Offline performance analysis for zlib (gzip format).
Compress a given file with deflate algorithm , in order to test the compression ratio and speed for all possible compression levels (1-9).
The file is loaded to memory first, and then it is being compressed as a single complete buffer.
Compression strategy is set to default: Deflate with Huffman.
For each level it compresses the file repeatedly in memory until at least 200 mSec pass. Then it calculates how long a single compression took (average) and it presents it in 'Time_uSec' and 'Speed_Mbps' columns.
The column 'L1_Gain' is relevant for levels 2-9 only as it shows how many bytes were reduced when comparing to the level 1 output. Similarly, the column 'Time_Add' tells how much time was "paid" to get the higher ratio.
Output can be easily imported to Excel, using space as a delimiter and marking "merge multiple delimiters".
There is one optional enhancement to the basic operation: it can also calculate the optimal level(s) to use under a strict CPU budget.
To present the optimization calculations, one need to set the "levels-analysis-requests-step" parameter to a reasonable value, like 1 or 5 (for example).