Projects‎ > ‎

PACK - Predictive ACK: Traffic Redundancy Elimination for the Cloud

Downloads

libnfnetlink
============
    cd ~/Downloads/
    wget "http://ftp.netfilter.org/pub/libnfnetlink/libnfnetlink-1.0.0.tar.bz2"
    tar xvfj libnfnetlink-1.0.0.tar.bz2 
    cd libnfnetlink-1.0.0
    ./configure --prefix=/usr
    make
    sudo make install


libnetfilter queue
==================
    cd ~/Downloads/
    wget "http://ftp.netfilter.org/pub/libnetfilter_queue/libnetfilter_queue-1.0.0.tar.bz2"
    tar xvfj libnetfilter_queue-1.0.0.tar.bz2
    cd libnetfilter_queue-1.0.0
    ./configure --prefix=/usr
    make
    sudo make install

NFQueueJNI
==========
You may need to:
    sudo ln -sf /usr/lib/gcc/i486-linux-gnu/4.4 /usr/lib/gcc/i486-linux-gnu/4.4.3

Notes
-----
Don't "sudo" the ant command because you may have problems with the JDK_HOME var. 
If you need to sudo you will have to write a script to export the JDK_HOME and then run that script with "sudo sh <script name>".

If "gmake" is missing you may need to do this:
    sudo ln -s /usr/bin/make /usr/bin/gmake
    
If you have C++ other than 4.4 (4.5 for example):
    cd /usr/include/c++
    ln -sf /usr/include/c++/4.5 4.4
    
If jni.h is missing you may need to do this:
sudo find / -name jni.h|head -n 1
    export JAVA_HOME=/usr/lib/jvm/jdk1.6.0_17
    
To see all ant options for this project:
    ant -projecthelp


Savarese Virtual Services TCP/IP
================================
    cd ~/workspace
    wget "http://www.savarese.org/downloads/vserv-tcpip/vserv-tcpip-0.9.2-src.tar.gz"
    tar xvf vserv-tcpip-0.9.2-src.tar.gz
    cd vserv-tcpip-0.9.2
    ant jar

    cd ~/workspace
    wget "http://www.savarese.com/downloads/rocksaw/rocksaw-1.0.1-bin.tar.gz"
    tar xvf rocksaw-1.0.1-bin.tar.gz
    wget "http://www.savarese.com/downloads/rocksaw/rocksaw-1.0.1-src.tar.gz"
    tar xvf rocksaw-1.0.1-src.tar.gz
    cd rocksaw-1.0.1
    ant jar

If javac is not found, you may need to set:
    export JAVA_HOME=/usr/lib/jvm/jdk1.6.0_17/

Eclipse Projects
================

NFQueueJNI
----------
Add library. Select the project, and then from eclipse menu:
    Project | Properties | Java Build Path | Libraries (tab) | Add External JARs...
Then select ~/workspace/vserv-tcpip-0.9.2/lib/vserv-tcpip-0.9.2.jar

sim3
----
Add libraries. Select the project, and then from eclipse menu:
    Project | Properties | Java Build Path | Libraries (tab) | Add External JARs...
Then select ~/workspace/vserv-tcpip-0.9.2/lib/vserv-tcpip-0.9.2.jar
And repeat for ~/workspace/rocksaw-1.0.1/rocksaw-1.0.1.jar

Similarly, add the JUnit 4 library:
    Project | Properties | Java Build Path | Libraries (tab) | Add Library... | JUnit | JUnit 4

Add project. Select the project, and then from eclipse menu:
    Project | Properties | Java Build Path | Projects (tab) | Add External JARs...
Then select NFQueueJNI.

Iptables
========
There are 2 queues for each side:
    1 side sender direction in
    2 side sender direction out
    3 side receiver direction in
    4 side receiver direction out
    
This one captures traffic from and to www.oversi.com:
    sudo iptables -I INPUT -s 209.88.189.38 -p tcp --source-port 80 -j NFQUEUE --queue-num 3
    sudo iptables -I OUTPUT -d 209.88.189.38 -p tcp --destination-port 80 -j NFQUEUE --queue-num 4

This one captures traffic from and to web servers:
    sudo iptables -I INPUT -p tcp --source-port 80 -j NFQUEUE --queue-num 3
    sudo iptables -I OUTPUT -p tcp --destination-port 80 -j NFQUEUE --queue-num 4

Watching iptables in 1 sec intervals:
    sudo watch -n 1 -d iptables -nvL
    
How to Run
==========

Program arguments (optional):
    nodiskscan norestorechains eth0

Java parameters (don't use ~ in path name!):
    -Xms2000m -Xmx2000m -server -XX:+UseConcMarkSweepGC -Djava.library.path=/root/workspace/NFQueueJNI/dist:/root/workspace/rocksaw-1.0.1/lib
  
Testing    
=======

1. Install lighttpd

2. Run it as 127.0.0.2

3. Set iptables rules
    sudo iptables -F
    sudo iptables -I OUTPUT -d 127.0.0.1 -p tcp --source-port 80 -j NFQUEUE --queue-num 2
    sudo iptables -I INPUT -d 127.0.0.1 -p tcp --source-port 80 -j NFQUEUE --queue-num 3
    sudo iptables -I OUTPUT -d 127.0.0.2 -p tcp --destination-port 80 -j NFQUEUE --queue-num 4
    sudo iptables -I INPUT -d 127.0.0.2 -p tcp --destination-port 80 --tcp-option 29 -j NFQUEUE --queue-num 1
    sudo iptables -I INPUT -d 127.0.0.2 -p tcp --destination-port 80 --tcp-option 30 -j NFQUEUE --queue-num 1
    sudo iptables -I OUTPUT -p tcp --source-port 80 --tcp-option 29 -j ACCEPT
    sudo iptables -I OUTPUT -p tcp --source-port 80 --tcp-option 30 -j ACCEPT
    sudo iptables -I OUTPUT -p tcp --source-port 80 --tcp-flags ALL NONE -j ACCEPT
    sudo iptables -I INPUT -d 127.0.0.1 -p tcp --source-port 80 --tcp-flags ALL NONE -j NFQUEUE --queue-num 3

4. Set lo MTU to be just like a normal Ethernet NIC, otherwise the MTU will be too large for the queue (16KB for example, which is too much for the 4KB buffer):
    sudo ifconfig lo mtu 1500

5. Disable TCP timestamps:
    sudo echo "net.ipv4.tcp_timestamps = 0" >> /etc/sysctl.conf
    sudo sysctl -p

6. Use wget to download by client 127.0.0.1 from server 127.0.0.2
    wget --bind-address=127.0.0.1 "http://127.0.0.2/1.pdf" -O /tmp/1.pdf
    
It is also possible to limit speed with ethtool::

    sudo ethtool -s eth0 speed 10 duplex full autoneg off
    
Nfnetlink_queue
===============

A proc interface is available for nfnetlink_queue. Information is available from the file:

$ sudo cat /proc/net/netfilter/nfnetlink_queue
    0  10073     8 2 65535     0     0       10  1
    2  10525     0 2 65535     0     0        0  1
Each line contains information about a specific queue:

queue_num: id of the queue
peer_pid: pid of process handling the queue
queue_total: number of packets waiting for a decision
copy_mode: indicate how userspace receive packets
copy_range: size of copy
queue_dropped: number of items dropped by the kernel because too many packets were waiting a decision. It queue_total is superior to queue_max_len (1024 per default) the packets are dropped.
queue_user_dropped: number of packets dropped by userspace (due to kernel send failure on the netlink socket)
id_sequence: sequence number of packets queued. It gives a correct approximation of the number of queued packets.
use: internal value (number of entity using the queue)

Testing with 2 Machines
=======================

S - Sender running HTTP server
R - Receiver using wget to download

Sender
------
    sudo sysctl -w net.ipv4.tcp_wmem="65536 65536 16777216"

    export IP=192.168.0.1
    export NIC=eth1

    sudo ifconfig $NIC $IP netmask 255.255.255.0 mtu 1500 up

    sudo iptables -F
    sudo iptables -I INPUT -d $IP -p tcp --destination-port 80 -j NFQUEUE --queue-num 1
    sudo iptables -I OUTPUT -s $IP -p tcp --source-port 80 -j NFQUEUE --queue-num 2
    sudo iptables -I OUTPUT -s $IP -p tcp --source-port 80 --tcp-option 29 -j ACCEPT
    sudo iptables -I OUTPUT -s $IP -p tcp --source-port 80 --tcp-option 30 -j ACCEPT
    sudo iptables -I OUTPUT -s $IP -p tcp --source-port 80 --tcp-flags ALL NONE -j ACCEPT     

Receiver
--------
    sudo sysctl -w net.ipv4.tcp_rmem="4000 16000 87000"
    sudo sysctl -w net.ipv4.tcp_rmem="32000 32000 320000"
    
    export IP=192.168.0.2
    export NIC=eth0
    
    sudo ifconfig $NIC $IP netmask 255.255.255.0 mtu 1500 up

    sudo iptables -F
    sudo iptables -I INPUT -d $IP -p tcp --source-port 80 -j NFQUEUE --queue-num 3
    sudo iptables -I INPUT -d $IP -p tcp --source-port 80 --tcp-flags ALL PSH -j ACCEPT
    sudo iptables -I OUTPUT -s $IP -p tcp --destination-port 80 -j NFQUEUE --queue-num 4
    sudo iptables -I OUTPUT -s $IP -p tcp --destination-port 80 --tcp-option 29 -j ACCEPT
    sudo iptables -I OUTPUT -s $IP -p tcp --destination-port 80 --tcp-option 30 -j ACCEPT
    sudo iptables -I OUTPUT -d $IP -p tcp --source-port 80 --tcp-flags ALL NONE -j ACCEPT

Test command line:
/usr/lib/jvm/java-6-openjdk/bin/java -Xms1000m -Xmx1000m -server -XX:+UseConcMarkSweepGC -Djava.library.path=/home/eyalzo/workspace/NFQueueJNI/dist:/home/eyalzo/workspace/rocksaw-1.0.1/lib -Dfile.encoding=UTF-8 -classpath /home/eyalzo/workspace/NFQueueJNI/bin:/home/eyalzo/workspace/vserv-tcpip-0.9.2/lib/vserv-tcpip-0.9.2.jar:/home/eyalzo/workspace/sim3/bin:/usr/lib/eclipse/plugins/org.junit4_4.5.0.v20090824/junit.jar:/usr/lib/eclipse/plugins/org.hamcrest.core_1.1.0.jar:/home/eyalzo/workspace/rocksaw-1.0.1/lib/rocksaw-1.0.1.jar il.ac.technion.eyalzo.pack.Main nodiskscan norestorechains eth0

SNMP
----
To enable to queries from the receiver change /etc/snmp/snmpd.conf to have "agnetaddress 192.168.0.1" and chage public to "readwrite".

Check idle CPU on the sender:
    snmpwalk 192.168.0.1 -Of -c public -v2c .iso.org.dod.internet.private.enterprises.ucdavis.systemStats.ssCpuIdle.0
    
Wget
----
    for i in {1..5}; do wget -t 1 "http://192.168.0.1/$i.test" -O /tmp/$i.test -b --limit-rate 1000000; done

Test procedure for "Implementation" in Paper4
=============================================
export SPEED=500000
wget -t 1 "http://192.168.0.1/1.test" -O /tmp/1.test -b --limit-rate $SPEED
wget -t 1 "http://192.168.0.1/2.test" -O /tmp/1.test -b --limit-rate $SPEED
wget -t 1 "http://192.168.0.1/3.test" -O /tmp/1.test -b --limit-rate $SPEED
sleep 10
wget -t 1 "http://192.168.0.1/4.test" -O /tmp/1.test -b --limit-rate $SPEED
wget -t 1 "http://192.168.0.1/5.test" -O /tmp/1.test -b --limit-rate $SPEED
sleep 5
wget -t 1 "http://192.168.0.1/1.test" -O /tmp/1.test -b --limit-rate $SPEED
wget -t 1 "http://192.168.0.1/2.test" -O /tmp/1.test -b --limit-rate $SPEED
sleep 5
wget -t 1 "http://192.168.0.1/3.test" -O /tmp/1.test -b --limit-rate $SPEED
sleep 5
wget -t 1 "http://192.168.0.1/4.test" -O /tmp/1.test -b --limit-rate $SPEED
sleep 5
wget -t 1 "http://192.168.0.1/5.test" -O /tmp/1.test -b --limit-rate $SPEED
sleep 5

Test Files
==========

mail
----
wget -t 1 "http://192.168.0.1/mail/2010-01" -O /tmp/mail
wget -t 1 "http://192.168.0.1/mail/2010-02" -O /tmp/mail
wget -t 1 "http://192.168.0.1/mail/2010-03" -O /tmp/mail
wget -t 1 "http://192.168.0.1/mail/2010-04" -O /tmp/mail
wget -t 1 "http://192.168.0.1/mail/2010-05" -O /tmp/mail
wget -t 1 "http://192.168.0.1/mail/2010-06" -O /tmp/mail
wget -t 1 "http://192.168.0.1/mail/2010-07" -O /tmp/mail
wget -t 1 "http://192.168.0.1/mail/2010-08" -O /tmp/mail
wget -t 1 "http://192.168.0.1/mail/2010-09" -O /tmp/mail
wget -t 1 "http://192.168.0.1/mail/2010-10" -O /tmp/mail
wget -t 1 "http://192.168.0.1/mail/2010-11" -O /tmp/mail
wget -t 1 "http://192.168.0.1/mail/2010-12" -O /tmp/mail

3.pdf
-----
  5,790- 11,693 signature 454f1cad  5,904
 11,694- 23,579 signature b6ea2372 11,886
 23,580- 38,364 signature 51e6a44d 14,785
 38,365- 45,153 signature 87722876  6,789
 45,154- 48,066 signature 861359e0  2,913
 48,067- 63,372 signature ca015077 15,306
 63,373- 72,581 signature 9bb76aa1  9,209
 72,582- 96,624 signature 3f4a8bf6 24,043
 96,625- 98,740 signature a24b5555  2,116
 98,741-105,987 signature 8919a9c9  7,247
105,988-107,989 signature ce0cb0b3  2,002

Linux sources
-------------
export SPEED=50000000
export SERVER=http://192.168.0.1
ifconfig eth0|grep RX
wget -t 1 "$SERVER/linux/linux-2.4.0.tar" -O /tmp/0.test --limit-rate $SPEED
ifconfig eth0|grep RX
wget -t 1 "$SERVER/linux/linux-2.4.1.tar" -O /tmp/1.test --limit-rate $SPEED
ifconfig eth0|grep RX