From FBSD_tips

INCOMPLETE DRAFT INCOMPLETE DRAFT INCOMPLETE DRAFT

Contents 1 Background and Description 2 Network set up 3 Running the simulation 3.1 Setup and confirm 3.2 ACK starvation 3.2.1 Downloading 3.2.2 Uploading 3.3 Full utilization 4 Discussion

[edit] Background and Description

A data flow over TCP/IP only appears as a continuous stream at the higher levels, the actual implementation is a series of discrete segments of data. These segments are broken up, transmitted over some medium and reassembled at the receiving end. Much work goes into keeping evidence of this dissembly and reassembly out of the view of the coder and program (unless this control is needed). One of the features of TCP is that each data packet must be acknowledged (referred to as an ACK henceforth) by the receiver. If acknowledgment of receipt of a packet is not received by the sender of that packet it will need to be retransmitted. This is to accomplish another aspect of TCP: guaranteed in sequence delivery of the original data stream. One way that congestion shows up on networks is that when packets start getting dropped in transit, ACKs for them do not come back, and the sending host backs off on it's rate of transmission. The sender and receiver of TCP data agree upon a certain number of packets that will be sent in advance of receiving an ACK before backing off in the rate of sending. This is known as the TCP window.

In this lab we will simulate the fairly common situation where a highly asymmetric link which gets saturated in its slower direction will not be able to keep a steady flow of ACKs going in the other direction. This disruption of ACK packets brings the effective throughput of TCP connections down to a fraction of the available bandwidth of the higher speed side of the link. Once we have reproduced this effect, we will put in place a mitigating measure on the downstream or consumer's side of the link.

[edit] Network set up

Use the test network from virtual_network_testbed. For convenience I include the diagram from it here :

In addition to the sofware from the Virtual Network Lab this exercise will require from the ports collection :

1. Wireshark
2. iftop

[edit] Running the simulation

[edit] Setup and confirm

First we boot up and bridge the 3 VMs as described in virtual_network_testbed. Then in the 'client2' machine we start a stream that will staturate the bridged interface.

Now we start up wireshark and snoop the tap1 interface (the tap interface bridged to the router's ed1 interface). Once you are capturing packets from the tap1 interface, go to the "statistics" menu in Wireshark and select "iographs", thsi will start a realtime line chart of the traffic.

Now in the router window, we will shape the traffic to (roughly) the bandwidth of a basic ADSL line, 128kbps up / 768kbps down. We do this by creating 1 pipes for each rate we will impose, then put the traffic of ed1 through each using the 'via [interface] [direction]' syntax.

Here is the console from the router:

You should see the wireshark graph drop down precipitously :

[edit] ACK starvation

[edit] Downloading

Now we have our 128kbps/768kbps "DSL" line simulation. We will now load it in the download direction by dd-ing /dev/zero on client1 one down to client via ssh. Iftop is run on the host machine on tap1 to confirm that the full downstream bandwidth is used :

[edit] Uploading

Now we kick off an upload that fills the available upstream by dd-ing /dev/zero on client2 to /dev/null on client1 via ssh. In a few moments, iftop shows the downstream has fallen to 10% - 20% of the available bandwidth.

[edit] Full utilization

Now we run the following script on client2. You have to set the pipe for just a bit below the upstream bandwidth, so the upstream does not quite fill ever. Then you let the queue balance the ACK and non ACK packaets, giving heavy preference to the ACK packets of the non payload carrying variety (sized 0-80).

#!/bin/sh

       sysctl net.inet.ip.fw.one_pass=0

       ipfw queue 1 config pipe 1 weight 100
       ipfw queue 3 config pipe 1 weight 1 mask all

       ipfw pipe 1 config bw 115kbit/s

       ipfw add 450 queue 1 tcp from any to any tcpflags ack oiplen 0-80 t via ed0
       ipfw add 451 queue 3 all from any to any out via fxp1

And in a few moments, iftop reflects an almost full bandwidth utilization again.

[edit] Discussion

On the downstream side of the asymmetric link, simulating the upstream bandwidth with a pipe of a fraction less than it is the best you can do, so you will never get 100 % utilization. The only way to do this "right" is to have the upstream router prioritize the small ACK packets.