Tuning a TCP Connection
Table of Contents
I wrote this a long time ago so I don't remember the source or really the intent. I think it was about trying to maximize the throughput when running iperf over TCP.
TCP Background
The TCP Congestion Window
- Larger window implies higher throughput
- The max allowed is based on the buffer space the kernel allocates for each socket
- Each socket has a default buffer size
- Sender and receiver can both adjust the size
Optimal Buffer Size
- Too small - Sender will be throttled
- More likely if the Sender is faster than the receiver
- Too big - Receiver might be overloaded and packets will be dropped
- If the receiver has lots of memory, less likely to happen
Changing the Buffer Size
- TCP chooses the smaller of both sides of the transaction.
- Set the receiver high and let the sender negotiate
Picking a Buffer Size
The Maximum Throughput
If the network isn't congested the throughput is characterized by TCP buffer size and network latency.
\begin{equation*} max \; throughput \gets \frac{buffer \; size}{latency} \end{equation*}Bandwidth Delay Product
This is a rule of thumb to calculate the optimal buffer size.
\begin{equation*} bdp \gets bottleneck \; bandwidth \times round \; trip \; time \end{equation*}- Get the rtt from ping.
- Get bottleneck bandwidth from the theoretical maximum rate of the slowest link.
Iperf Tuning
Iperf Warnings
- Iperf can set the buffer size up to a point – but the OS sets the upper limit on window sizes.
- If the request is too high, iperf will use the maximum allowed and gives a warning.
Parallel Streams
- Iperf lets you run multiple parallel sessions using the -P flag.
- If the aggregate (SUM) is greater than a single stream, this is an indication that something is wrong – most likely one of:
- The TCP window is too small
- The OS implementation has bugs
- The network has problems
Maximum Transmission Unit (MTU)
- the most effective way to set it is if both hosts support Path MTU Discovery and set it themselves.
- iperf's -m flag displays what mss is being used
- mss: Maximum Segment Size – the maximum TCP segment size
- Using -m is mainly to watch for the warning that the node lacks path mtu discovery