Tuning a TCP Connection

TCP Background

The TCP Congestion Window
  • Larger window implies higher throughput
  • The max allowed is based on the buffer space the kernel allocates for each socket
  • Each socket has a default buffer size
  • Sender and receiver can both adjust the size
Optimal Buffer Size
  • Too small -- Sender will be throttled
  • Too big -- receiver might be overloaded and packets will be dropped
    • More likely if the Sender is faster than the receiver
    • If the receiver has lots of memory, less likely to happen

Changing the Buffer Size

  • TCP chooses the smaller of both sides of the transaction.
  • Set the receiver high and let the sender negotiate

Picking a Buffer Size

The Maximum Throughput
If the network isn't congested the throughput is characterized by TCP buffer size and network latency.
\begin{equation*} max \; throughput \gets \frac{buffer \; size}{latency} \end{equation*}
Bandwidth Delay Product
This is a rule of thumb to calculate the optimal buffer size.
\begin{equation*} bdp \gets bottleneck \; bandwidth \times round \; trip \; time \end{equation*}
\begin{equation*} * Get the *rtt* from *ping* \end{equation*}
\begin{equation*} * Get *bottleneck bandwidth* from the theoretical maximum rate of the slowest link. \end{equation*}

Iperf Tuning

Iperf Warnings
  • Iperf can set the buffer size up to a point -- but the OS sets the upper limit on window sizes.
  • If the request is too high, iperf will use the maximum allowed and gives a warning.
Parallel Streams
  • Iperf lets you run multiple parallel sessions using the -P flag.
  • If the aggregate (SUM) is greater than a single stream, this is an indication that something is wrong -- most likely one of:
    • The TCP window is too small
    • The OS implementation has bugs
    • The network has problems
Maximum Transmission Unit (MTU)
System Message: WARNING/2 (<string>, line 85)
Title underline too short.
Maximum Transmission Unit (MTU)
  • the most effective way to set it is if both hosts support Path MTU Discovery and set it themselves.
  • iperf's -m flag displays what mss is being used
  • mss: Maximum Segment Size -- the maximum TCP segment size
\begin{equation*} mss \gets MTU - protocol \; headers \end{equation*}
  • Using -m is mainly to watch for the warning that the node lacks path mtu discovery