Tuning a TCP Connection

TCP Background

The TCP Congestion Window
  • Larger window implies higher throughput
  • The max allowed is based on the buffer space the kernel allocates for each socket
  • Each socket has a default buffer size
  • Sender and receiver can both adjust the size
Optimal Buffer Size
  • Too small -- Sender will be throttled
  • Too big -- receiver might be overloaded and packets will be dropped
    • More likely if the Sender is faster than the receiver
    • If the receiver has lots of memory, less likely to happen

Changing the Buffer Size

  • TCP chooses the smaller of both sides of the transaction.
  • Set the receiver high and let the sender negotiate

Picking a Buffer Size

The Maximum Throughput
If the network isn't congested the throughput is characterized by TCP buffer size and network latency.
\begin{equation*} max \; throughput \gets \frac{buffer \; size}{latency} \end{equation*}
Bandwidth Delay Product
This is a rule of thumb to calculate the optimal buffer size.
\begin{equation*} bdp \gets bottleneck \; bandwidth \times round \; trip \; time \end{equation*}
\begin{equation*} * Get the *rtt* from *ping* \end{equation*}
\begin{equation*} * Get *bottleneck bandwidth* from the theoretical maximum rate of the slowest link. \end{equation*}

Iperf Tuning

Iperf Warnings
  • Iperf can set the buffer size up to a point -- but the OS sets the upper limit on window sizes.
  • If the request is too high, iperf will use the maximum allowed and gives a warning.
Parallel Streams
  • Iperf lets you run multiple parallel sessions using the -P flag.
  • If the aggregate (SUM) is greater than a single stream, this is an indication that something is wrong -- most likely one of:
    • The TCP window is too small
    • The OS implementation has bugs
    • The network has problems
Maximum Transmission Unit (MTU)
System Message: WARNING/2 (<string>, line 85)
Title underline too short.
Maximum Transmission Unit (MTU)
------------------------------
  • the most effective way to set it is if both hosts support Path MTU Discovery and set it themselves.
  • iperf's -m flag displays what mss is being used
  • mss: Maximum Segment Size -- the maximum TCP segment size
\begin{equation*} mss \gets MTU - protocol \; headers \end{equation*}
  • Using -m is mainly to watch for the warning that the node lacks path mtu discovery

Getting Wireless Information From Netsh

netsh is a command-line utility to configure a Windows machine from the command-line. Here I'm going to show the commands to get information about the wireless interface.

The Base Query Command

The command takes the form:
netsh wlan show <what>
The Whats
all:
netsh wlan show all
Shows all the available information concatenated together.
drivers:
netsh wlan show drivers
Shows driver information for the interface.
interface:
netsh wlan show interface
Shows the interface information. Sample output:
There is 1 interface on the system:

Name : Wireless Network Connection
Description : Intel(R) Centrino(R) Wireless-N 100
GUID : 580dc1eb-f5cd-4eb9-8483-cff1a9e2f0f8
Physical address : 78:92:9c:8d:e1:5e
State : connected
SSID : allionstaff
BSSID : 18:33:9d:f9:ad:10
Network type : Infrastructure
Radio type : 802.11g
Authentication : WPA2-Personal
Cipher : CCMP
Connection mode : Auto Connect
Channel : 6
Receive rate (Mbps) : 72
Transmit rate (Mbps) : 72
Signal : 99%
Profile : allionstaff

Hosted network status : Not started

Getting The IP Address

To get the IP address you need to use a different subset (not wlan):
netsh interface ipv4 show addresses
Sample output:
Configuration for interface "Wireless Network Connection 3"
DHCP enabled: Yes
InterfaceMetric: 5

Configuration for interface "Wireless Network Connection 2"
DHCP enabled: Yes
InterfaceMetric: 5

Configuration for interface "Wireless Network Connection"
DHCP enabled: Yes
IP Address: 192.168.2.84
Subnet Prefix: 192.168.2.0/24 (mask 255.255.255.0)
Default Gateway: 192.168.2.1
Gateway Metric: 0
InterfaceMetric: 25

Configuration for interface "Local Area Connection"
DHCP enabled: No
IP Address: 192.168.10.63
Subnet Prefix: 192.168.10.0/24 (mask 255.255.255.0)
Default Gateway: 192.168.10.1
Gateway Metric: 256
InterfaceMetric: 10

Configuration for interface "Loopback Pseudo-Interface 1"
DHCP enabled: No
IP Address: 127.0.0.1
Subnet Prefix: 127.0.0.0/8 (mask 255.0.0.0)
InterfaceMetric: 50

Using WMIC to Enable and Disable The Wireless Network Adapter

The Utility

WMIC is the Windows Management Interface Command-line Utility a command-line interface to the WMI libraries. Here I'm going to focus on enabling and disabling the wireless interface.
Note: Since these are DOS commands they are case-insensitive (even inside quotes) so the mixed cases are only used to make it easier to read.

The Library

To enable and disable the interface you use the Win32_NetworkAdapter class.

The Interface

To get the interface name you can list the interfaces:
wmic path Win32_NetworkAdapter
But this will dump a mess of lines so here we'll assume you just know it's Wireless Network Connection (if you click on the wireless icon on the TaskBar you'll see that this is the name above the list of visible access points). This value is used for the NetConnectionID in the command.

To Dump Interface Information

If you want to dump all the Information:
wmic path Win32_NetworkAdapter where NetConnectionID="Wireless Network Connection"
If you want to get a single item (e.g. MAC Address):
wmic path Win32_NetworkAdapter where NetConnectionID="Wireless Network Connection" get MacAddress

To Enable the Interface

Call the enable method:
wmic path Win32_NetworkAdapter where NetConnectionID="Wireless Network Connection" call enable

To Disable The Interface

Call the disable method:
wmic path Win32_NetworkAdapter where NetConnectionID="Wireless Network Connection" call disable

Errors

Call Errors
A successful change of the interface:
Executing (\\IGOR\root\cimv2:Win32_NetworkAdapter.DeviceID="11")->enable()
Method execution successful.
Out Parameters:
instance of __PARAMETERS
{
ReturnValue = 0;
};
An unsuccessful one will have a non-zero ReturnValue and message.
Interface Name Errors
If you use an invalid name you'll see:
No Instance(s) Available.

Maddening Variations

Cygwin
For unknown reasons, if you use a single pair of quotes in the command, you get the error:
Node - IGOR
ERROR:
Description = Invalid query
Where IGOR is the name of the computer. To fix this you need to nest another pair of quotation marks inside the first. For example:
wmic path win32_networkadapter where NetConnectionID="'Wireless networK connection'" get Macaddress
Non-Interactive SSH
If you pass in the command as a string to an SSH Connection (e.g. in Paramiko), you need to put quotes around the whole thing and escape the nested quotation marks:
'wmic path win32_networkadapter where NetConnectionID="\'Wireless networK connection\'" get Macaddress'

Function Coverage

What's the Idea?

  • Partition the input domain automatically based on features of the source code.

How do you measure success?

  • Function Coverage is successful if every Function in SUT is executed during testing.
  • \[Test Coverage Metric \gets \frac{functions covered}{total number of functions}\]

What if you get less than 100% coverage?

  • Re-factor old and create new tests to cover the missing functions

What if you can't get 100% coverage?

Two possibilities:
  • The code is dead (never used)
  • You don't understand the system well enough

What are the drawbacks to this method?

  • Tests are based on code, not requirements -- they can't tell you if you're missing something
  • 100% has a clear meaning, anything less is up to judgement -- what's good enough?
  • Most systems are too complex to get 100% Coverage
  • Just because you've tested a function doesn't mean that you've tested all the code inside of it completely
From Udacity's Software Testing Unit 2

Mock a File

Creating The Readable Mock

To mock a file you can use StringIO and an input string. Since the StringIO will only traverse once (like a regular file) it might be convenient to make a mock builder.
def mock_open(data_string):
mock_file = MagicMock(spec=file)
mock_file.return_value = StringIO(data_string)
return mock_file

Using the Mock

To use the mock you patch the built-in open function.
opened = mock_file()

with patch('__builtin__.open', opened, create=True):
run_tests_here()

Checking the Mock

To see if the file was opened correctly:
opened.assert_called_with(filename, r_or_w)
To check write calls look at opened.write.mock_calls.

Types of Testing

White-Box, Black-Box

White-Box: The tester knows details about how the system works. Black-Box: Testing is done purely by knowing the expected inputs and outputs.

Unit Testing

  • Small modules of code are tested in isolation
  • The goal is to find defects in the internal logic of the code as soon as possible
  • No pre-conceived patterns of use are assumed (test all possible inputs)

Integration Testing

  • Take modules that have already been unit-tested and test them together
  • The goal is to test the interfaces between modules

System Testing

  • Usually Black-Box Testing
  • Tests if the system as a whole works
  • Focuses on how it will be used in the most important use-cases

Differential Testing

  1. create some input
  2. test 2 different implementations using the same input
  3. Test the outputs of both implementations for differences

Stress Testing

The System is tested at or beyond the limits of normal use.

Random Testing

Inputs are generated at random and fed to the System (e.g. CrashMe).

Regression Testing

Inputs that previosly caused the system to fail are tested on the new version.
From Udacity's Software Testing course unit 1.

Crashme

Problem

Operating Systems have to handle an open-ended number of bad system-calls without crashing.

Solution

  • Generate random bytes
  • Turn off system-level exception handling
  • Run the system using the random bytes as input to system calls
  • If it crashes, harden the code to handle it

Trust Boundaries

  • These are the interfaces where the user of the code enters input and you have to trust them to enter valid input.
  • Trust Boundaries have to be tested with as many inputs outside the domain as possible
From Udacity's Software Testing course Unit 1.

Non-Functional Inputs

Problem

Some things that affect the software are not under the control of your code or any of the libraries you are using.

Examples

  • Context switching between threads
  • Network errors

Solution

  • Minimize the use of code that relies on these inputs.
  • If possible, put something between you and the external system and inject faults to test the SUT.
From Udacity's Software Testing course Unit I.

Time-Dependent Problems

Problem

  • Sometimes the time between events has significance (the SUT behaves differently depending on the timing)
  • Sometimes race conditions can cause unpredictable behavior

Symptoms

Look in the code for:
  • timeouts
  • timed sleeps
  • values or computations that depend on the time

How To Test

Generate inputs that are varied in time (i.e. some close together, some not close together).
From Udacity's Software Testing Unit 1.