CIFAR-10
Table of Contents
- Introduction
- Set Up
- Visualize a Batch of Training Data
- Define the Network Architecture
- Define a model with multiple convolutional layers, and define the feedforward network behavior.
- Output volume for a convolutional layer
- Specify Loss Function and Optimizer
- Train the Network
- Load the Model with the Lowest Validation Loss
- Test the Trained Network
- Make it Easier
- Take two
- Change the Training and Validation Sets
Introduction
This is from Udacity's Deep Learning Repository which supports their Deep Learning Nanodegree. This will use a Convolutional Neural Network (CNN) to classify images from the CIFAR-10 data set.
The images in this data set are small color images that fall into one of ten classes:
- airplane
- automobile
- bird
- cat
- deer
- dog
- frog
- horse
- ship
- truck
There is another description of it on the University of Toronto's page for it.
Set Up
Imports
From Python
from datetime import datetime
from pathlib import Path
from typing import Tuple
import os
import pickle
From PyPi
from dotenv import load_dotenv
from sklearn.model_selection import train_test_split
from torchvision import datasets
from torch.utils.data.sampler import SubsetRandomSampler
import matplotlib.pyplot as pyplot
import numpy
import seaborn
import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optimize
import torchvision.transforms as transforms
This Project
from neurotic.tangles.data_paths import DataPathTwo
Plotting
get_ipython().run_line_magic('matplotlib', 'inline')
get_ipython().run_line_magic('config', "InlineBackend.figure_format = 'retina'")
seaborn.set(style="whitegrid",
rc={"axes.grid": False,
"font.family": ["sans-serif"],
"font.sans-serif": ["Latin Modern Sans", "Lato"],
"figure.figsize": (8, 6)},
font_scale=3)
Test for CUDA
The test-code uses the check later on so I'll save it to the train_on_gpu variable.
if os.environ.get("USER") == "brunhilde":
train_on_gpu = False
device = torch.device("cpu")
else:
train_on_gpu = torch.cuda.is_available()
device = torch.device("cuda:0" if train_on_gpu else "cpu")
print("Using: {}".format(device))
Using: cuda:0
Load the Data
# subprocesses to use
NUM_WORKERS = 0
# how many samples per batch to load
BATCH_SIZE = 20
# percentage of training set to use as validation
VALIDATION_FRACTION = 0.2
IMAGE_SIZE = 32
Convert the data to a normalized torch.FloatTensor using a pipeline. I'm also going to introduce some randomness to help the model generalize.
means = deviations = (0.5, 0.5, 0.5)
train_transform = transforms.Compose([
transforms.RandomRotation(30),
transforms.RandomResizedCrop(IMAGE_SIZE),
transforms.RandomHorizontalFlip(),
transforms.ToTensor(),
transforms.Normalize(means, deviations)
])
test_transforms = transforms.Compose([
transforms.ToTensor(),
transforms.Normalize(means,
deviations)])
Choose the training and test datasets.
load_dotenv()
path = DataPathTwo(folder_key="CIFAR")
print(path.folder)
/home/hades/datasets/CIFAR
training_data = datasets.CIFAR10(path.folder, train=True,
download=True, transform=train_transform)
test_data = datasets.CIFAR10(path.folder, train=False,
download=True, transform=test_transforms)
Files already downloaded and verified Files already downloaded and verified
for item in path.folder.iterdir():
print(item)
/home/hades/datasets/CIFAR/cifar-10-batches-py /home/hades/datasets/CIFAR/cifar-10-python.tar.gz
Obtain Training Indices For Validation
indices = list(range(len(training_data)))
training_indices, validation_indices = train_test_split(
indices,
test_size=VALIDATION_FRACTION)
Define Samplers For Training And Validation Batches
train_sampler = SubsetRandomSampler(training_indices)
valid_sampler = SubsetRandomSampler(validation_indices)
Prepare Data Loaders
train_loader = torch.utils.data.DataLoader(training_data, batch_size=BATCH_SIZE,
sampler=train_sampler, num_workers=NUM_WORKERS)
valid_loader = torch.utils.data.DataLoader(training_data, batch_size=BATCH_SIZE,
sampler=valid_sampler, num_workers=NUM_WORKERS)
test_loader = torch.utils.data.DataLoader(test_data, batch_size=BATCH_SIZE,
num_workers=NUM_WORKERS)
The Image Classes
classes = ['airplane', 'automobile', 'bird', 'cat', 'deer',
'dog', 'frog', 'horse', 'ship', 'truck']
Visualize a Batch of Training Data
helper function to un-normalize and display an image
def imshow(img):
img = img / 2 + 0.5 # unnormalize
pyplot.imshow(numpy.transpose(img, (1, 2, 0))) # convert from Tensor image
obtain one batch of training images
dataiter = iter(train_loader)
images, labels = dataiter.next()
images = images.numpy() # convert images to numpy for display
plot the images in the batch, along with the corresponding labels
figure = pyplot.figure(figsize=(25, 4))
# display 20 images
figure.suptitle("Batch Sample", weight="bold")
for idx in numpy.arange(20):
ax = figure.add_subplot(2, 20/2, idx+1, xticks=[], yticks=[])
imshow(images[idx])
ax.set_title(classes[labels[idx]])
#pyplot.subplots_adjust(top=0.7)
pyplot.tight_layout(rect=[0, 0.03, 1, 0.95])
View an Image in More Detail
Here, we look at the normalized red, green, and blue (RGB) color channels as three separate, grayscale intensity images.
rgb_img = numpy.squeeze(images[3])
channels = ['red channel', 'green channel', 'blue channel']
fig = pyplot.figure(figsize = (36, 36))
for idx in numpy.arange(rgb_img.shape[0]):
ax = fig.add_subplot(1, 3, idx + 1)
img = rgb_img[idx]
ax.imshow(img, cmap='gray')
ax.set_title(channels[idx])
width, height = img.shape
thresh = img.max()/2.5
for x in range(width):
for y in range(height):
val = round(img[x][y],2) if img[x][y] !=0 else 0
ax.annotate(str(val), xy=(y,x),
horizontalalignment='center',
verticalalignment='center', size=8,
color='white' if img[x][y]<thresh else 'black')
Define the Network Architecture
This time, you'll define a CNN architecture. Instead of an MLP, which used linear, fully-connected layers, you'll use the following:
- Convolutional layers, which can be thought of as stack of filtered images.
- Maxpooling layers, which reduce the x-y size of an input, keeping only the most active pixels from the previous layer.
- The usual Linear + Dropout layers to avoid overfitting and produce a 10-dim output.
Define a model with multiple convolutional layers, and define the feedforward network behavior.
The more convolutional layers you include, the more complex patterns in color and shape a model can detect. It's suggested that your final model include 2 or 3 convolutional layers as well as linear layers + dropout in between to avoid overfitting.
It's good practice to look at existing research and implementations of related models as a starting point for defining your own models. You may find it useful to look at this PyTorch classification example or this, more complex Keras example to help decide on a final structure.
This is taken from the pytorch tutorial, with padding and dropout added. I also changed the kernel size to 3.
See:
KERNEL_SIZE = 3
CHANNELS_IN = 3
CHANNELS_OUT_1 = 6
CHANNELS_OUT_2 = 16
CLASSES = 10
PADDING = 1
STRIDE = 1
convolutional_1 = nn.Conv2d(CHANNELS_IN, CHANNELS_OUT_1,
KERNEL_SIZE,
stride=STRIDE, padding=PADDING)
pool = nn.MaxPool2d(2, 2)
convolutional_2 = nn.Conv2d(CHANNELS_OUT_1, CHANNELS_OUT_2,
KERNEL_SIZE,
stride=STRIDE, padding=PADDING)
c_no_padding_1 = nn.Conv2d(CHANNELS_IN, CHANNELS_OUT_1, KERNEL_SIZE)
c_no_padding_2 = nn.Conv2d(CHANNELS_OUT_1, CHANNELS_OUT_2, KERNEL_SIZE)
fully_connected_1 = nn.Linear(CHANNELS_OUT_2 * (KERNEL_SIZE + PADDING)**3, 120)
fully_connected_1A = nn.Linear(CHANNELS_OUT_2 * (KERNEL_SIZE)**2, 120)
fully_connected_2 = nn.Linear(120, 84)
fully_connected_3 = nn.Linear(84, CLASSES)
cnn_dropout = nn.Dropout(0.25)
connected_dropout = nn.Dropout(0.5)
dataiter = iter(train_loader)
images, labels = dataiter.next()
input_image = torch.Tensor(images)
print("Input Shape: {}".format(input_image.shape))
x = cnn_dropout(pool(F.relu(convolutional_1(input_image))))
print("Output 1: {}".format(x.shape))
x = cnn_dropout(pool(F.relu(convolutional_2(x))))
print("Output 2: {}".format(x.shape))
x = x.view(x.size()[0], -1)
print("reshaped: {}".format(x.shape))
x = connected_dropout(F.relu(fully_connected_1(x)))
print("Connected Shape: {}".format(x.shape))
x = F.relu(fully_connected_2(x))
print("Connected Shape 2: {}".format(x.shape))
x = fully_connected_3(x)
print("Connected Shape 3: {}".format(x.shape))
Input Shape: torch.Size([20, 3, 32, 32]) Output 1: torch.Size([20, 6, 16, 16]) Output 2: torch.Size([20, 16, 8, 8]) reshaped: torch.Size([20, 1024]) Connected Shape: torch.Size([20, 120]) Connected Shape 2: torch.Size([20, 84]) Connected Shape 3: torch.Size([20, 10])
print("Input Shape: {}".format(input_image.shape))
x = cnn_dropout(pool(F.relu(c_no_padding_1(input_image))))
print("Output 1: {}".format(x.shape))
x = cnn_dropout(pool(F.relu(c_no_padding_2(x))))
print("Output 2: {}".format(x.shape))
x = x.view(-1, CHANNELS_OUT_2 * (KERNEL_SIZE)**2)
print("reshaped: {}".format(x.shape))
x = connected_dropout(F.relu(fully_connected_1A(x)))
print("Connected Shape: {}".format(x.shape))
x = F.relu(fully_connected_2(x))
print("Connected Shape 2: {}".format(x.shape))
x = fully_connected_3(x)
print("Connected Shape 3: {}".format(x.shape))
Input Shape: torch.Size([20, 3, 32, 32]) Output 1: torch.Size([20, 6, 15, 15]) Output 2: torch.Size([20, 16, 6, 6]) reshaped: torch.Size([80, 144]) Connected Shape: torch.Size([80, 120]) Connected Shape 2: torch.Size([80, 84]) Connected Shape 3: torch.Size([80, 10])
class CNN(nn.Module):
"""A convolutional neural network for CIFAR-10 images"""
def __init__(self, filter_size=5) -> None:
super().__init__()
self.convolutional_1 = nn.Conv2d(CHANNELS_IN, CHANNELS_OUT_1,
KERNEL_SIZE,
stride=STRIDE, padding=PADDING)
self.pool = nn.MaxPool2d(2, 2)
self.convolutional_2 = nn.Conv2d(CHANNELS_OUT_1, CHANNELS_OUT_2,
KERNEL_SIZE,
stride=STRIDE, padding=PADDING)
self.fully_connected_1 = nn.Linear(CHANNELS_OUT_2 * (KERNEL_SIZE + PADDING)**3, 120)
self.fully_connected_2 = nn.Linear(120, 84)
self.fully_connected_3 = nn.Linear(84, CLASSES)
self.cnn_dropout = nn.Dropout(0.25)
self.connected_dropout = nn.Dropout(0.5)
return
def forward(self, x: torch.Tensor) -> torch.Tensor:
"""Passes the image through the layers of the network
Args:
image: CIFAR image to process
"""
x = self.cnn_dropout(self.pool(F.relu(self.convolutional_1(x))))
x = self.cnn_dropout(self.pool(F.relu(self.convolutional_2(x))))
# flatten to a vector
x = x.view(x.size()[0], -1)
x = self.connected_dropout(F.relu(self.fully_connected_1(x)))
x = F.relu(self.fully_connected_2(x))
return self.fully_connected_3(x)
model = CNN()
dataiter = iter(train_loader)
images, labels = dataiter.next()
print(images.shape)
print(labels.shape)
output = model(images)
print(output.shape)
torch.Size([20, 3, 32, 32]) torch.Size([20]) torch.Size([20, 10])
Output volume for a convolutional layer
To compute the output size of a given convolutional layer we can perform the following calculation (taken from Stanford's cs231n course):
We can compute the spatial size of the output volume as a function of the input volume size (W), the kernel/filter size (F), the stride with which they are applied (S), and the amount of zero padding used (P) on the border. The correct formula for calculating how many neurons define the output_W is given by (W−F+2P)/S+1.
For example for a 7x7 input and a 3x3 filter with stride 1 and pad 0 we would get a 5x5 output. With stride 2 we would get a 3x3 output.
Specify Loss Function and Optimizer
Decide on a loss and optimization function that is best suited for this classification task. The linked code examples from above, may be a good starting point; this PyTorch classification example or this, more complex Keras example. Pay close attention to the value for learning rate as this value determines how your model converges to a small error.
criterion = nn.CrossEntropyLoss()
Train the Network
Remember to look at how the training and validation loss decreases over time; if the validation loss ever increases it indicates possible overfitting.
def train(model: nn.Module, epochs: int=10, model_number: int=0,
epoch_offset: int=1, print_every: int=10) -> tuple:
"""Train, validate, and save the model
This trains the model and validates it, saving the best
(based on validation loss) as =model_<number>_cifar.pth=
Args:
model: the network to train
epochs: number of times to repeat training
model_number: an identifier for the saved hyperparameters file
epoch_offset: amount of epochs that have occurred previously
print_every: how often to print output
Returns:
filename, training-loss, validation-loss, improvements: the outcomes for the training
"""
optimizer = optimize.SGD(model.parameters(), lr=0.001, momentum=0.9)
criterion = nn.CrossEntropyLoss()
output_file = "model_{}_cifar.pth".format(model_number)
training_losses = []
validation_losses = []
improvements = []
valid_loss_min = numpy.Inf # track change in validation loss
epoch_start = epoch_offset
last_epoch = epoch_start + epochs + 1
for epoch in range(epoch_start, last_epoch):
# keep track of training and validation loss
train_loss = 0.0
valid_loss = 0.0
model.train()
for data, target in train_loader:
# move tensors to GPU if CUDA is available
data, target = data.to(device), target.to(device)
# clear the gradients of all optimized variables
optimizer.zero_grad()
# forward pass: compute predicted outputs by passing inputs to the model
output = model(data)
# calculate the batch loss
loss = criterion(output, target)
# backward pass: compute gradient of the loss with respect to model parameters
loss.backward()
# perform a single optimization step (parameter update)
optimizer.step()
# update training loss
train_loss += loss.item() * data.size(0)
model.eval()
for data, target in valid_loader:
# move tensors to GPU if CUDA is available
data, target = data.to(device), target.to(device)
# forward pass: compute predicted outputs by passing inputs to the model
output = model(data)
# calculate the batch loss
loss = criterion(output, target)
# update total validation loss
valid_loss += loss.item() * data.size(0)
# calculate average losses
train_loss = train_loss/len(train_loader.dataset)
valid_loss = valid_loss/len(valid_loader.dataset)
# print training/validation statistics
if not (epoch % print_every):
print('Epoch: {} \tTraining Loss: {:.6f} \tValidation Loss: {:.6f}'.format(
epoch, train_loss, valid_loss))
training_losses.append(train_loss)
validation_losses.append(valid_loss)
# save model if validation loss has decreased
if valid_loss <= valid_loss_min:
print('Validation loss decreased ({:.6f} --> {:.6f}). Saving model ...'.format(
valid_loss_min,
valid_loss))
torch.save(model.state_dict(), output_file)
valid_loss_min = valid_loss
improvements.append(epoch - 1)
return output_file, training_losses, validation_losses, improvements
Pytorch Tutorial Model
EPOCHS = 250
This is only to avoid re-running the initial training and use the saved model. Note: If you use DataParallel you need to save the model using model.module.state_dict() in order to load it later without it. This won't matter if you always use it or never use it, but here I have a model that was trained on a GPU and I'm trying to extend the training with a computer whos GPU is too old for pytorch to use it, so it crashes unless I disable the DataParallel (because I didn't originally save it with model.module.state_dict).
Note 2: But if you don't have it in DataParallel then don't use model.module.state_dict because it won't have the module attribute.
def train_and_pickle(model:nn.Module, epochs:int=EPOCHS,
model_number:int=2, print_every: int=10) -> dict:
"""Trains and pickles the outcomes of training"""
path = Path("model_{}_outcomes.pkl".format(model_number))
existed = False
epoch_offset = 0
if path.is_file():
existed = True
with path.open("rb") as reader:
outcomes = pickle.load(reader)
epoch_offset = len(outcomes["training_loss"])
model.load_state_dict(torch.load(
outcome["hyperparameters_file"],
map_location=device))
filename, training_loss, validation_loss, improvements = train(
model,
epochs=epochs,
model_number=model_number,
epoch_offset=epoch_offset,
print_every=print_every,
)
if existed:
outcomes["training_loss"] += outcomes["training_loss"]
outcomes["validation_loss"] += outcomes["validation_loss"]
outcomes["improvements"] += outcomes["improvements"]
else:
outcomes = dict(
hyperparameters_file=filename,
outcomes_pickle=path.name,
training_loss=training_loss,
validation_loss=validation_loss,
improvements=improvements,
)
with path.open("wb") as writer:
pickle.dump(outcomes, writer)
return outcomes
def update_outcome(outcome: dict, new_outcome: dict) -> dict:
"""Updates the lists in the outcome
Args:
outcome: original output of train_and_pickle
new_outcome: new output of train_and_pickle
Returns:
outcome: updated outcome
"""
for key in ("training_loss", "validation_loss", "improvements"):
outcome[key] += new_outcome[key]
return outcome
First Model Training
model_2 = CNN()
model_2.to(device)
start = datetime.now()
outcome = train_and_pickle(
model_2,
epochs=100,
model_number=2)
print("Elapsed: {}".format(datetime.now() - start))
Epoch: 0 Training Loss: 1.834230 Validation Loss: 0.446434 Validation loss decreased (inf --> 0.446434). Saving model ... Epoch: 1 Training Loss: 1.685185 Validation Loss: 0.403314 Validation loss decreased (0.446434 --> 0.403314). Saving model ... Epoch: 2 Training Loss: 1.602409 Validation Loss: 0.389758 Validation loss decreased (0.403314 --> 0.389758). Saving model ... Epoch: 3 Training Loss: 1.551087 Validation Loss: 0.376669 Validation loss decreased (0.389758 --> 0.376669). Saving model ... Epoch: 4 Training Loss: 1.524230 Validation Loss: 0.371581 Validation loss decreased (0.376669 --> 0.371581). Saving model ... Epoch: 5 Training Loss: 1.496748 Validation Loss: 0.367056 Validation loss decreased (0.371581 --> 0.367056). Saving model ... Epoch: 6 Training Loss: 1.479645 Validation Loss: 0.359889 Validation loss decreased (0.367056 --> 0.359889). Saving model ... Epoch: 7 Training Loss: 1.462357 Validation Loss: 0.358887 Validation loss decreased (0.359889 --> 0.358887). Saving model ... Epoch: 8 Training Loss: 1.454448 Validation Loss: 0.353885 Validation loss decreased (0.358887 --> 0.353885). Saving model ... Epoch: 9 Training Loss: 1.442392 Validation Loss: 0.349046 Validation loss decreased (0.353885 --> 0.349046). Saving model ... Epoch: 10 Training Loss: 1.435758 Validation Loss: 0.345204 Validation loss decreased (0.349046 --> 0.345204). Saving model ... Epoch: 11 Training Loss: 1.428880 Validation Loss: 0.344610 Validation loss decreased (0.345204 --> 0.344610). Saving model ... Epoch: 12 Training Loss: 1.420400 Validation Loss: 0.343866 Validation loss decreased (0.344610 --> 0.343866). Saving model ... Epoch: 13 Training Loss: 1.409974 Validation Loss: 0.341221 Validation loss decreased (0.343866 --> 0.341221). Saving model ... Epoch: 14 Training Loss: 1.400003 Validation Loss: 0.340469 Validation loss decreased (0.341221 --> 0.340469). Saving model ... Epoch: 15 Training Loss: 1.396430 Validation Loss: 0.338332 Validation loss decreased (0.340469 --> 0.338332). Saving model ... Epoch: 16 Training Loss: 1.396793 Validation Loss: 0.338963 Epoch: 17 Training Loss: 1.391945 Validation Loss: 0.337340 Validation loss decreased (0.338332 --> 0.337340). Saving model ... Epoch: 18 Training Loss: 1.383872 Validation Loss: 0.335848 Validation loss decreased (0.337340 --> 0.335848). Saving model ... Epoch: 19 Training Loss: 1.371348 Validation Loss: 0.335116 Validation loss decreased (0.335848 --> 0.335116). Saving model ... Epoch: 20 Training Loss: 1.374097 Validation Loss: 0.330697 Validation loss decreased (0.335116 --> 0.330697). Saving model ... Epoch: 21 Training Loss: 1.373342 Validation Loss: 0.334281 Epoch: 22 Training Loss: 1.366379 Validation Loss: 0.331197 Epoch: 23 Training Loss: 1.366043 Validation Loss: 0.332052 Epoch: 24 Training Loss: 1.359814 Validation Loss: 0.328743 Validation loss decreased (0.330697 --> 0.328743). Saving model ... Epoch: 25 Training Loss: 1.359745 Validation Loss: 0.328860 Epoch: 26 Training Loss: 1.353130 Validation Loss: 0.329480 Epoch: 27 Training Loss: 1.352457 Validation Loss: 0.329386 Epoch: 28 Training Loss: 1.348608 Validation Loss: 0.331024 Epoch: 29 Training Loss: 1.346584 Validation Loss: 0.325815 Validation loss decreased (0.328743 --> 0.325815). Saving model ... Epoch: 30 Training Loss: 1.341498 Validation Loss: 0.332342 Epoch: 31 Training Loss: 1.339088 Validation Loss: 0.325358 Validation loss decreased (0.325815 --> 0.325358). Saving model ... Epoch: 32 Training Loss: 1.347376 Validation Loss: 0.326178 Epoch: 33 Training Loss: 1.342424 Validation Loss: 0.331979 Epoch: 34 Training Loss: 1.339343 Validation Loss: 0.324638 Validation loss decreased (0.325358 --> 0.324638). Saving model ... Epoch: 35 Training Loss: 1.332784 Validation Loss: 0.322740 Validation loss decreased (0.324638 --> 0.322740). Saving model ... Epoch: 36 Training Loss: 1.335403 Validation Loss: 0.324083 Epoch: 37 Training Loss: 1.332313 Validation Loss: 0.334746 Epoch: 38 Training Loss: 1.329136 Validation Loss: 0.324193 Epoch: 39 Training Loss: 1.327429 Validation Loss: 0.327056 Epoch: 40 Training Loss: 1.328106 Validation Loss: 0.327257 Epoch: 41 Training Loss: 1.330462 Validation Loss: 0.321711 Validation loss decreased (0.322740 --> 0.321711). Saving model ... Epoch: 42 Training Loss: 1.326317 Validation Loss: 0.324698 Epoch: 43 Training Loss: 1.325379 Validation Loss: 0.324895 Epoch: 44 Training Loss: 1.322629 Validation Loss: 0.322434 Epoch: 45 Training Loss: 1.320261 Validation Loss: 0.326130 Epoch: 46 Training Loss: 1.316204 Validation Loss: 0.325013 Epoch: 47 Training Loss: 1.315747 Validation Loss: 0.324042 Epoch: 48 Training Loss: 1.313305 Validation Loss: 0.324592 Epoch: 49 Training Loss: 1.313723 Validation Loss: 0.318290 Validation loss decreased (0.321711 --> 0.318290). Saving model ... Epoch: 50 Training Loss: 1.313054 Validation Loss: 0.320845 Epoch: 51 Training Loss: 1.316062 Validation Loss: 0.321215 Epoch: 52 Training Loss: 1.316187 Validation Loss: 0.319871 Epoch: 53 Training Loss: 1.312232 Validation Loss: 0.324769 Epoch: 54 Training Loss: 1.315246 Validation Loss: 0.321788 Epoch: 55 Training Loss: 1.307923 Validation Loss: 0.318943 Epoch: 56 Training Loss: 1.316049 Validation Loss: 0.324919 Epoch: 57 Training Loss: 1.310584 Validation Loss: 0.319344 Epoch: 58 Training Loss: 1.305451 Validation Loss: 0.320848 Epoch: 59 Training Loss: 1.309900 Validation Loss: 0.322148 Epoch: 60 Training Loss: 1.306200 Validation Loss: 0.323148 Epoch: 61 Training Loss: 1.303626 Validation Loss: 0.322406 Epoch: 62 Training Loss: 1.304654 Validation Loss: 0.322471 Epoch: 63 Training Loss: 1.302740 Validation Loss: 0.322596 Epoch: 64 Training Loss: 1.306964 Validation Loss: 0.323696 Epoch: 65 Training Loss: 1.301964 Validation Loss: 0.319375 Epoch: 66 Training Loss: 1.302925 Validation Loss: 0.320327 Epoch: 67 Training Loss: 1.302062 Validation Loss: 0.319882 Epoch: 68 Training Loss: 1.299821 Validation Loss: 0.318813 Epoch: 69 Training Loss: 1.298885 Validation Loss: 0.325837 Epoch: 70 Training Loss: 1.303130 Validation Loss: 0.320493 Epoch: 71 Training Loss: 1.301353 Validation Loss: 0.321375 Epoch: 72 Training Loss: 1.294933 Validation Loss: 0.315513 Validation loss decreased (0.318290 --> 0.315513). Saving model ... Epoch: 73 Training Loss: 1.303322 Validation Loss: 0.322531 Epoch: 74 Training Loss: 1.298327 Validation Loss: 0.323503 Epoch: 75 Training Loss: 1.298817 Validation Loss: 0.318616 Epoch: 76 Training Loss: 1.296895 Validation Loss: 0.323739 Epoch: 77 Training Loss: 1.301932 Validation Loss: 0.325410 Epoch: 78 Training Loss: 1.291901 Validation Loss: 0.327083 Epoch: 79 Training Loss: 1.295766 Validation Loss: 0.317765 Epoch: 80 Training Loss: 1.295147 Validation Loss: 0.316187 Epoch: 81 Training Loss: 1.294392 Validation Loss: 0.318913 Epoch: 82 Training Loss: 1.290720 Validation Loss: 0.320984 Epoch: 83 Training Loss: 1.296386 Validation Loss: 0.322005 Epoch: 84 Training Loss: 1.294445 Validation Loss: 0.319135 Epoch: 85 Training Loss: 1.288677 Validation Loss: 0.317673 Epoch: 86 Training Loss: 1.292154 Validation Loss: 0.318644 Epoch: 87 Training Loss: 1.292221 Validation Loss: 0.317595 Epoch: 88 Training Loss: 1.295039 Validation Loss: 0.319856 Epoch: 89 Training Loss: 1.289999 Validation Loss: 0.320703 Epoch: 90 Training Loss: 1.290199 Validation Loss: 0.317269 Epoch: 91 Training Loss: 1.289213 Validation Loss: 0.318887 Epoch: 92 Training Loss: 1.284553 Validation Loss: 0.320420 Epoch: 93 Training Loss: 1.292121 Validation Loss: 0.319414 Epoch: 94 Training Loss: 1.281610 Validation Loss: 0.314129 Validation loss decreased (0.315513 --> 0.314129). Saving model ... Epoch: 95 Training Loss: 1.292147 Validation Loss: 0.317541 Epoch: 96 Training Loss: 1.288869 Validation Loss: 0.316178 Epoch: 97 Training Loss: 1.284419 Validation Loss: 0.326122 Epoch: 98 Training Loss: 1.292448 Validation Loss: 0.314851 Epoch: 99 Training Loss: 1.287391 Validation Loss: 0.315212 Epoch: 100 Training Loss: 1.285748 Validation Loss: 0.320298 Elapsed: 1:26:31.644031
pickle_path = Path("model_2_outcomes.pkl")
with pickle_path.open("rb") as reader:
outcome = pickle.load(reader)
model_2 = CNN()
model_2.to(device)
start = datetime.now()
model_2.load_state_dict(torch.load(outcome["hyperparameters_file"],
map_location=device))
outcome_2 = train_and_pickle(model_2, epochs=200, model_number=2)
outcome = update_outcome(outcome, outcome_2)
print("Elapsed: {}".format(datetime.now() - start))
Epoch: 101 Training Loss: 1.293572 Validation Loss: 0.323292 Validation loss decreased (inf --> 0.323292). Saving model ... Epoch: 102 Training Loss: 1.286175 Validation Loss: 0.316041 Validation loss decreased (0.323292 --> 0.316041). Saving model ... Epoch: 103 Training Loss: 1.292286 Validation Loss: 0.318805 Epoch: 104 Training Loss: 1.287122 Validation Loss: 0.318283 Epoch: 105 Training Loss: 1.285004 Validation Loss: 0.316454 Epoch: 106 Training Loss: 1.288655 Validation Loss: 0.328694 Epoch: 107 Training Loss: 1.286483 Validation Loss: 0.311118 Validation loss decreased (0.316041 --> 0.311118). Saving model ... Epoch: 108 Training Loss: 1.286722 Validation Loss: 0.322617 Epoch: 109 Training Loss: 1.281688 Validation Loss: 0.317284 Epoch: 110 Training Loss: 1.286374 Validation Loss: 0.316699 Epoch: 111 Training Loss: 1.285399 Validation Loss: 0.315800 Epoch: 112 Training Loss: 1.283735 Validation Loss: 0.321917 Epoch: 113 Training Loss: 1.283596 Validation Loss: 0.311436 Epoch: 114 Training Loss: 1.285218 Validation Loss: 0.314240 Epoch: 115 Training Loss: 1.282439 Validation Loss: 0.315108 Epoch: 116 Training Loss: 1.282893 Validation Loss: 0.317056 Epoch: 117 Training Loss: 1.282942 Validation Loss: 0.313947 Epoch: 118 Training Loss: 1.287284 Validation Loss: 0.316639 Epoch: 119 Training Loss: 1.285622 Validation Loss: 0.321113 Epoch: 120 Training Loss: 1.284308 Validation Loss: 0.319277 Epoch: 121 Training Loss: 1.282111 Validation Loss: 0.314455 Epoch: 122 Training Loss: 1.283129 Validation Loss: 0.313159 Epoch: 123 Training Loss: 1.284335 Validation Loss: 0.322168 Epoch: 124 Training Loss: 1.278320 Validation Loss: 0.318971 Epoch: 125 Training Loss: 1.281218 Validation Loss: 0.313987 Epoch: 126 Training Loss: 1.279132 Validation Loss: 0.328925 Epoch: 127 Training Loss: 1.279555 Validation Loss: 0.316594 Epoch: 128 Training Loss: 1.273169 Validation Loss: 0.315559 Epoch: 129 Training Loss: 1.277613 Validation Loss: 0.319802 Epoch: 130 Training Loss: 1.280081 Validation Loss: 0.322822 Epoch: 131 Training Loss: 1.281299 Validation Loss: 0.317239 Epoch: 132 Training Loss: 1.280862 Validation Loss: 0.317907 Epoch: 133 Training Loss: 1.280196 Validation Loss: 0.323627 Epoch: 134 Training Loss: 1.278056 Validation Loss: 0.315584 Epoch: 135 Training Loss: 1.271644 Validation Loss: 0.317295 Epoch: 136 Training Loss: 1.276935 Validation Loss: 0.325810 Epoch: 137 Training Loss: 1.279832 Validation Loss: 0.320269 Epoch: 138 Training Loss: 1.276127 Validation Loss: 0.320572 Epoch: 139 Training Loss: 1.276283 Validation Loss: 0.319130 Epoch: 140 Training Loss: 1.274293 Validation Loss: 0.324264 Epoch: 141 Training Loss: 1.276226 Validation Loss: 0.318521 Epoch: 142 Training Loss: 1.273648 Validation Loss: 0.317698 Epoch: 143 Training Loss: 1.280384 Validation Loss: 0.318762 Epoch: 144 Training Loss: 1.271613 Validation Loss: 0.321056 Epoch: 145 Training Loss: 1.279159 Validation Loss: 0.319677 Epoch: 146 Training Loss: 1.277133 Validation Loss: 0.313412 Epoch: 147 Training Loss: 1.273115 Validation Loss: 0.316693 Epoch: 148 Training Loss: 1.276824 Validation Loss: 0.324270 Epoch: 149 Training Loss: 1.271500 Validation Loss: 0.317610 Epoch: 150 Training Loss: 1.274339 Validation Loss: 0.319794 Epoch: 151 Training Loss: 1.276326 Validation Loss: 0.316618 Epoch: 152 Training Loss: 1.274265 Validation Loss: 0.317560 Epoch: 153 Training Loss: 1.273693 Validation Loss: 0.315664 Epoch: 154 Training Loss: 1.271308 Validation Loss: 0.314383 Epoch: 155 Training Loss: 1.275785 Validation Loss: 0.311731 Epoch: 156 Training Loss: 1.269926 Validation Loss: 0.317802 Epoch: 157 Training Loss: 1.272163 Validation Loss: 0.326034 Epoch: 158 Training Loss: 1.272792 Validation Loss: 0.323937 Epoch: 159 Training Loss: 1.270623 Validation Loss: 0.314596 Epoch: 160 Training Loss: 1.274752 Validation Loss: 0.318708 Epoch: 161 Training Loss: 1.269636 Validation Loss: 0.315447 Epoch: 162 Training Loss: 1.268630 Validation Loss: 0.318611 Epoch: 163 Training Loss: 1.269201 Validation Loss: 0.321739 Epoch: 164 Training Loss: 1.268440 Validation Loss: 0.318679 Epoch: 165 Training Loss: 1.267896 Validation Loss: 0.317043 Epoch: 166 Training Loss: 1.268580 Validation Loss: 0.319146 Epoch: 167 Training Loss: 1.275538 Validation Loss: 0.317928 Epoch: 168 Training Loss: 1.268560 Validation Loss: 0.323980 Epoch: 169 Training Loss: 1.268632 Validation Loss: 0.313479 Epoch: 170 Training Loss: 1.264794 Validation Loss: 0.318113 Epoch: 171 Training Loss: 1.270822 Validation Loss: 0.313195 Epoch: 172 Training Loss: 1.267813 Validation Loss: 0.317769 Epoch: 173 Training Loss: 1.270347 Validation Loss: 0.315005 Epoch: 174 Training Loss: 1.266662 Validation Loss: 0.314660 Epoch: 175 Training Loss: 1.268849 Validation Loss: 0.319801 Epoch: 176 Training Loss: 1.271820 Validation Loss: 0.320086 Epoch: 177 Training Loss: 1.273374 Validation Loss: 0.318641 Epoch: 178 Training Loss: 1.265961 Validation Loss: 0.314708 Epoch: 179 Training Loss: 1.271811 Validation Loss: 0.322507 Epoch: 180 Training Loss: 1.263662 Validation Loss: 0.323136 Epoch: 181 Training Loss: 1.269750 Validation Loss: 0.314223 Epoch: 182 Training Loss: 1.269853 Validation Loss: 0.321011 Epoch: 183 Training Loss: 1.267138 Validation Loss: 0.313789 Epoch: 184 Training Loss: 1.271545 Validation Loss: 0.321742 Epoch: 185 Training Loss: 1.268025 Validation Loss: 0.316022 Epoch: 186 Training Loss: 1.272954 Validation Loss: 0.324468 Epoch: 187 Training Loss: 1.267895 Validation Loss: 0.314698 Epoch: 188 Training Loss: 1.266716 Validation Loss: 0.318999 Epoch: 189 Training Loss: 1.263130 Validation Loss: 0.319963 Epoch: 190 Training Loss: 1.270730 Validation Loss: 0.319453 Epoch: 191 Training Loss: 1.265955 Validation Loss: 0.314691 Epoch: 192 Training Loss: 1.267399 Validation Loss: 0.321611 Epoch: 193 Training Loss: 1.264792 Validation Loss: 0.320243 Epoch: 194 Training Loss: 1.262446 Validation Loss: 0.314628 Epoch: 195 Training Loss: 1.262605 Validation Loss: 0.312932 Epoch: 196 Training Loss: 1.265456 Validation Loss: 0.313259 Epoch: 197 Training Loss: 1.269357 Validation Loss: 0.311136 Epoch: 198 Training Loss: 1.262179 Validation Loss: 0.312693 Epoch: 199 Training Loss: 1.266902 Validation Loss: 0.313880 Epoch: 200 Training Loss: 1.265160 Validation Loss: 0.312400 Epoch: 201 Training Loss: 1.266844 Validation Loss: 0.316210 Epoch: 202 Training Loss: 1.264941 Validation Loss: 0.317070 Epoch: 203 Training Loss: 1.267308 Validation Loss: 0.321297 Epoch: 204 Training Loss: 1.265302 Validation Loss: 0.318993 Epoch: 205 Training Loss: 1.265829 Validation Loss: 0.313469 Epoch: 206 Training Loss: 1.261570 Validation Loss: 0.321749 Epoch: 207 Training Loss: 1.266412 Validation Loss: 0.310708 Validation loss decreased (0.311118 --> 0.310708). Saving model ... Epoch: 208 Training Loss: 1.266944 Validation Loss: 0.318451 Epoch: 209 Training Loss: 1.265850 Validation Loss: 0.315396 Epoch: 210 Training Loss: 1.264065 Validation Loss: 0.315393 Epoch: 211 Training Loss: 1.258434 Validation Loss: 0.315945 Epoch: 212 Training Loss: 1.262104 Validation Loss: 0.317880 Epoch: 213 Training Loss: 1.266053 Validation Loss: 0.326606 Epoch: 214 Training Loss: 1.264815 Validation Loss: 0.317249 Epoch: 215 Training Loss: 1.265139 Validation Loss: 0.319844 Epoch: 216 Training Loss: 1.266425 Validation Loss: 0.320103 Epoch: 217 Training Loss: 1.265218 Validation Loss: 0.313683 Epoch: 218 Training Loss: 1.261013 Validation Loss: 0.316373 Epoch: 219 Training Loss: 1.262247 Validation Loss: 0.313101 Epoch: 220 Training Loss: 1.264393 Validation Loss: 0.314501 Epoch: 221 Training Loss: 1.264149 Validation Loss: 0.315623 Epoch: 222 Training Loss: 1.259319 Validation Loss: 0.318756 Epoch: 223 Training Loss: 1.258570 Validation Loss: 0.319732 Epoch: 224 Training Loss: 1.259029 Validation Loss: 0.311516 Epoch: 225 Training Loss: 1.266348 Validation Loss: 0.314770 Epoch: 226 Training Loss: 1.259851 Validation Loss: 0.321516 Epoch: 227 Training Loss: 1.262397 Validation Loss: 0.314634 Epoch: 228 Training Loss: 1.258319 Validation Loss: 0.314885 Epoch: 229 Training Loss: 1.257705 Validation Loss: 0.313776 Epoch: 230 Training Loss: 1.265772 Validation Loss: 0.317983 Epoch: 231 Training Loss: 1.256625 Validation Loss: 0.315058 Epoch: 232 Training Loss: 1.259640 Validation Loss: 0.315233 Epoch: 233 Training Loss: 1.257951 Validation Loss: 0.312612 Epoch: 234 Training Loss: 1.259246 Validation Loss: 0.318067 Epoch: 235 Training Loss: 1.254118 Validation Loss: 0.319640 Epoch: 236 Training Loss: 1.261764 Validation Loss: 0.323842 Epoch: 237 Training Loss: 1.257337 Validation Loss: 0.312940 Epoch: 238 Training Loss: 1.261468 Validation Loss: 0.312802 Epoch: 239 Training Loss: 1.256006 Validation Loss: 0.317805 Epoch: 240 Training Loss: 1.259415 Validation Loss: 0.313486 Epoch: 241 Training Loss: 1.256178 Validation Loss: 0.314875 Epoch: 242 Training Loss: 1.256519 Validation Loss: 0.313054 Epoch: 243 Training Loss: 1.255753 Validation Loss: 0.310222 Validation loss decreased (0.310708 --> 0.310222). Saving model ... Epoch: 244 Training Loss: 1.258942 Validation Loss: 0.329567 Epoch: 245 Training Loss: 1.258942 Validation Loss: 0.311769 Epoch: 246 Training Loss: 1.262446 Validation Loss: 0.313582 Epoch: 247 Training Loss: 1.261230 Validation Loss: 0.318076 Epoch: 248 Training Loss: 1.261161 Validation Loss: 0.314736 Epoch: 249 Training Loss: 1.259770 Validation Loss: 0.313956 Epoch: 250 Training Loss: 1.256420 Validation Loss: 0.312800 Epoch: 251 Training Loss: 1.262006 Validation Loss: 0.316093 Epoch: 252 Training Loss: 1.259628 Validation Loss: 0.314459 Epoch: 253 Training Loss: 1.255323 Validation Loss: 0.320948 Epoch: 254 Training Loss: 1.251152 Validation Loss: 0.312966 Epoch: 255 Training Loss: 1.263651 Validation Loss: 0.324031 Epoch: 256 Training Loss: 1.258022 Validation Loss: 0.317772 Epoch: 257 Training Loss: 1.260936 Validation Loss: 0.316249 Epoch: 258 Training Loss: 1.257661 Validation Loss: 0.318002 Epoch: 259 Training Loss: 1.253739 Validation Loss: 0.317531 Epoch: 260 Training Loss: 1.259165 Validation Loss: 0.318186 Epoch: 261 Training Loss: 1.255523 Validation Loss: 0.315747 Epoch: 262 Training Loss: 1.260258 Validation Loss: 0.323450 Epoch: 263 Training Loss: 1.256247 Validation Loss: 0.315790 Epoch: 264 Training Loss: 1.256523 Validation Loss: 0.322588 Epoch: 265 Training Loss: 1.256251 Validation Loss: 0.316159 Epoch: 266 Training Loss: 1.254540 Validation Loss: 0.317133 Epoch: 267 Training Loss: 1.256788 Validation Loss: 0.320573 Epoch: 268 Training Loss: 1.261198 Validation Loss: 0.326142 Epoch: 269 Training Loss: 1.255286 Validation Loss: 0.311760 Epoch: 270 Training Loss: 1.256038 Validation Loss: 0.320824 Epoch: 271 Training Loss: 1.252561 Validation Loss: 0.313171 Epoch: 272 Training Loss: 1.257770 Validation Loss: 0.318307 Epoch: 273 Training Loss: 1.254161 Validation Loss: 0.309804 Validation loss decreased (0.310222 --> 0.309804). Saving model ... Epoch: 274 Training Loss: 1.256829 Validation Loss: 0.318989 Epoch: 275 Training Loss: 1.264886 Validation Loss: 0.317026 Epoch: 276 Training Loss: 1.250972 Validation Loss: 0.315094 Epoch: 277 Training Loss: 1.255500 Validation Loss: 0.324168 Epoch: 278 Training Loss: 1.253158 Validation Loss: 0.321396 Epoch: 279 Training Loss: 1.258170 Validation Loss: 0.320225 Epoch: 280 Training Loss: 1.258867 Validation Loss: 0.318569 Epoch: 281 Training Loss: 1.254345 Validation Loss: 0.316465 Epoch: 282 Training Loss: 1.255778 Validation Loss: 0.314160 Epoch: 283 Training Loss: 1.254325 Validation Loss: 0.313069 Epoch: 284 Training Loss: 1.253357 Validation Loss: 0.328138 Epoch: 285 Training Loss: 1.251017 Validation Loss: 0.316133 Epoch: 286 Training Loss: 1.252227 Validation Loss: 0.316984 Epoch: 287 Training Loss: 1.253182 Validation Loss: 0.313943 Epoch: 288 Training Loss: 1.250671 Validation Loss: 0.318114 Epoch: 289 Training Loss: 1.255845 Validation Loss: 0.316618 Epoch: 290 Training Loss: 1.255237 Validation Loss: 0.312792 Epoch: 291 Training Loss: 1.262059 Validation Loss: 0.314828 Epoch: 292 Training Loss: 1.255877 Validation Loss: 0.318905 Epoch: 293 Training Loss: 1.254416 Validation Loss: 0.314216 Epoch: 294 Training Loss: 1.253497 Validation Loss: 0.314790 Epoch: 295 Training Loss: 1.255368 Validation Loss: 0.321991 Epoch: 296 Training Loss: 1.257793 Validation Loss: 0.317706 Epoch: 297 Training Loss: 1.251250 Validation Loss: 0.316808 Epoch: 298 Training Loss: 1.252172 Validation Loss: 0.315334 Epoch: 299 Training Loss: 1.251001 Validation Loss: 0.314154 Epoch: 300 Training Loss: 1.252786 Validation Loss: 0.320209 Epoch: 301 Training Loss: 1.257268 Validation Loss: 0.319915 Elapsed: 1:15:46.335776
It seems to be improving, but really slowly.
test(model_2)
Test Loss: 1.307058 Test Accuracy of airplane: 57% (572/1000) Test Accuracy of automobile: 73% (735/1000) Test Accuracy of bird: 26% (266/1000) Test Accuracy of cat: 35% (357/1000) Test Accuracy of deer: 52% (525/1000) Test Accuracy of dog: 19% (193/1000) Test Accuracy of frog: 79% (798/1000) Test Accuracy of horse: 59% (598/1000) Test Accuracy of ship: 81% (810/1000) Test Accuracy of truck: 49% (494/1000) Test Accuracy (Overall): 53% (5348/10000)
model_2 = CNN()
model_2.to(device)
start = datetime.now()
model_2.load_state_dict(torch.load(outcome["hyperparameters_file"],
map_location=device))
outcome_2 = train_and_pickle(model_2, epochs=200, model_number=2)
outcome = update_outcome(outcome, outcome_2)
print("Elapsed: {}".format(datetime.now() - start))
model_2.load_state_dict(torch.load(outcome["hyperparameters_file"],
map_location=device))
test(model_2)
Epoch: 202 Training Loss: 1.256388 Validation Loss: 0.313784 Validation loss decreased (inf --> 0.313784). Saving model ... Epoch: 203 Training Loss: 1.258825 Validation Loss: 0.317360 Epoch: 204 Training Loss: 1.256599 Validation Loss: 0.316243 Epoch: 205 Training Loss: 1.253339 Validation Loss: 0.322061 Epoch: 206 Training Loss: 1.260164 Validation Loss: 0.319589 Epoch: 207 Training Loss: 1.252303 Validation Loss: 0.318219 Epoch: 208 Training Loss: 1.257676 Validation Loss: 0.326530 Epoch: 209 Training Loss: 1.258256 Validation Loss: 0.322288 Epoch: 210 Training Loss: 1.257436 Validation Loss: 0.316848 Epoch: 211 Training Loss: 1.256364 Validation Loss: 0.313047 Validation loss decreased (0.313784 --> 0.313047). Saving model ... Epoch: 212 Training Loss: 1.259785 Validation Loss: 0.321005 Epoch: 213 Training Loss: 1.254453 Validation Loss: 0.307325 Validation loss decreased (0.313047 --> 0.307325). Saving model ... Epoch: 214 Training Loss: 1.254806 Validation Loss: 0.320826 Epoch: 215 Training Loss: 1.252779 Validation Loss: 0.320929 Epoch: 216 Training Loss: 1.252038 Validation Loss: 0.320515 Epoch: 217 Training Loss: 1.252444 Validation Loss: 0.317522 Epoch: 218 Training Loss: 1.254665 Validation Loss: 0.313467 Epoch: 219 Training Loss: 1.255900 Validation Loss: 0.315710 Epoch: 220 Training Loss: 1.252430 Validation Loss: 0.321523 Epoch: 221 Training Loss: 1.256561 Validation Loss: 0.310884 Epoch: 222 Training Loss: 1.255160 Validation Loss: 0.309861 Epoch: 223 Training Loss: 1.254754 Validation Loss: 0.319757 Epoch: 224 Training Loss: 1.255497 Validation Loss: 0.318309 Epoch: 225 Training Loss: 1.260697 Validation Loss: 0.314599 Epoch: 226 Training Loss: 1.253136 Validation Loss: 0.318721 Epoch: 227 Training Loss: 1.257839 Validation Loss: 0.312620 Epoch: 228 Training Loss: 1.248965 Validation Loss: 0.320385 Epoch: 229 Training Loss: 1.251453 Validation Loss: 0.318191 Epoch: 230 Training Loss: 1.252814 Validation Loss: 0.324980 Epoch: 231 Training Loss: 1.256732 Validation Loss: 0.318312 Epoch: 232 Training Loss: 1.251452 Validation Loss: 0.319930 Epoch: 233 Training Loss: 1.251726 Validation Loss: 0.311095 Epoch: 234 Training Loss: 1.250112 Validation Loss: 0.318118 Epoch: 235 Training Loss: 1.255064 Validation Loss: 0.311329 Epoch: 236 Training Loss: 1.250156 Validation Loss: 0.322847 Epoch: 237 Training Loss: 1.249897 Validation Loss: 0.310835 Epoch: 238 Training Loss: 1.251495 Validation Loss: 0.322079 Epoch: 239 Training Loss: 1.247715 Validation Loss: 0.321563 Epoch: 240 Training Loss: 1.248373 Validation Loss: 0.328171 Epoch: 241 Training Loss: 1.250492 Validation Loss: 0.321683 Epoch: 242 Training Loss: 1.255231 Validation Loss: 0.313710 Epoch: 243 Training Loss: 1.247742 Validation Loss: 0.318332 Epoch: 244 Training Loss: 1.251414 Validation Loss: 0.315995 Epoch: 245 Training Loss: 1.258454 Validation Loss: 0.317433 Epoch: 246 Training Loss: 1.253335 Validation Loss: 0.317605 Epoch: 247 Training Loss: 1.253148 Validation Loss: 0.316049 Epoch: 248 Training Loss: 1.251510 Validation Loss: 0.312951 Epoch: 249 Training Loss: 1.251977 Validation Loss: 0.321403 Epoch: 250 Training Loss: 1.256146 Validation Loss: 0.320409 Epoch: 251 Training Loss: 1.248189 Validation Loss: 0.317272 Epoch: 252 Training Loss: 1.254679 Validation Loss: 0.317682 Epoch: 253 Training Loss: 1.253137 Validation Loss: 0.317845 Epoch: 254 Training Loss: 1.258417 Validation Loss: 0.317278 Epoch: 255 Training Loss: 1.253359 Validation Loss: 0.319818 Epoch: 256 Training Loss: 1.247390 Validation Loss: 0.320857 Epoch: 257 Training Loss: 1.255359 Validation Loss: 0.317702 Epoch: 258 Training Loss: 1.247608 Validation Loss: 0.316204 Epoch: 259 Training Loss: 1.249561 Validation Loss: 0.312899 Epoch: 260 Training Loss: 1.248591 Validation Loss: 0.322027 Epoch: 261 Training Loss: 1.248232 Validation Loss: 0.316189 Epoch: 262 Training Loss: 1.252761 Validation Loss: 0.317912 Epoch: 263 Training Loss: 1.246621 Validation Loss: 0.317565 Epoch: 264 Training Loss: 1.249730 Validation Loss: 0.321344 Epoch: 265 Training Loss: 1.253313 Validation Loss: 0.317789 Epoch: 266 Training Loss: 1.250943 Validation Loss: 0.319828 Epoch: 267 Training Loss: 1.248345 Validation Loss: 0.319927 Epoch: 268 Training Loss: 1.248811 Validation Loss: 0.316677 Epoch: 269 Training Loss: 1.250617 Validation Loss: 0.311661 Epoch: 270 Training Loss: 1.250927 Validation Loss: 0.324976 Epoch: 271 Training Loss: 1.246129 Validation Loss: 0.321428 Epoch: 272 Training Loss: 1.247270 Validation Loss: 0.313739 Epoch: 273 Training Loss: 1.252439 Validation Loss: 0.314271 Epoch: 274 Training Loss: 1.249031 Validation Loss: 0.315256 Epoch: 275 Training Loss: 1.248926 Validation Loss: 0.318519 Epoch: 276 Training Loss: 1.253851 Validation Loss: 0.317292 Epoch: 277 Training Loss: 1.248241 Validation Loss: 0.312578 Epoch: 278 Training Loss: 1.246958 Validation Loss: 0.317017 Epoch: 279 Training Loss: 1.247038 Validation Loss: 0.317870 Epoch: 280 Training Loss: 1.247711 Validation Loss: 0.320040 Epoch: 281 Training Loss: 1.250939 Validation Loss: 0.319092 Epoch: 282 Training Loss: 1.250168 Validation Loss: 0.318878 Epoch: 283 Training Loss: 1.249140 Validation Loss: 0.323233 Epoch: 284 Training Loss: 1.247192 Validation Loss: 0.320423 Epoch: 285 Training Loss: 1.248637 Validation Loss: 0.321254 Epoch: 286 Training Loss: 1.246468 Validation Loss: 0.322253 Epoch: 287 Training Loss: 1.247990 Validation Loss: 0.316660 Epoch: 288 Training Loss: 1.245704 Validation Loss: 0.327530 Epoch: 289 Training Loss: 1.244317 Validation Loss: 0.316667 Epoch: 290 Training Loss: 1.247457 Validation Loss: 0.316587 Epoch: 291 Training Loss: 1.244423 Validation Loss: 0.323431 Epoch: 292 Training Loss: 1.245140 Validation Loss: 0.319670 Epoch: 293 Training Loss: 1.247903 Validation Loss: 0.315965 Epoch: 294 Training Loss: 1.248071 Validation Loss: 0.314560 Epoch: 295 Training Loss: 1.244779 Validation Loss: 0.321430 Epoch: 296 Training Loss: 1.250301 Validation Loss: 0.314018 Epoch: 297 Training Loss: 1.251302 Validation Loss: 0.316015 Epoch: 298 Training Loss: 1.253560 Validation Loss: 0.315506 Epoch: 299 Training Loss: 1.246812 Validation Loss: 0.323061 Epoch: 300 Training Loss: 1.248937 Validation Loss: 0.315299 Epoch: 301 Training Loss: 1.248918 Validation Loss: 0.318701 Epoch: 302 Training Loss: 1.247325 Validation Loss: 0.315778 Epoch: 303 Training Loss: 1.241974 Validation Loss: 0.315274 Epoch: 304 Training Loss: 1.250347 Validation Loss: 0.315380 Epoch: 305 Training Loss: 1.244912 Validation Loss: 0.316511 Epoch: 306 Training Loss: 1.247815 Validation Loss: 0.317746 Epoch: 307 Training Loss: 1.250566 Validation Loss: 0.314758 Epoch: 308 Training Loss: 1.249454 Validation Loss: 0.317377 Epoch: 309 Training Loss: 1.249325 Validation Loss: 0.316275 Epoch: 310 Training Loss: 1.248658 Validation Loss: 0.319433 Epoch: 311 Training Loss: 1.244979 Validation Loss: 0.312409 Epoch: 312 Training Loss: 1.250389 Validation Loss: 0.319627 Epoch: 313 Training Loss: 1.245450 Validation Loss: 0.318461 Epoch: 314 Training Loss: 1.247308 Validation Loss: 0.318554 Epoch: 315 Training Loss: 1.247195 Validation Loss: 0.316582 Epoch: 316 Training Loss: 1.244136 Validation Loss: 0.318103 Epoch: 317 Training Loss: 1.249054 Validation Loss: 0.319848 Epoch: 318 Training Loss: 1.248777 Validation Loss: 0.323786 Epoch: 319 Training Loss: 1.247198 Validation Loss: 0.315047 Epoch: 320 Training Loss: 1.251294 Validation Loss: 0.318657 Epoch: 321 Training Loss: 1.249177 Validation Loss: 0.337516 Epoch: 322 Training Loss: 1.247499 Validation Loss: 0.326684 Epoch: 323 Training Loss: 1.246539 Validation Loss: 0.319658 Epoch: 324 Training Loss: 1.248925 Validation Loss: 0.313511 Epoch: 325 Training Loss: 1.243196 Validation Loss: 0.315549 Epoch: 326 Training Loss: 1.244999 Validation Loss: 0.321060 Epoch: 327 Training Loss: 1.248777 Validation Loss: 0.317293 Epoch: 328 Training Loss: 1.248694 Validation Loss: 0.317218 Epoch: 329 Training Loss: 1.251560 Validation Loss: 0.317921 Epoch: 330 Training Loss: 1.252284 Validation Loss: 0.317201 Epoch: 331 Training Loss: 1.246083 Validation Loss: 0.321029 Epoch: 332 Training Loss: 1.244893 Validation Loss: 0.316990 Epoch: 333 Training Loss: 1.240543 Validation Loss: 0.317590 Epoch: 334 Training Loss: 1.246393 Validation Loss: 0.325721 Epoch: 335 Training Loss: 1.248191 Validation Loss: 0.320632 Epoch: 336 Training Loss: 1.241560 Validation Loss: 0.324130 Epoch: 337 Training Loss: 1.243119 Validation Loss: 0.318852 Epoch: 338 Training Loss: 1.242660 Validation Loss: 0.319926 Epoch: 339 Training Loss: 1.249028 Validation Loss: 0.315266 Epoch: 340 Training Loss: 1.244741 Validation Loss: 0.324272 Epoch: 341 Training Loss: 1.244523 Validation Loss: 0.318710 Epoch: 342 Training Loss: 1.241070 Validation Loss: 0.319939 Epoch: 343 Training Loss: 1.244101 Validation Loss: 0.321822 Epoch: 344 Training Loss: 1.239239 Validation Loss: 0.315630 Epoch: 345 Training Loss: 1.245509 Validation Loss: 0.318808 Epoch: 346 Training Loss: 1.245012 Validation Loss: 0.320597 Epoch: 347 Training Loss: 1.251397 Validation Loss: 0.318575 Epoch: 348 Training Loss: 1.240546 Validation Loss: 0.313607 Epoch: 349 Training Loss: 1.245582 Validation Loss: 0.317309 Epoch: 350 Training Loss: 1.240588 Validation Loss: 0.319662 Epoch: 351 Training Loss: 1.241194 Validation Loss: 0.316204 Epoch: 352 Training Loss: 1.243081 Validation Loss: 0.321423 Epoch: 353 Training Loss: 1.244287 Validation Loss: 0.316278 Epoch: 354 Training Loss: 1.248997 Validation Loss: 0.322080 Epoch: 355 Training Loss: 1.243133 Validation Loss: 0.314357 Epoch: 356 Training Loss: 1.240463 Validation Loss: 0.317619 Epoch: 357 Training Loss: 1.249085 Validation Loss: 0.317623 Epoch: 358 Training Loss: 1.244508 Validation Loss: 0.316843 Epoch: 359 Training Loss: 1.252762 Validation Loss: 0.317262 Epoch: 360 Training Loss: 1.246585 Validation Loss: 0.321501 Epoch: 361 Training Loss: 1.240622 Validation Loss: 0.318065 Epoch: 362 Training Loss: 1.246144 Validation Loss: 0.317386 Epoch: 363 Training Loss: 1.246127 Validation Loss: 0.314560 Epoch: 364 Training Loss: 1.244285 Validation Loss: 0.318059 Epoch: 365 Training Loss: 1.244826 Validation Loss: 0.317295 Epoch: 366 Training Loss: 1.244527 Validation Loss: 0.313897 Epoch: 367 Training Loss: 1.244683 Validation Loss: 0.325274 Epoch: 368 Training Loss: 1.245969 Validation Loss: 0.325050 Epoch: 369 Training Loss: 1.245889 Validation Loss: 0.317678 Epoch: 370 Training Loss: 1.240173 Validation Loss: 0.321540 Epoch: 371 Training Loss: 1.244970 Validation Loss: 0.318374 Epoch: 372 Training Loss: 1.242400 Validation Loss: 0.322875 Epoch: 373 Training Loss: 1.245613 Validation Loss: 0.319608 Epoch: 374 Training Loss: 1.243773 Validation Loss: 0.322040 Epoch: 375 Training Loss: 1.243070 Validation Loss: 0.320554 Epoch: 376 Training Loss: 1.245695 Validation Loss: 0.321315 Epoch: 377 Training Loss: 1.245310 Validation Loss: 0.321394 Epoch: 378 Training Loss: 1.240203 Validation Loss: 0.316470 Epoch: 379 Training Loss: 1.245251 Validation Loss: 0.317234 Epoch: 380 Training Loss: 1.250027 Validation Loss: 0.330051 Epoch: 381 Training Loss: 1.243686 Validation Loss: 0.322005 Epoch: 382 Training Loss: 1.243251 Validation Loss: 0.315280 Epoch: 383 Training Loss: 1.243953 Validation Loss: 0.326072 Epoch: 384 Training Loss: 1.245808 Validation Loss: 0.316741 Epoch: 385 Training Loss: 1.242827 Validation Loss: 0.315943 Epoch: 386 Training Loss: 1.244012 Validation Loss: 0.310488 Epoch: 387 Training Loss: 1.245015 Validation Loss: 0.314874 Epoch: 388 Training Loss: 1.244292 Validation Loss: 0.317309 Epoch: 389 Training Loss: 1.250823 Validation Loss: 0.313929 Epoch: 390 Training Loss: 1.248937 Validation Loss: 0.314966 Epoch: 391 Training Loss: 1.249134 Validation Loss: 0.321290 Epoch: 392 Training Loss: 1.246164 Validation Loss: 0.316047 Epoch: 393 Training Loss: 1.249995 Validation Loss: 0.318678 Epoch: 394 Training Loss: 1.240377 Validation Loss: 0.327256 Epoch: 395 Training Loss: 1.247659 Validation Loss: 0.317254 Epoch: 396 Training Loss: 1.238285 Validation Loss: 0.314723 Epoch: 397 Training Loss: 1.245013 Validation Loss: 0.324809 Epoch: 398 Training Loss: 1.247650 Validation Loss: 0.330501 Epoch: 399 Training Loss: 1.250368 Validation Loss: 0.318667 Epoch: 400 Training Loss: 1.246211 Validation Loss: 0.323798 Epoch: 401 Training Loss: 1.239634 Validation Loss: 0.322877 Epoch: 402 Training Loss: 1.248236 Validation Loss: 0.321464 Elapsed: 1:17:57.592411 Test Loss: 1.336450 Test Accuracy of airplane: 55% (553/1000) Test Accuracy of automobile: 58% (583/1000) Test Accuracy of bird: 23% (234/1000) Test Accuracy of cat: 30% (307/1000) Test Accuracy of deer: 36% (365/1000) Test Accuracy of dog: 25% (257/1000) Test Accuracy of frog: 88% (880/1000) Test Accuracy of horse: 69% (694/1000) Test Accuracy of ship: 76% (766/1000) Test Accuracy of truck: 61% (611/1000) Test Accuracy (Overall): 52% (5250/10000)
model_2 = CNN()
model_2.to(device)
start = datetime.now()
model_2.load_state_dict(torch.load(outcome["hyperparameters_file"],
map_location=device))
outcome_2 = train_and_pickle(model_2, epochs=200, model_number=2)
outcome = update_outcome(outcome, outcome_2)
print("Elapsed: {}".format(datetime.now() - start))
model_2.load_state_dict(torch.load(outcome["hyperparameters_file"],
map_location=device))
test(model_2)
Epoch: 400 Training Loss: 1.085763 Validation Loss: 0.282825 Validation loss decreased (inf --> 0.282825). Saving model ... Epoch: 401 Training Loss: 1.094224 Validation Loss: 0.282336 Validation loss decreased (0.282825 --> 0.282336). Saving model ... Epoch: 402 Training Loss: 1.090027 Validation Loss: 0.283988 Epoch: 403 Training Loss: 1.088251 Validation Loss: 0.282374 Epoch: 404 Training Loss: 1.088617 Validation Loss: 0.280398 Validation loss decreased (0.282336 --> 0.280398). Saving model ... Epoch: 405 Training Loss: 1.092081 Validation Loss: 0.280428 Epoch: 406 Training Loss: 1.091815 Validation Loss: 0.278766 Validation loss decreased (0.280398 --> 0.278766). Saving model ... Epoch: 407 Training Loss: 1.088024 Validation Loss: 0.281447 Epoch: 408 Training Loss: 1.094500 Validation Loss: 0.283386 Epoch: 409 Training Loss: 1.089597 Validation Loss: 0.281148 Epoch: 410 Training Loss: 1.091652 Validation Loss: 0.283893 Epoch: 411 Training Loss: 1.087357 Validation Loss: 0.281366 Epoch: 412 Training Loss: 1.091122 Validation Loss: 0.286320 Epoch: 413 Training Loss: 1.089693 Validation Loss: 0.282684 Epoch: 414 Training Loss: 1.088109 Validation Loss: 0.284077 Epoch: 415 Training Loss: 1.087701 Validation Loss: 0.280002 Epoch: 416 Training Loss: 1.085328 Validation Loss: 0.282377 Epoch: 417 Training Loss: 1.089087 Validation Loss: 0.282623 Epoch: 418 Training Loss: 1.086825 Validation Loss: 0.278291 Validation loss decreased (0.278766 --> 0.278291). Saving model ... Epoch: 419 Training Loss: 1.086601 Validation Loss: 0.282585 Epoch: 420 Training Loss: 1.082824 Validation Loss: 0.282660 Epoch: 421 Training Loss: 1.089363 Validation Loss: 0.281838 Epoch: 422 Training Loss: 1.087070 Validation Loss: 0.279197 Epoch: 423 Training Loss: 1.084032 Validation Loss: 0.281605 Epoch: 424 Training Loss: 1.087307 Validation Loss: 0.281069 Epoch: 425 Training Loss: 1.090275 Validation Loss: 0.286235 Epoch: 426 Training Loss: 1.084863 Validation Loss: 0.286024 Epoch: 427 Training Loss: 1.086919 Validation Loss: 0.283765 Epoch: 428 Training Loss: 1.087431 Validation Loss: 0.287237 Epoch: 429 Training Loss: 1.084115 Validation Loss: 0.279592 Epoch: 430 Training Loss: 1.093677 Validation Loss: 0.283081 Epoch: 431 Training Loss: 1.090348 Validation Loss: 0.281837 Epoch: 432 Training Loss: 1.088213 Validation Loss: 0.277247 Validation loss decreased (0.278291 --> 0.277247). Saving model ... Epoch: 433 Training Loss: 1.089605 Validation Loss: 0.278821 Epoch: 434 Training Loss: 1.085192 Validation Loss: 0.276951 Validation loss decreased (0.277247 --> 0.276951). Saving model ... Epoch: 435 Training Loss: 1.085776 Validation Loss: 0.281023 Epoch: 436 Training Loss: 1.086465 Validation Loss: 0.283929 Epoch: 437 Training Loss: 1.087985 Validation Loss: 0.282887 Epoch: 438 Training Loss: 1.086791 Validation Loss: 0.278656 Epoch: 439 Training Loss: 1.087146 Validation Loss: 0.284559 Epoch: 440 Training Loss: 1.086268 Validation Loss: 0.284008 Epoch: 441 Training Loss: 1.074737 Validation Loss: 0.282008 Epoch: 442 Training Loss: 1.090836 Validation Loss: 0.280691 Epoch: 443 Training Loss: 1.086444 Validation Loss: 0.283169 Epoch: 444 Training Loss: 1.083751 Validation Loss: 0.277424 Epoch: 445 Training Loss: 1.084478 Validation Loss: 0.282735 Epoch: 446 Training Loss: 1.087853 Validation Loss: 0.279917 Epoch: 447 Training Loss: 1.087905 Validation Loss: 0.278547 Epoch: 448 Training Loss: 1.083655 Validation Loss: 0.284014 Epoch: 449 Training Loss: 1.085713 Validation Loss: 0.284066 Epoch: 450 Training Loss: 1.082967 Validation Loss: 0.283472 Epoch: 451 Training Loss: 1.087737 Validation Loss: 0.281544 Epoch: 452 Training Loss: 1.084897 Validation Loss: 0.283131 Epoch: 453 Training Loss: 1.085416 Validation Loss: 0.283956 Epoch: 454 Training Loss: 1.079511 Validation Loss: 0.284032 Epoch: 455 Training Loss: 1.081187 Validation Loss: 0.277546 Epoch: 456 Training Loss: 1.081564 Validation Loss: 0.283062 Epoch: 457 Training Loss: 1.090161 Validation Loss: 0.277227 Epoch: 458 Training Loss: 1.082555 Validation Loss: 0.281654 Epoch: 459 Training Loss: 1.084783 Validation Loss: 0.282357 Epoch: 460 Training Loss: 1.086960 Validation Loss: 0.283228 Epoch: 461 Training Loss: 1.088104 Validation Loss: 0.283043 Epoch: 462 Training Loss: 1.079098 Validation Loss: 0.280849 Epoch: 463 Training Loss: 1.077743 Validation Loss: 0.279460 Epoch: 464 Training Loss: 1.080590 Validation Loss: 0.281254 Epoch: 465 Training Loss: 1.083514 Validation Loss: 0.280558 Epoch: 466 Training Loss: 1.089853 Validation Loss: 0.277356 Epoch: 467 Training Loss: 1.080071 Validation Loss: 0.279764 Epoch: 468 Training Loss: 1.083149 Validation Loss: 0.280320 Epoch: 469 Training Loss: 1.086154 Validation Loss: 0.278509 Epoch: 470 Training Loss: 1.075413 Validation Loss: 0.277589 Epoch: 471 Training Loss: 1.090838 Validation Loss: 0.284972 Epoch: 472 Training Loss: 1.083023 Validation Loss: 0.280417 Epoch: 473 Training Loss: 1.078518 Validation Loss: 0.279890 Epoch: 474 Training Loss: 1.081342 Validation Loss: 0.282047 Epoch: 475 Training Loss: 1.082641 Validation Loss: 0.277632 Epoch: 476 Training Loss: 1.077731 Validation Loss: 0.282896 Epoch: 477 Training Loss: 1.074824 Validation Loss: 0.278524 Epoch: 478 Training Loss: 1.081040 Validation Loss: 0.282670 Epoch: 479 Training Loss: 1.078880 Validation Loss: 0.281313 Epoch: 480 Training Loss: 1.077215 Validation Loss: 0.280679 Epoch: 481 Training Loss: 1.081206 Validation Loss: 0.278332 Epoch: 482 Training Loss: 1.084885 Validation Loss: 0.278158 Epoch: 483 Training Loss: 1.075072 Validation Loss: 0.277820 Epoch: 484 Training Loss: 1.081011 Validation Loss: 0.284402 Epoch: 485 Training Loss: 1.081351 Validation Loss: 0.281961 Epoch: 486 Training Loss: 1.083745 Validation Loss: 0.279679 Epoch: 487 Training Loss: 1.081245 Validation Loss: 0.280318 Epoch: 488 Training Loss: 1.075557 Validation Loss: 0.278577 Epoch: 489 Training Loss: 1.079408 Validation Loss: 0.278910 Epoch: 490 Training Loss: 1.082496 Validation Loss: 0.280904 Epoch: 491 Training Loss: 1.078611 Validation Loss: 0.277847 Epoch: 492 Training Loss: 1.087269 Validation Loss: 0.280784 Epoch: 493 Training Loss: 1.080308 Validation Loss: 0.280509 Epoch: 494 Training Loss: 1.079977 Validation Loss: 0.280467 Epoch: 495 Training Loss: 1.071035 Validation Loss: 0.277071 Epoch: 496 Training Loss: 1.081492 Validation Loss: 0.279537 Epoch: 497 Training Loss: 1.076939 Validation Loss: 0.277763 Epoch: 498 Training Loss: 1.076834 Validation Loss: 0.277170 Epoch: 499 Training Loss: 1.077066 Validation Loss: 0.281241 Epoch: 500 Training Loss: 1.078915 Validation Loss: 0.278007 Elapsed: 1:41:06.408824
test(model_2)
Test Loss: 1.336450 Test Accuracy of airplane: 55% (553/1000) Test Accuracy of automobile: 58% (583/1000) Test Accuracy of bird: 23% (234/1000) Test Accuracy of cat: 30% (307/1000) Test Accuracy of deer: 36% (365/1000) Test Accuracy of dog: 25% (257/1000) Test Accuracy of frog: 88% (880/1000) Test Accuracy of horse: 69% (694/1000) Test Accuracy of ship: 76% (766/1000) Test Accuracy of truck: 61% (611/1000) Test Accuracy (Overall): 52% (5250/10000)
figure, axe = pyplot.subplots()
figure.suptitle("Filter Size 5 Training/Validation Loss", weight="bold")
x = numpy.arange(len(training_loss_2))
axe.plot(x, training_loss_2, label="Training")
axe.plot(x, validation_loss_2, label="Validation")
axe.set_xlabel("Epoch")
axe.set_ylabel("Cross-Entropy Loss")
labeled = False
for improvement in improvements_2:
label = "_" if labeled else "Model Improved"
axe.axvline(improvement, color='r', linestyle='--', label=label)
labeled = True
legend = axe.legend()
It looks like the model from the Pytorch tutorial starts to overfit after the 15th epoch (by count, not index).
Udacity Model
model_1 = CNN(3)
model_1.to(device)
filename_1, training_loss_1, validation_loss_1, improvements_1 = train(model_1, epochs=30, model_number=1)
Epoch: 1 Training Loss: 1.764122 Validation Loss: 0.408952 Validation loss decreased (inf --> 0.408952). Saving model ... Epoch: 2 Training Loss: 1.586364 Validation Loss: 0.383241 Validation loss decreased (0.408952 --> 0.383241). Saving model ... Epoch: 3 Training Loss: 1.519929 Validation Loss: 0.371740 Validation loss decreased (0.383241 --> 0.371740). Saving model ... Epoch: 4 Training Loss: 1.488349 Validation Loss: 0.362653 Validation loss decreased (0.371740 --> 0.362653). Saving model ... Epoch: 5 Training Loss: 1.455125 Validation Loss: 0.358624 Validation loss decreased (0.362653 --> 0.358624). Saving model ... Epoch: 6 Training Loss: 1.431836 Validation Loss: 0.353852 Validation loss decreased (0.358624 --> 0.353852). Saving model ... Epoch: 7 Training Loss: 1.406383 Validation Loss: 0.351643 Validation loss decreased (0.353852 --> 0.351643). Saving model ... Epoch: 8 Training Loss: 1.396167 Validation Loss: 0.342488 Validation loss decreased (0.351643 --> 0.342488). Saving model ... Epoch: 9 Training Loss: 1.374800 Validation Loss: 0.344513 Epoch: 10 Training Loss: 1.365321 Validation Loss: 0.339705 Validation loss decreased (0.342488 --> 0.339705). Saving model ... Epoch: 11 Training Loss: 1.350646 Validation Loss: 0.334100 Validation loss decreased (0.339705 --> 0.334100). Saving model ... Epoch: 12 Training Loss: 1.336463 Validation Loss: 0.342720 Epoch: 13 Training Loss: 1.327740 Validation Loss: 0.329569 Validation loss decreased (0.334100 --> 0.329569). Saving model ... Epoch: 14 Training Loss: 1.318054 Validation Loss: 0.330011 Epoch: 15 Training Loss: 1.318000 Validation Loss: 0.331113 Epoch: 16 Training Loss: 1.307698 Validation Loss: 0.325177 Validation loss decreased (0.329569 --> 0.325177). Saving model ... Epoch: 17 Training Loss: 1.300564 Validation Loss: 0.324221 Validation loss decreased (0.325177 --> 0.324221). Saving model ... Epoch: 18 Training Loss: 1.298909 Validation Loss: 0.323380 Validation loss decreased (0.324221 --> 0.323380). Saving model ... Epoch: 19 Training Loss: 1.284629 Validation Loss: 0.317989 Validation loss decreased (0.323380 --> 0.317989). Saving model ... Epoch: 20 Training Loss: 1.284566 Validation Loss: 0.316856 Validation loss decreased (0.317989 --> 0.316856). Saving model ... Epoch: 21 Training Loss: 1.276280 Validation Loss: 0.320113 Epoch: 22 Training Loss: 1.274713 Validation Loss: 0.320777 Epoch: 23 Training Loss: 1.267952 Validation Loss: 0.317876 Epoch: 24 Training Loss: 1.270328 Validation Loss: 0.311076 Validation loss decreased (0.316856 --> 0.311076). Saving model ... Epoch: 25 Training Loss: 1.258179 Validation Loss: 0.313508 Epoch: 26 Training Loss: 1.253091 Validation Loss: 0.314421 Epoch: 27 Training Loss: 1.254100 Validation Loss: 0.312774 Epoch: 28 Training Loss: 1.244802 Validation Loss: 0.311225 Epoch: 29 Training Loss: 1.242637 Validation Loss: 0.310512 Validation loss decreased (0.311076 --> 0.310512). Saving model ... Epoch: 30 Training Loss: 1.245316 Validation Loss: 0.311031
figure, axe = pyplot.subplots()
figure.suptitle("Filter Size 3 Training/Validation Loss", weight="bold")
x = numpy.arange(len(training_loss_1))
axe.plot(x, training_loss_1, label="Training")
axe.plot(x, validation_loss_1, label="Validation")
axe.set_xlabel("Epoch")
axe.set_ylabel("Cross-Entropy Loss")
labeled = False
for improvement in improvements_1:
label = "_" if labeled else "Model Improved"
axe.axvline(improvement, color='r', linestyle='--', label=label)
labeled = True
legend = axe.legend()
So it looks like there isn't much difference between the models, but the filter size of 3 did slightly better.
Load the Model with the Lowest Validation Loss
model_2.load_state_dict(torch.load(outcome["hyperparameters_file"]))
best_model = model_2
Test the Trained Network
Test your trained model on previously unseen data! A "good" result will be a CNN that gets around 70% (or more, try your best!) accuracy on these test images.
def test(best_model):
criterion = nn.CrossEntropyLoss()
# track test loss
test_loss = 0.0
class_correct = list(0. for i in range(10))
class_total = list(0. for i in range(10))
best_model.to(device)
best_model.eval()
# iterate over test data
for data, target in test_loader:
# move tensors to GPU if CUDA is available
data, target = data.to(device), target.to(device)
# forward pass: compute predicted outputs by passing inputs to the model
output = best_model(data)
# calculate the batch loss
loss = criterion(output, target)
# update test loss
test_loss += loss.item() * data.size(0)
# convert output probabilities to predicted class
_, pred = torch.max(output, 1)
# compare predictions to true label
correct_tensor = pred.eq(target.data.view_as(pred))
correct = (
numpy.squeeze(correct_tensor.numpy())
if not train_on_gpu
else numpy.squeeze(correct_tensor.cpu().numpy()))
# calculate test accuracy for each object class
for i in range(BATCH_SIZE):
label = target.data[i]
class_correct[label] += correct[i].item()
class_total[label] += 1
# average test loss
test_loss = test_loss/len(test_loader.dataset)
print('Test Loss: {:.6f}\n'.format(test_loss))
for i in range(10):
if class_total[i] > 0:
print('Test Accuracy of %5s: %2d%% (%2d/%2d)' % (
classes[i], 100 * class_correct[i] / class_total[i],
numpy.sum(class_correct[i]), numpy.sum(class_total[i])))
else:
print('Test Accuracy of %5s: N/A (no training examples)' % (classes[i]))
print('\nTest Accuracy (Overall): %2d%% (%2d/%2d)' % (
100. * numpy.sum(class_correct) / numpy.sum(class_total),
numpy.sum(class_correct), numpy.sum(class_total)))
dataiter = iter(test_loader) images, labels = dataiter.next() images.numpy()
if train_on_gpu: images = images.cuda()
output = model(images)
_, preds_tensor = torch.max(output, 1) preds = np.squeeze(preds_tensor.numpy()) if not train_on_gpu else np.squeeze(preds_tensor.cpu().numpy())
fig = plt.figure(figsize=(25, 4)) for idx in np.arange(20): ax = fig.add_subplot(2, 20/2, idx+1, xticks=[], yticks=[]) imshow(images[idx]) ax.set_title("{} ({})".format(classes[preds[idx]], classes[labels[idx]]), color=("green" if preds[idx]==labels[idx].item() else "red"))
Make it Easier
means = deviations = (0.5, 0.5, 0.5)
train_transform = transforms.Compose([
transforms.ToTensor(),
transforms.Normalize(means, deviations)
])
test_transforms = transforms.Compose([
transforms.ToTensor(),
transforms.Normalize(means,
deviations)])
training_data = datasets.CIFAR10(path.folder, train=True,
download=True, transform=train_transform)
test_data = datasets.CIFAR10(path.folder, train=False,
download=True, transform=test_transforms)
Files already downloaded and verified Files already downloaded and verified
indices = list(range(len(training_data)))
training_indices, validation_indices = train_test_split(
indices,
test_size=VALIDATION_FRACTION)
train_sampler = SubsetRandomSampler(training_indices)
valid_sampler = SubsetRandomSampler(validation_indices)
train_loader = torch.utils.data.DataLoader(training_data, batch_size=BATCH_SIZE,
sampler=train_sampler, num_workers=NUM_WORKERS)
valid_loader = torch.utils.data.DataLoader(training_data, batch_size=BATCH_SIZE,
sampler=valid_sampler, num_workers=NUM_WORKERS)
test_loader = torch.utils.data.DataLoader(test_data, batch_size=BATCH_SIZE,
num_workers=NUM_WORKERS)
def load_and_train(model_number:int=3, epochs:int=100) -> dict:
"""Load the model using hyperparameters in the dict
Args:
model_number: identifier for the model (and its pickles)
epochs: how many times to repeat training
Returns:
outcome: trained model and outcome dict
"""
model = CNN()
model = model.to(device)
start = datetime.now()
outcome = train_and_pickle(
model,
epochs=epochs,
model_number=model_number)
model.load_state_dict(torch.load(outcome["hyperparameters_file"],
map_location=device))
test(model)
ended = datetime.now()
print("Ended: {}".format(ended))
print("Elapsed: {}".format(ended - start))
return outcome
model_3 = CNN()
model_3.to(device)
start = datetime.now()
outcome = train_and_pickle(
model_3,
epochs=100,
model_number=3)
print("Elapsed: {}".format(datetime.now() - start))
model_3.load_state_dict(torch.load(outcome["hyperparameters_file"],
map_location=device))
test(model_3)
Epoch: 404 Training Loss: 1.766535 Validation Loss: 0.404297 Validation loss decreased (inf --> 0.404297). Saving model ... Epoch: 405 Training Loss: 1.539740 Validation Loss: 0.351437 Validation loss decreased (0.404297 --> 0.351437). Saving model ... Epoch: 406 Training Loss: 1.408876 Validation Loss: 0.327341 Validation loss decreased (0.351437 --> 0.327341). Saving model ... Epoch: 407 Training Loss: 1.325226 Validation Loss: 0.303032 Validation loss decreased (0.327341 --> 0.303032). Saving model ... Epoch: 408 Training Loss: 1.260864 Validation Loss: 0.291623 Validation loss decreased (0.303032 --> 0.291623). Saving model ... Epoch: 409 Training Loss: 1.214102 Validation Loss: 0.283056 Validation loss decreased (0.291623 --> 0.283056). Saving model ... Epoch: 410 Training Loss: 1.178166 Validation Loss: 0.275751 Validation loss decreased (0.283056 --> 0.275751). Saving model ... Epoch: 411 Training Loss: 1.146879 Validation Loss: 0.264309 Validation loss decreased (0.275751 --> 0.264309). Saving model ... Epoch: 412 Training Loss: 1.121644 Validation Loss: 0.258764 Validation loss decreased (0.264309 --> 0.258764). Saving model ... Epoch: 413 Training Loss: 1.097969 Validation Loss: 0.252846 Validation loss decreased (0.258764 --> 0.252846). Saving model ... Epoch: 414 Training Loss: 1.078815 Validation Loss: 0.250729 Validation loss decreased (0.252846 --> 0.250729). Saving model ... Epoch: 415 Training Loss: 1.055899 Validation Loss: 0.241823 Validation loss decreased (0.250729 --> 0.241823). Saving model ... Epoch: 416 Training Loss: 1.041387 Validation Loss: 0.238933 Validation loss decreased (0.241823 --> 0.238933). Saving model ... Epoch: 417 Training Loss: 1.029270 Validation Loss: 0.234940 Validation loss decreased (0.238933 --> 0.234940). Saving model ... Epoch: 418 Training Loss: 1.016113 Validation Loss: 0.232727 Validation loss decreased (0.234940 --> 0.232727). Saving model ... Epoch: 419 Training Loss: 1.005521 Validation Loss: 0.226466 Validation loss decreased (0.232727 --> 0.226466). Saving model ... Epoch: 420 Training Loss: 0.992684 Validation Loss: 0.226542 Epoch: 421 Training Loss: 0.978596 Validation Loss: 0.225691 Validation loss decreased (0.226466 --> 0.225691). Saving model ... Epoch: 422 Training Loss: 0.976063 Validation Loss: 0.228258 Epoch: 423 Training Loss: 0.961974 Validation Loss: 0.221933 Validation loss decreased (0.225691 --> 0.221933). Saving model ... Epoch: 424 Training Loss: 0.954803 Validation Loss: 0.220159 Validation loss decreased (0.221933 --> 0.220159). Saving model ... Epoch: 425 Training Loss: 0.948879 Validation Loss: 0.219641 Validation loss decreased (0.220159 --> 0.219641). Saving model ... Epoch: 426 Training Loss: 0.945494 Validation Loss: 0.220472 Epoch: 427 Training Loss: 0.935160 Validation Loss: 0.215726 Validation loss decreased (0.219641 --> 0.215726). Saving model ... Epoch: 428 Training Loss: 0.928077 Validation Loss: 0.215445 Validation loss decreased (0.215726 --> 0.215445). Saving model ... Epoch: 429 Training Loss: 0.925603 Validation Loss: 0.212353 Validation loss decreased (0.215445 --> 0.212353). Saving model ... Epoch: 430 Training Loss: 0.921984 Validation Loss: 0.208420 Validation loss decreased (0.212353 --> 0.208420). Saving model ... Epoch: 431 Training Loss: 0.912180 Validation Loss: 0.218620 Epoch: 432 Training Loss: 0.909916 Validation Loss: 0.208612 Epoch: 433 Training Loss: 0.902665 Validation Loss: 0.208177 Validation loss decreased (0.208420 --> 0.208177). Saving model ... Epoch: 434 Training Loss: 0.899616 Validation Loss: 0.210920 Epoch: 435 Training Loss: 0.895718 Validation Loss: 0.212328 Epoch: 436 Training Loss: 0.883933 Validation Loss: 0.204341 Validation loss decreased (0.208177 --> 0.204341). Saving model ... Epoch: 437 Training Loss: 0.888972 Validation Loss: 0.206792 Epoch: 438 Training Loss: 0.878481 Validation Loss: 0.204317 Validation loss decreased (0.204341 --> 0.204317). Saving model ... Epoch: 439 Training Loss: 0.879559 Validation Loss: 0.204447 Epoch: 440 Training Loss: 0.871985 Validation Loss: 0.203039 Validation loss decreased (0.204317 --> 0.203039). Saving model ... Epoch: 441 Training Loss: 0.870123 Validation Loss: 0.202717 Validation loss decreased (0.203039 --> 0.202717). Saving model ... Epoch: 442 Training Loss: 0.870877 Validation Loss: 0.201654 Validation loss decreased (0.202717 --> 0.201654). Saving model ... Epoch: 443 Training Loss: 0.863020 Validation Loss: 0.204858 Epoch: 444 Training Loss: 0.861419 Validation Loss: 0.202981 Epoch: 445 Training Loss: 0.864864 Validation Loss: 0.200853 Validation loss decreased (0.201654 --> 0.200853). Saving model ... Epoch: 446 Training Loss: 0.859879 Validation Loss: 0.202888 Epoch: 447 Training Loss: 0.859062 Validation Loss: 0.199505 Validation loss decreased (0.200853 --> 0.199505). Saving model ... Epoch: 448 Training Loss: 0.853924 Validation Loss: 0.196931 Validation loss decreased (0.199505 --> 0.196931). Saving model ... Epoch: 449 Training Loss: 0.849512 Validation Loss: 0.201266 Epoch: 450 Training Loss: 0.845482 Validation Loss: 0.196021 Validation loss decreased (0.196931 --> 0.196021). Saving model ... Epoch: 451 Training Loss: 0.844360 Validation Loss: 0.195308 Validation loss decreased (0.196021 --> 0.195308). Saving model ... Epoch: 452 Training Loss: 0.844023 Validation Loss: 0.197164 Epoch: 453 Training Loss: 0.839186 Validation Loss: 0.194882 Validation loss decreased (0.195308 --> 0.194882). Saving model ... Epoch: 454 Training Loss: 0.838193 Validation Loss: 0.198097 Epoch: 455 Training Loss: 0.837155 Validation Loss: 0.197095 Epoch: 456 Training Loss: 0.831614 Validation Loss: 0.195633 Epoch: 457 Training Loss: 0.827912 Validation Loss: 0.195327 Epoch: 458 Training Loss: 0.830631 Validation Loss: 0.192197 Validation loss decreased (0.194882 --> 0.192197). Saving model ... Epoch: 459 Training Loss: 0.825767 Validation Loss: 0.195351 Epoch: 460 Training Loss: 0.824248 Validation Loss: 0.192982 Epoch: 461 Training Loss: 0.822047 Validation Loss: 0.191864 Validation loss decreased (0.192197 --> 0.191864). Saving model ... Epoch: 462 Training Loss: 0.824057 Validation Loss: 0.191095 Validation loss decreased (0.191864 --> 0.191095). Saving model ... Epoch: 463 Training Loss: 0.821909 Validation Loss: 0.190179 Validation loss decreased (0.191095 --> 0.190179). Saving model ... Epoch: 464 Training Loss: 0.820941 Validation Loss: 0.193425 Epoch: 465 Training Loss: 0.820359 Validation Loss: 0.193750 Epoch: 466 Training Loss: 0.815640 Validation Loss: 0.194663 Epoch: 467 Training Loss: 0.818372 Validation Loss: 0.192682 Epoch: 468 Training Loss: 0.817113 Validation Loss: 0.192452 Epoch: 469 Training Loss: 0.817581 Validation Loss: 0.196727 Epoch: 470 Training Loss: 0.809651 Validation Loss: 0.190927 Epoch: 471 Training Loss: 0.811329 Validation Loss: 0.194151 Epoch: 472 Training Loss: 0.806093 Validation Loss: 0.192417 Epoch: 473 Training Loss: 0.806517 Validation Loss: 0.189602 Validation loss decreased (0.190179 --> 0.189602). Saving model ... Epoch: 474 Training Loss: 0.807954 Validation Loss: 0.191487 Epoch: 475 Training Loss: 0.807010 Validation Loss: 0.191636 Epoch: 476 Training Loss: 0.801799 Validation Loss: 0.190896 Epoch: 477 Training Loss: 0.798797 Validation Loss: 0.187708 Validation loss decreased (0.189602 --> 0.187708). Saving model ... Epoch: 478 Training Loss: 0.799128 Validation Loss: 0.189194 Epoch: 479 Training Loss: 0.799459 Validation Loss: 0.194036 Epoch: 480 Training Loss: 0.795995 Validation Loss: 0.190724 Epoch: 481 Training Loss: 0.798655 Validation Loss: 0.190721 Epoch: 482 Training Loss: 0.792206 Validation Loss: 0.188309 Epoch: 483 Training Loss: 0.799025 Validation Loss: 0.187985 Epoch: 484 Training Loss: 0.791694 Validation Loss: 0.186556 Validation loss decreased (0.187708 --> 0.186556). Saving model ... Epoch: 485 Training Loss: 0.784249 Validation Loss: 0.184879 Validation loss decreased (0.186556 --> 0.184879). Saving model ... Epoch: 486 Training Loss: 0.793165 Validation Loss: 0.185806 Epoch: 487 Training Loss: 0.791051 Validation Loss: 0.189010 Epoch: 488 Training Loss: 0.787608 Validation Loss: 0.186931 Epoch: 489 Training Loss: 0.789344 Validation Loss: 0.195780 Epoch: 490 Training Loss: 0.792061 Validation Loss: 0.191099 Epoch: 491 Training Loss: 0.786356 Validation Loss: 0.189476 Epoch: 492 Training Loss: 0.784223 Validation Loss: 0.192026 Epoch: 493 Training Loss: 0.785188 Validation Loss: 0.189652 Epoch: 494 Training Loss: 0.782519 Validation Loss: 0.188833 Epoch: 495 Training Loss: 0.786059 Validation Loss: 0.192020 Epoch: 496 Training Loss: 0.782317 Validation Loss: 0.187162 Epoch: 497 Training Loss: 0.785475 Validation Loss: 0.191352 Epoch: 498 Training Loss: 0.778186 Validation Loss: 0.193208 Epoch: 499 Training Loss: 0.780198 Validation Loss: 0.190525 Epoch: 500 Training Loss: 0.778074 Validation Loss: 0.194126 Epoch: 501 Training Loss: 0.778832 Validation Loss: 0.186440 Epoch: 502 Training Loss: 0.776556 Validation Loss: 0.188577 Epoch: 503 Training Loss: 0.774062 Validation Loss: 0.190385 Epoch: 504 Training Loss: 0.776408 Validation Loss: 0.188763 Elapsed: 0:28:16.205032 Test Loss: 0.925722 Test Accuracy of airplane: 70% (703/1000) Test Accuracy of automobile: 75% (753/1000) Test Accuracy of bird: 47% (470/1000) Test Accuracy of cat: 56% (562/1000) Test Accuracy of deer: 69% (697/1000) Test Accuracy of dog: 53% (536/1000) Test Accuracy of frog: 80% (803/1000) Test Accuracy of horse: 67% (670/1000) Test Accuracy of ship: 82% (825/1000) Test Accuracy of truck: 75% (756/1000) Test Accuracy (Overall): 67% (6775/10000)
test(model_3)
Test Loss: 0.954966 Test Accuracy of airplane: 62% (629/1000) Test Accuracy of automobile: 76% (766/1000) Test Accuracy of bird: 50% (506/1000) Test Accuracy of cat: 44% (449/1000) Test Accuracy of deer: 66% (666/1000) Test Accuracy of dog: 52% (521/1000) Test Accuracy of frog: 82% (827/1000) Test Accuracy of horse: 72% (721/1000) Test Accuracy of ship: 82% (829/1000) Test Accuracy of truck: 67% (672/1000) Test Accuracy (Overall): 65% (6586/10000)
outcome = load_and_train(outcome)
Validation loss decreased (inf --> 0.396734). Saving model ... Validation loss decreased (0.396734 --> 0.353406). Saving model ... Validation loss decreased (0.353406 --> 0.312288). Saving model ... Validation loss decreased (0.312288 --> 0.292421). Saving model ... Validation loss decreased (0.292421 --> 0.281344). Saving model ... Validation loss decreased (0.281344 --> 0.267129). Saving model ... Validation loss decreased (0.267129 --> 0.259950). Saving model ... Validation loss decreased (0.259950 --> 0.255096). Saving model ... Validation loss decreased (0.255096 --> 0.249626). Saving model ... Epoch: 110 Training Loss: 1.074378 Validation Loss: 0.241995 Validation loss decreased (0.249626 --> 0.241995). Saving model ... Validation loss decreased (0.241995 --> 0.234253). Saving model ... Validation loss decreased (0.234253 --> 0.233744). Saving model ... Validation loss decreased (0.233744 --> 0.226195). Saving model ... Validation loss decreased (0.226195 --> 0.225804). Saving model ... Validation loss decreased (0.225804 --> 0.223489). Saving model ... Validation loss decreased (0.223489 --> 0.221263). Saving model ... Validation loss decreased (0.221263 --> 0.217546). Saving model ... Validation loss decreased (0.217546 --> 0.215720). Saving model ... Validation loss decreased (0.215720 --> 0.213332). Saving model ... Epoch: 120 Training Loss: 0.952941 Validation Loss: 0.209708 Validation loss decreased (0.213332 --> 0.209708). Saving model ... Validation loss decreased (0.209708 --> 0.207232). Saving model ... Validation loss decreased (0.207232 --> 0.205873). Saving model ... Validation loss decreased (0.205873 --> 0.199750). Saving model ... Epoch: 130 Training Loss: 0.898597 Validation Loss: 0.197858 Validation loss decreased (0.199750 --> 0.197858). Saving model ... Validation loss decreased (0.197858 --> 0.195818). Saving model ... Validation loss decreased (0.195818 --> 0.194920). Saving model ... Validation loss decreased (0.194920 --> 0.194267). Saving model ... Validation loss decreased (0.194267 --> 0.193904). Saving model ... Epoch: 140 Training Loss: 0.856769 Validation Loss: 0.203387 Validation loss decreased (0.193904 --> 0.187780). Saving model ... Epoch: 150 Training Loss: 0.842055 Validation Loss: 0.190620 Validation loss decreased (0.187780 --> 0.186874). Saving model ... Validation loss decreased (0.186874 --> 0.183554). Saving model ... Epoch: 160 Training Loss: 0.821771 Validation Loss: 0.186012 Validation loss decreased (0.183554 --> 0.183435). Saving model ... Validation loss decreased (0.183435 --> 0.183237). Saving model ... Epoch: 170 Training Loss: 0.807321 Validation Loss: 0.185445 Epoch: 180 Training Loss: 0.796137 Validation Loss: 0.182606 Validation loss decreased (0.183237 --> 0.182606). Saving model ... Validation loss decreased (0.182606 --> 0.180978). Saving model ... Validation loss decreased (0.180978 --> 0.179344). Saving model ... Epoch: 190 Training Loss: 0.792454 Validation Loss: 0.181462 Epoch: 200 Training Loss: 0.777160 Validation Loss: 0.187384 Ended: 2018-12-14 15:45:54.887063 Elapsed: 1:09:07.537337 Test Loss: 0.913029 Test Accuracy of airplane: 71% (715/1000) Test Accuracy of automobile: 80% (803/1000) Test Accuracy of bird: 44% (445/1000) Test Accuracy of cat: 49% (496/1000) Test Accuracy of deer: 73% (733/1000) Test Accuracy of dog: 54% (540/1000) Test Accuracy of frog: 78% (788/1000) Test Accuracy of horse: 74% (747/1000) Test Accuracy of ship: 84% (840/1000) Test Accuracy of truck: 75% (750/1000) Test Accuracy (Overall): 68% (6857/10000)
outcome = load_and_train(outcome)
Validation loss decreased (inf --> 0.405255). Saving model ... Validation loss decreased (0.405255 --> 0.347472). Saving model ... Validation loss decreased (0.347472 --> 0.318586). Saving model ... Validation loss decreased (0.318586 --> 0.301050). Saving model ... Validation loss decreased (0.301050 --> 0.287208). Saving model ... Validation loss decreased (0.287208 --> 0.279541). Saving model ... Epoch: 410 Training Loss: 1.188729 Validation Loss: 0.269707 Validation loss decreased (0.279541 --> 0.269707). Saving model ... Validation loss decreased (0.269707 --> 0.262379). Saving model ... Validation loss decreased (0.262379 --> 0.254531). Saving model ... Validation loss decreased (0.254531 --> 0.252625). Saving model ... Validation loss decreased (0.252625 --> 0.240125). Saving model ... Validation loss decreased (0.240125 --> 0.235959). Saving model ... Validation loss decreased (0.235959 --> 0.234190). Saving model ... Validation loss decreased (0.234190 --> 0.231890). Saving model ... Validation loss decreased (0.231890 --> 0.226316). Saving model ... Epoch: 420 Training Loss: 1.003592 Validation Loss: 0.228944 Validation loss decreased (0.226316 --> 0.224643). Saving model ... Validation loss decreased (0.224643 --> 0.222303). Saving model ... Validation loss decreased (0.222303 --> 0.221804). Saving model ... Validation loss decreased (0.221804 --> 0.219019). Saving model ... Validation loss decreased (0.219019 --> 0.211782). Saving model ... Epoch: 430 Training Loss: 0.933949 Validation Loss: 0.211028 Validation loss decreased (0.211782 --> 0.211028). Saving model ... Validation loss decreased (0.211028 --> 0.210736). Saving model ... Validation loss decreased (0.210736 --> 0.207784). Saving model ... Validation loss decreased (0.207784 --> 0.204068). Saving model ... Validation loss decreased (0.204068 --> 0.202933). Saving model ... Validation loss decreased (0.202933 --> 0.201327). Saving model ... Epoch: 440 Training Loss: 0.893833 Validation Loss: 0.201305 Validation loss decreased (0.201327 --> 0.201305). Saving model ... Validation loss decreased (0.201305 --> 0.200246). Saving model ... Validation loss decreased (0.200246 --> 0.199212). Saving model ... Validation loss decreased (0.199212 --> 0.198127). Saving model ... Validation loss decreased (0.198127 --> 0.197780). Saving model ... Epoch: 450 Training Loss: 0.869887 Validation Loss: 0.193194 Validation loss decreased (0.197780 --> 0.193194). Saving model ... Validation loss decreased (0.193194 --> 0.192689). Saving model ... Validation loss decreased (0.192689 --> 0.191178). Saving model ... Epoch: 460 Training Loss: 0.845976 Validation Loss: 0.193641 Validation loss decreased (0.191178 --> 0.190434). Saving model ... Epoch: 470 Training Loss: 0.831866 Validation Loss: 0.190801 Validation loss decreased (0.190434 --> 0.189088). Saving model ... Epoch: 480 Training Loss: 0.814776 Validation Loss: 0.190064 Validation loss decreased (0.189088 --> 0.189077). Saving model ... Validation loss decreased (0.189077 --> 0.188256). Saving model ... Validation loss decreased (0.188256 --> 0.185333). Saving model ... Epoch: 490 Training Loss: 0.804873 Validation Loss: 0.190370 Epoch: 500 Training Loss: 0.792694 Validation Loss: 0.188568 Ended: 2018-12-14 22:04:39.786682 Elapsed: 0:28:03.912152 Test Loss: 0.933276 Test Accuracy of airplane: 75% (756/1000) Test Accuracy of automobile: 74% (744/1000) Test Accuracy of bird: 48% (482/1000) Test Accuracy of cat: 44% (443/1000) Test Accuracy of deer: 68% (686/1000) Test Accuracy of dog: 50% (502/1000) Test Accuracy of frog: 80% (801/1000) Test Accuracy of horse: 68% (681/1000) Test Accuracy of ship: 82% (823/1000) Test Accuracy of truck: 77% (770/1000) Test Accuracy (Overall): 66% (6688/10000)
The overall test-accuracy is going down - is it overfitting?
with open("model_3_outcomes.pkl", "rb") as reader:
outcome = pickle.load(reader)
outcome = load_and_train(outcome)
Validation loss decreased (inf --> 0.400317). Saving model ... Validation loss decreased (0.400317 --> 0.339392). Saving model ... Epoch: 810 Training Loss: 1.361320 Validation Loss: 0.310385 Validation loss decreased (0.339392 --> 0.310385). Saving model ... Validation loss decreased (0.310385 --> 0.295311). Saving model ... Validation loss decreased (0.295311 --> 0.283410). Saving model ... Validation loss decreased (0.283410 --> 0.274456). Saving model ... Validation loss decreased (0.274456 --> 0.266069). Saving model ... Validation loss decreased (0.266069 --> 0.262745). Saving model ... Validation loss decreased (0.262745 --> 0.247262). Saving model ... Validation loss decreased (0.247262 --> 0.237769). Saving model ... Epoch: 820 Training Loss: 1.028606 Validation Loss: 0.236005 Validation loss decreased (0.237769 --> 0.236005). Saving model ... Validation loss decreased (0.236005 --> 0.230968). Saving model ... Validation loss decreased (0.230968 --> 0.228058). Saving model ... Validation loss decreased (0.228058 --> 0.224573). Saving model ... Validation loss decreased (0.224573 --> 0.223884). Saving model ... Validation loss decreased (0.223884 --> 0.219913). Saving model ... Validation loss decreased (0.219913 --> 0.217769). Saving model ... Epoch: 830 Training Loss: 0.942998 Validation Loss: 0.215061 Validation loss decreased (0.217769 --> 0.215061). Saving model ... Validation loss decreased (0.215061 --> 0.212656). Saving model ... Validation loss decreased (0.212656 --> 0.212616). Saving model ... Validation loss decreased (0.212616 --> 0.210596). Saving model ... Validation loss decreased (0.210596 --> 0.207554). Saving model ... Epoch: 840 Training Loss: 0.900498 Validation Loss: 0.208390 Validation loss decreased (0.207554 --> 0.206364). Saving model ... Validation loss decreased (0.206364 --> 0.205531). Saving model ... Validation loss decreased (0.205531 --> 0.203900). Saving model ... Epoch: 850 Training Loss: 0.872049 Validation Loss: 0.205466 Validation loss decreased (0.203900 --> 0.198664). Saving model ... Validation loss decreased (0.198664 --> 0.196482). Saving model ... Validation loss decreased (0.196482 --> 0.195664). Saving model ... Epoch: 860 Training Loss: 0.845757 Validation Loss: 0.198456 Validation loss decreased (0.195664 --> 0.193952). Saving model ... Epoch: 870 Training Loss: 0.826413 Validation Loss: 0.195060 Validation loss decreased (0.193952 --> 0.193670). Saving model ... Validation loss decreased (0.193670 --> 0.192782). Saving model ... Validation loss decreased (0.192782 --> 0.188631). Saving model ... Epoch: 880 Training Loss: 0.818928 Validation Loss: 0.199424 Epoch: 890 Training Loss: 0.808009 Validation Loss: 0.191352 Epoch: 900 Training Loss: 0.801281 Validation Loss: 0.196643 Ended: 2018-12-14 22:37:13.843477 Elapsed: 0:29:26.736300 Test Loss: 0.945705 Test Accuracy of airplane: 72% (725/1000) Test Accuracy of automobile: 78% (782/1000) Test Accuracy of bird: 47% (473/1000) Test Accuracy of cat: 48% (488/1000) Test Accuracy of deer: 70% (705/1000) Test Accuracy of dog: 52% (527/1000) Test Accuracy of frog: 77% (776/1000) Test Accuracy of horse: 69% (696/1000) Test Accuracy of ship: 84% (844/1000) Test Accuracy of truck: 70% (703/1000) Test Accuracy (Overall): 67% (6719/10000)
It looks like the overall accuracy dropped slightly beacause the best categories (truck, ship, frog) did worse but the worst categories did slightly better - although not bird for some reason.
Take two
It looks like I wasn't loading the model between each round of epochs…
outcome = load_and_train(model_number=4, epochs=200)
Epoch: 0 Training Loss: 1.784692 Validation Loss: 0.410727 Validation loss decreased (inf --> 0.410727). Saving model ... Validation loss decreased (0.410727 --> 0.360800). Saving model ... Validation loss decreased (0.360800 --> 0.314237). Saving model ... Validation loss decreased (0.314237 --> 0.293987). Saving model ... Validation loss decreased (0.293987 --> 0.283064). Saving model ... Validation loss decreased (0.283064 --> 0.275761). Saving model ... Validation loss decreased (0.275761 --> 0.270119). Saving model ... Validation loss decreased (0.270119 --> 0.261688). Saving model ... Validation loss decreased (0.261688 --> 0.254598). Saving model ... Epoch: 10 Training Loss: 1.092668 Validation Loss: 0.254406 Validation loss decreased (0.254598 --> 0.254406). Saving model ... Validation loss decreased (0.254406 --> 0.248653). Saving model ... Validation loss decreased (0.248653 --> 0.245797). Saving model ... Validation loss decreased (0.245797 --> 0.240849). Saving model ... Validation loss decreased (0.240849 --> 0.238558). Saving model ... Validation loss decreased (0.238558 --> 0.237812). Saving model ... Validation loss decreased (0.237812 --> 0.230956). Saving model ... Epoch: 20 Training Loss: 0.991010 Validation Loss: 0.225704 Validation loss decreased (0.230956 --> 0.225704). Saving model ... Validation loss decreased (0.225704 --> 0.221112). Saving model ... Validation loss decreased (0.221112 --> 0.218632). Saving model ... Epoch: 30 Training Loss: 0.938513 Validation Loss: 0.220019 Validation loss decreased (0.218632 --> 0.216886). Saving model ... Validation loss decreased (0.216886 --> 0.215869). Saving model ... Validation loss decreased (0.215869 --> 0.214766). Saving model ... Validation loss decreased (0.214766 --> 0.212452). Saving model ... Epoch: 40 Training Loss: 0.896510 Validation Loss: 0.212819 Validation loss decreased (0.212452 --> 0.209142). Saving model ... Validation loss decreased (0.209142 --> 0.208595). Saving model ... Validation loss decreased (0.208595 --> 0.205967). Saving model ... Validation loss decreased (0.205967 --> 0.205484). Saving model ... Epoch: 50 Training Loss: 0.875811 Validation Loss: 0.207912 Validation loss decreased (0.205484 --> 0.205164). Saving model ... Epoch: 60 Training Loss: 0.856581 Validation Loss: 0.208312 Validation loss decreased (0.205164 --> 0.204649). Saving model ... Validation loss decreased (0.204649 --> 0.203608). Saving model ... Epoch: 70 Training Loss: 0.846062 Validation Loss: 0.214614 Validation loss decreased (0.203608 --> 0.203064). Saving model ... Epoch: 80 Training Loss: 0.826153 Validation Loss: 0.212527 Validation loss decreased (0.203064 --> 0.201932). Saving model ... Validation loss decreased (0.201932 --> 0.200173). Saving model ... Epoch: 90 Training Loss: 0.823697 Validation Loss: 0.204494 Validation loss decreased (0.200173 --> 0.199886). Saving model ... Validation loss decreased (0.199886 --> 0.198804). Saving model ... Epoch: 100 Training Loss: 0.808043 Validation Loss: 0.205323 Epoch: 110 Training Loss: 0.805417 Validation Loss: 0.201136 Epoch: 120 Training Loss: 0.805155 Validation Loss: 0.204370 Epoch: 130 Training Loss: 0.793174 Validation Loss: 0.214048 Validation loss decreased (0.198804 --> 0.194650). Saving model ... Epoch: 140 Training Loss: 0.783871 Validation Loss: 0.200537 Epoch: 150 Training Loss: 0.781592 Validation Loss: 0.203295 Epoch: 160 Training Loss: 0.774657 Validation Loss: 0.199732 Epoch: 170 Training Loss: 0.770487 Validation Loss: 0.205331 Epoch: 180 Training Loss: 0.767693 Validation Loss: 0.202990 Epoch: 190 Training Loss: 0.767225 Validation Loss: 0.203797 Epoch: 200 Training Loss: 0.769268 Validation Loss: 0.196108 Test Loss: 0.974566 Test Accuracy of airplane: 70% (707/1000) Test Accuracy of automobile: 73% (732/1000) Test Accuracy of bird: 45% (453/1000) Test Accuracy of cat: 53% (533/1000) Test Accuracy of deer: 71% (719/1000) Test Accuracy of dog: 42% (429/1000) Test Accuracy of frog: 81% (814/1000) Test Accuracy of horse: 66% (666/1000) Test Accuracy of ship: 82% (823/1000) Test Accuracy of truck: 72% (720/1000) Test Accuracy (Overall): 65% (6596/10000) Ended: 2018-12-15 08:33:22.925579 Elapsed: 0:55:24.733457
outcome = load_and_train(model_number=4, epochs=200)
Validation loss decreased (inf --> 0.203577). Saving model ... Validation loss decreased (0.203577 --> 0.201161). Saving model ... Validation loss decreased (0.201161 --> 0.198027). Saving model ... Epoch: 210 Training Loss: 0.785905 Validation Loss: 0.199885 Epoch: 220 Training Loss: 0.780148 Validation Loss: 0.199842 Validation loss decreased (0.198027 --> 0.197471). Saving model ... Epoch: 230 Training Loss: 0.773492 Validation Loss: 0.206471 Validation loss decreased (0.197471 --> 0.195811). Saving model ... Epoch: 240 Training Loss: 0.777896 Validation Loss: 0.201046 Epoch: 250 Training Loss: 0.767602 Validation Loss: 0.203973 Epoch: 260 Training Loss: 0.765374 Validation Loss: 0.205219 Epoch: 270 Training Loss: 0.764604 Validation Loss: 0.202613 Epoch: 280 Training Loss: 0.755534 Validation Loss: 0.201307 Epoch: 290 Training Loss: 0.754538 Validation Loss: 0.199495 Epoch: 300 Training Loss: 0.759395 Validation Loss: 0.206451 Epoch: 310 Training Loss: 0.750621 Validation Loss: 0.203110 Epoch: 320 Training Loss: 0.751456 Validation Loss: 0.206920 Epoch: 330 Training Loss: 0.747122 Validation Loss: 0.199856 Epoch: 340 Training Loss: 0.742640 Validation Loss: 0.211159 Epoch: 350 Training Loss: 0.743110 Validation Loss: 0.214833 Epoch: 360 Training Loss: 0.741861 Validation Loss: 0.207520 Epoch: 370 Training Loss: 0.740826 Validation Loss: 0.210348 Epoch: 380 Training Loss: 0.740333 Validation Loss: 0.207724 Epoch: 390 Training Loss: 0.739157 Validation Loss: 0.204985 Epoch: 400 Training Loss: 0.742582 Validation Loss: 0.204150 Test Loss: 0.979350 Test Accuracy of airplane: 64% (648/1000) Test Accuracy of automobile: 75% (751/1000) Test Accuracy of bird: 43% (430/1000) Test Accuracy of cat: 50% (507/1000) Test Accuracy of deer: 76% (766/1000) Test Accuracy of dog: 44% (443/1000) Test Accuracy of frog: 81% (818/1000) Test Accuracy of horse: 63% (630/1000) Test Accuracy of ship: 86% (868/1000) Test Accuracy of truck: 68% (680/1000) Test Accuracy (Overall): 65% (6541/10000) Ended: 2018-12-15 11:19:36.845565 Elapsed: 0:55:22.008796
Change the Training and Validation Sets
INDICES = list(range(len(training_data)))
DataIterators = (torch.utils.data.dataloader.DataLoader,
torch.utils.data.dataloader.DataLoader)
def split_data() -> DataIterators:
training_indices, validation_indices = train_test_split(
INDICES,
test_size=VALIDATION_FRACTION)
train_sampler = SubsetRandomSampler(training_indices)
valid_sampler = SubsetRandomSampler(validation_indices)
train_loader = torch.utils.data.DataLoader(
training_data, batch_size=BATCH_SIZE,
sampler=train_sampler, num_workers=NUM_WORKERS)
valid_loader = torch.utils.data.DataLoader(
training_data, batch_size=BATCH_SIZE,
sampler=valid_sampler, num_workers=NUM_WORKERS)
return train_loader, valid_loader
train_loader, valid_loader = split_data()
for epoch in range(8):
outcome = load_and_train(model_number=4, epochs=50)
Validation loss decreased (inf --> 0.178021). Saving model ... Validation loss decreased (0.178021 --> 0.164977). Saving model ... Epoch: 410 Training Loss: 0.790843 Validation Loss: 0.180614 Epoch: 420 Training Loss: 0.779451 Validation Loss: 0.184705 Epoch: 430 Training Loss: 0.776067 Validation Loss: 0.188225 Epoch: 440 Training Loss: 0.767443 Validation Loss: 0.189623 Epoch: 450 Training Loss: 0.763348 Validation Loss: 0.190223 Test Loss: 0.994385 Test Accuracy of airplane: 63% (632/1000) Test Accuracy of automobile: 73% (738/1000) Test Accuracy of bird: 43% (432/1000) Test Accuracy of cat: 55% (551/1000) Test Accuracy of deer: 73% (731/1000) Test Accuracy of dog: 38% (384/1000) Test Accuracy of frog: 82% (828/1000) Test Accuracy of horse: 63% (632/1000) Test Accuracy of ship: 88% (880/1000) Test Accuracy of truck: 65% (658/1000) Test Accuracy (Overall): 64% (6466/10000) Ended: 2018-12-15 11:57:44.922535 Elapsed: 0:14:05.152783 Validation loss decreased (inf --> 0.170476). Saving model ... Epoch: 810 Training Loss: 0.791785 Validation Loss: 0.185611 Epoch: 820 Training Loss: 0.775938 Validation Loss: 0.185072 Epoch: 830 Training Loss: 0.776210 Validation Loss: 0.187146 Epoch: 840 Training Loss: 0.768063 Validation Loss: 0.182017 Epoch: 850 Training Loss: 0.769061 Validation Loss: 0.196850 Test Loss: 1.012101 Test Accuracy of airplane: 62% (624/1000) Test Accuracy of automobile: 73% (738/1000) Test Accuracy of bird: 42% (429/1000) Test Accuracy of cat: 55% (551/1000) Test Accuracy of deer: 73% (730/1000) Test Accuracy of dog: 42% (420/1000) Test Accuracy of frog: 85% (854/1000) Test Accuracy of horse: 60% (604/1000) Test Accuracy of ship: 84% (843/1000) Test Accuracy of truck: 67% (679/1000) Test Accuracy (Overall): 64% (6472/10000) Ended: 2018-12-15 12:12:04.058599 Elapsed: 0:14:19.132241 Validation loss decreased (inf --> 0.174863). Saving model ... Epoch: 1610 Training Loss: 0.797948 Validation Loss: 0.176395 Validation loss decreased (0.174863 --> 0.172779). Saving model ... Validation loss decreased (0.172779 --> 0.170694). Saving model ... Epoch: 1620 Training Loss: 0.789980 Validation Loss: 0.178468 Epoch: 1630 Training Loss: 0.772959 Validation Loss: 0.183980 Epoch: 1640 Training Loss: 0.776142 Validation Loss: 0.198711 Epoch: 1650 Training Loss: 0.767914 Validation Loss: 0.208851 Test Loss: 0.987713 Test Accuracy of airplane: 62% (624/1000) Test Accuracy of automobile: 74% (743/1000) Test Accuracy of bird: 43% (436/1000) Test Accuracy of cat: 52% (525/1000) Test Accuracy of deer: 73% (734/1000) Test Accuracy of dog: 47% (473/1000) Test Accuracy of frog: 83% (831/1000) Test Accuracy of horse: 63% (631/1000) Test Accuracy of ship: 84% (845/1000) Test Accuracy of truck: 68% (682/1000) Test Accuracy (Overall): 65% (6524/10000) Ended: 2018-12-15 12:26:50.701191 Elapsed: 0:14:46.638712 Validation loss decreased (inf --> 0.181906). Saving model ... Validation loss decreased (0.181906 --> 0.175381). Saving model ... Validation loss decreased (0.175381 --> 0.169833). Saving model ... Epoch: 3220 Training Loss: 0.776567 Validation Loss: 0.178259 Epoch: 3230 Training Loss: 0.777072 Validation Loss: 0.180300 Epoch: 3240 Training Loss: 0.770289 Validation Loss: 0.192919 Epoch: 3250 Training Loss: 0.762633 Validation Loss: 0.192530 Epoch: 3260 Training Loss: 0.760599 Validation Loss: 0.195964 Test Loss: 0.982302 Test Accuracy of airplane: 66% (665/1000) Test Accuracy of automobile: 75% (756/1000) Test Accuracy of bird: 44% (444/1000) Test Accuracy of cat: 56% (565/1000) Test Accuracy of deer: 68% (686/1000) Test Accuracy of dog: 40% (407/1000) Test Accuracy of frog: 85% (855/1000) Test Accuracy of horse: 63% (639/1000) Test Accuracy of ship: 84% (844/1000) Test Accuracy of truck: 68% (683/1000) Test Accuracy (Overall): 65% (6544/10000) Ended: 2018-12-15 12:41:47.333383 Elapsed: 0:14:56.629183 Validation loss decreased (inf --> 0.187802). Saving model ... Validation loss decreased (0.187802 --> 0.184430). Saving model ... Validation loss decreased (0.184430 --> 0.183925). Saving model ... Validation loss decreased (0.183925 --> 0.180367). Saving model ... Validation loss decreased (0.180367 --> 0.173719). Saving model ... Epoch: 6440 Training Loss: 0.778801 Validation Loss: 0.190905 Epoch: 6450 Training Loss: 0.771958 Validation Loss: 0.182070 Epoch: 6460 Training Loss: 0.764318 Validation Loss: 0.190349 Epoch: 6470 Training Loss: 0.766295 Validation Loss: 0.192508 Epoch: 6480 Training Loss: 0.761968 Validation Loss: 0.189583 Test Loss: 0.987995 Test Accuracy of airplane: 66% (661/1000) Test Accuracy of automobile: 76% (763/1000) Test Accuracy of bird: 44% (443/1000) Test Accuracy of cat: 55% (557/1000) Test Accuracy of deer: 72% (728/1000) Test Accuracy of dog: 41% (415/1000) Test Accuracy of frog: 85% (853/1000) Test Accuracy of horse: 60% (600/1000) Test Accuracy of ship: 84% (849/1000) Test Accuracy of truck: 66% (669/1000) Test Accuracy (Overall): 65% (6538/10000) Ended: 2018-12-15 12:56:04.438153 Elapsed: 0:14:17.094202 Validation loss decreased (inf --> 0.191682). Saving model ... Validation loss decreased (0.191682 --> 0.182732). Saving model ... Validation loss decreased (0.182732 --> 0.181846). Saving model ... Epoch: 12870 Training Loss: 0.770414 Validation Loss: 0.185177 Validation loss decreased (0.181846 --> 0.179913). Saving model ... Epoch: 12880 Training Loss: 0.772306 Validation Loss: 0.191702 Epoch: 12890 Training Loss: 0.768497 Validation Loss: 0.181795 Epoch: 12900 Training Loss: 0.760247 Validation Loss: 0.183884 Epoch: 12910 Training Loss: 0.757400 Validation Loss: 0.197759 Test Loss: 0.995634 Test Accuracy of airplane: 64% (648/1000) Test Accuracy of automobile: 75% (755/1000) Test Accuracy of bird: 37% (377/1000) Test Accuracy of cat: 55% (557/1000) Test Accuracy of deer: 72% (726/1000) Test Accuracy of dog: 45% (459/1000) Test Accuracy of frog: 85% (857/1000) Test Accuracy of horse: 59% (590/1000) Test Accuracy of ship: 84% (842/1000) Test Accuracy of truck: 69% (696/1000) Test Accuracy (Overall): 65% (6507/10000) Ended: 2018-12-15 13:10:05.720077 Elapsed: 0:14:01.278026 Validation loss decreased (inf --> 0.190403). Saving model ... Validation loss decreased (0.190403 --> 0.187068). Saving model ... Epoch: 25730 Training Loss: 0.768132 Validation Loss: 0.185507 Validation loss decreased (0.187068 --> 0.185507). Saving model ... Validation loss decreased (0.185507 --> 0.177258). Saving model ... Epoch: 25740 Training Loss: 0.772002 Validation Loss: 0.190112 Epoch: 25750 Training Loss: 0.760312 Validation Loss: 0.195855 Epoch: 25760 Training Loss: 0.759808 Validation Loss: 0.204542 Epoch: 25770 Training Loss: 0.756103 Validation Loss: 0.193606 Test Loss: 0.979529 Test Accuracy of airplane: 66% (663/1000) Test Accuracy of automobile: 76% (769/1000) Test Accuracy of bird: 39% (396/1000) Test Accuracy of cat: 57% (578/1000) Test Accuracy of deer: 74% (749/1000) Test Accuracy of dog: 41% (414/1000) Test Accuracy of frog: 83% (833/1000) Test Accuracy of horse: 61% (618/1000) Test Accuracy of ship: 84% (844/1000) Test Accuracy of truck: 68% (687/1000) Test Accuracy (Overall): 65% (6551/10000) Ended: 2018-12-15 13:24:12.319440 Elapsed: 0:14:06.595121 Validation loss decreased (inf --> 0.186117). Saving model ... Validation loss decreased (0.186117 --> 0.182822). Saving model ... Epoch: 51460 Training Loss: 0.767829 Validation Loss: 0.189161 Epoch: 51470 Training Loss: 0.763347 Validation Loss: 0.194681 Validation loss decreased (0.182822 --> 0.179458). Saving model ... Epoch: 51480 Training Loss: 0.756280 Validation Loss: 0.187176 Epoch: 51490 Training Loss: 0.757250 Validation Loss: 0.198088 Epoch: 51500 Training Loss: 0.754145 Validation Loss: 0.204468 Test Loss: 0.973007 Test Accuracy of airplane: 67% (676/1000) Test Accuracy of automobile: 74% (749/1000) Test Accuracy of bird: 41% (415/1000) Test Accuracy of cat: 57% (579/1000) Test Accuracy of deer: 75% (752/1000) Test Accuracy of dog: 41% (412/1000) Test Accuracy of frog: 81% (815/1000) Test Accuracy of horse: 65% (653/1000) Test Accuracy of ship: 85% (850/1000) Test Accuracy of truck: 69% (696/1000) Test Accuracy (Overall): 65% (6597/10000) Ended: 2018-12-15 13:38:06.475872 Elapsed: 0:13:54.151685
So, this model seems pretty much stuck. I cheated and peaked at the lecturer's solution, but this post is getting too long so I'll save that for another one.
figure, axe = pyplot.subplots()
figure.suptitle("Loss")
x = list(range(len(outcome["training_loss"])))
training = numpy.array(outcome["training_loss"])
limit = 500
axe.plot(x[:limit], training[:limit], ".", label="Training")
axe.plot(x[:limit], outcome["validation_loss"][:limit], ".", label="Validation")
legend = axe.legend()
So it looks like there's something wrong with my code. I'll have to figure this out (or just stick with straight epochs).