Cats and Dogs
Table of Contents
Beginning
This extends the previous lessons to harder images. We started with monochrome handwritten digits, moved to items of clothing and then artificially generated images of horses and humans. The problem with these images is that they were artificially uniform. In real life, you subject won't be centered and isolated, most pictures are messier, and this exercise will begin with images that were taken by people in real life, and thus aren't quite as orderly as the previous images were.
The dataset we're going to use here is the Dogs vs. Cats set hosted on Kaggle. It is a subset of a larger dataset compiled to test whether computers could pass the Animal Species Image Recognition for Restricting Access (ASIRRA) which was created as an alternative form of CAPTCHA which the creators of the dataset thought would prevent computers from being able to pass the test of selecting only the cats in a set of 12 photographs of cats and dogs.
Imports
Python
from csv import DictWriter
from datetime import datetime, timedelta
from functools import partial
from pathlib import Path
import random
PyPi
from holoviews.operation.datashader import datashade
from tabulate import tabulate
from tensorflow.keras.optimizers import RMSprop
from tensorflow.keras.preprocessing.image import (ImageDataGenerator,
img_to_array,
load_img)
import holoviews
import hvplot.pandas
import keras
import matplotlib.pyplot as pyplot
import numpy
import pandas
import tensorflow
Graeae
from graeae import SubPathLoader, EmbedHoloviews, Timer
Set Up
Plotting
Embed = partial(EmbedHoloviews,
folder_path="../../files/posts/keras/cats-and-dogs/")
holoviews.extension("bokeh")
The Timer
TIMER = Timer()
The Datasets
environment = SubPathLoader("DATASETS")
base_path = Path(environment["DOGS_VS_CATS"]).expanduser()
for item in base_path.iterdir():
print(item)
WARNING: Logging before flag parsing goes to stderr. I0727 20:33:31.896675 140643102242624 environment.py:35] Environment Path: /home/athena/.env I0727 20:33:31.958876 140643102242624 environment.py:90] Environment Path: /home/athena/.config/datasets/env /home/athena/data/datasets/images/dogs-vs-cats/train /home/athena/data/datasets/images/dogs-vs-cats/exercise /home/athena/data/datasets/images/dogs-vs-cats/test1
This is the dataset downloaded from kaggle. Since it's a competition, the test1 directory has the test images without labels (although as noted on the site, you could inspect them all and label them by hand) and the training set has the labeled images. Since the convention for the neural-network libraries is to ignore the file-names and to use instead the folder names I made two sub-directories - one for dogs and one for cats - and moved the images into them.
training_path = base_path/"train"
testing_path = base_path/"test1"
for item in training_path.iterdir():
print(item)
/home/athena/data/datasets/images/dogs-vs-cats/train/dogs /home/athena/data/datasets/images/dogs-vs-cats/train/cats
In the example exercise the datasets were further subdivided into training and validation sets, but keras has added validation set splitting to their image generator so it shouldn't be necessary.
The Model Storage
This is just the path to the folder to store models.
MODELS = Path("~/models/dogs-vs-cats").expanduser()
Middle
Helpers
def best_validation(data: pandas.DataFrame) -> tuple:
"""Gets and prints the rows with the best validation performance"""
best_loss = data[data.validation_loss==data.validation_loss.min()]
best_accuracy = data[data.validation_accuracy==data.validation_accuracy.max()]
print(f"Best Accuracy: {best_accuracy.validation_accuracy.iloc[0]:.2f} "
f"(loss={best_accuracy.validation_loss.iloc[0]:.2f})"
f" Epoch: {best_accuracy.index[0]}")
print(f"Best Loss: {best_loss.validation_loss.iloc[0]:.2f} "
f"(accuracy={best_loss.validation_accuracy.iloc[0]:.2f}, "
f"Epoch: {best_loss.index[0]})")
return best_accuracy, best_loss
Looking at the Cats and Dogs
How much data do we have?
cat_images = list((training_path/'cats').iterdir())
dog_images = list((training_path/'dogs').iterdir())
test_images = list(testing_path.iterdir())
print(f"Training Cat Images: {len(cat_images):,}")
print(f"Training Dog Images: {len(dog_images):,}")
print(f"Testing Images: {len(test_images):,}")
Training Cat Images: 12,500 Training Dog Images: 12,500 Testing Images: 12,500
Note that we haven't separated out the validation set yet. If we do an 80-20 split then we will have:
training_count = len(cat_images)
print(f"Training images per species: {int(training_count * .9):,}")
print(f"Validation images per species: {int(training_count * .1):,}")
Training images per species: 11,250 Validation images per species: 1,250
So 20,000 training images in total.
Looking at Some Images
height = width = 250
count = 4
columns = 4
indices = [random.randrange(len(cat_images)) for index in range(count)]
cat_plots = []
dog_plots = []
for index in indices:
cat_plots.append(holoviews.RGB.load_image(
str(cat_images[index])).opts(
height=height,
width=width))
dog_plots.append(holoviews.RGB.load_image(
str(dog_images[index])).opts(
height=height,
width=width))
plot = holoviews.Layout(cat_plots + dog_plots).cols(columns).opts(
title="Dogs vs Cats"
)
Embed(plot=plot, file_name="dogs_and_cats", height_in_pixels=600)()
So the quality and setting might vary, but they appear to be the mug-shot type photographs that most people tend to take. Some of them have borders, so they aren't uniformly sized images.
The Data Preprocessor
This will load and prepare the image batches for training and validation.
A Data Generator
This bundles up the steps to build the data generator.
class Data:
"""creates the data generators
Args:
path: path to the images
validation_split: fraction that goes to the validation set
batch_size: size for the batches in the epochs
"""
def __init__(self, path: str, validation_split: float=0.2,
batch_size: int=20) -> None:
self.path = path
self.validation_split = validation_split
self.batch_size = batch_size
self._data_generator = None
self._testing_data_generator = None
self._training_generator = None
self._validation_generator = None
return
@property
def data_generator(self) -> ImageDataGenerator:
"""The data generator for training and validation"""
if self._data_generator is None:
self._data_generator = ImageDataGenerator(
rescale=1/255,
rotation_range=40,
width_shift_range=0.2,
height_shift_range=0.2,
horizontal_flip=True,
shear_range=0.2,
zoom_range=0.2,
fill_mode="nearest",
validation_split=self.validation_split)
return self._data_generator
@property
def training_generator(self):
"""The training data generator"""
if self._training_generator is None:
self._training_generator = (self.data_generator
.flow_from_directory)(
self.path,
batch_size=self.batch_size,
class_mode="binary",
target_size=(150, 150),
subset="training",
)
return self._training_generator
@property
def validation_generator(self):
"""the validation data generator"""
if self._validation_generator is None:
self._validation_generator = (self.data_generator
.flow_from_directory)(
self.path,
batch_size=self.batch_size,
class_mode="binary",
target_size = (150, 150),
subset="validation",
)
return self._validation_generator
def __str__(self) -> str:
return (f"(Data) - Path: {self.path}, "
f"Validation Split: {self.validation_split},"
f"Batch Size: {self.batch_size}")
Some Callbacks To Stop Training
The Good Enough Callback
If our model is doing good enough on the validation set then let's stop.
class Stop(tensorflow.keras.callbacks.Callback):
"""Something to stop if we are good enough
Args:
minimum_accuracy: validation accuracy needed to quit
maximum_loss: validation loss needed to quit
check_after_batch: if True, try to interrupt after each batch, end of epoch otherwise
call_on_stopping: function or method to call when training is interrupted
"""
def __init__(self, minimum_accuracy: float=0.94, maximum_loss: float=0.24,
check_after_batch: bool=False, call_on_stopping: callable=None) -> None:
self.minimum_accuracy = minimum_accuracy
self.maximum_loss = maximum_loss
self.check_after_batch = check_after_batch
if check_after_batch:
self.on_batch_end = self.on_end_handler
else:
self.on_epoch_end = self.on_end_handler
return
def on_end_handler(self, count, logs={}):
if ("val_acc" in logs
and logs.get("val_acc") >= self.minimum_accuracy
and logs.get("val_loss") <= self.maximum_loss):
print(f"Stopping point reached at {count}")
self.model.stop_training = True
if self.call_on_stopping is not None:
self.call_on_stopping(self.model)
return
def __str__(self) -> str:
return (f"(Stop) - Minimum Accuracy: {self.minimum_accuracy}, "
f"Maximum Loss: {self.maximum_loss}, "
"By Batch: {self.check_after_batch}, "
f"Call on Stop: {self.call_on_stopping}")
A CSV Writer
class CSVLog(tensorflow.keras.callbacks.Callback):
"""A callback to store the performance
Args:
path: path to the output file
"""
def __init__(self, path: Path) -> None:
self.path = path
self._file_pointer = None
self._writer = None
return
@property
def file_pointer(self):
"""The write file"""
if self._file_pointer is None:
self._file_pointer = self.path.open("w")
return self._file_pointer
@property
def writer(self) -> DictWriter:
"""a file writer"""
if self._writer is None:
self._writer = DictWriter(
self.file_pointer,
["loss","acc","val_loss","val_acc"])
self._writer.writeheader()
return self._writer
def on_epoch_end(self, count, logs={}) -> None:
"""method to call at end of each epoch
writes to the csv file
Args:
count: number of epochs run so far
logs: dict with the epoch values
"""
if ("val_acc" in logs):
self.writer.writerow(logs)
return
A Timed Stop Callback
This timer callback comes from Make best use of a Kernel's limited uptime (Keras) by Digit Recognizer.
class TimedStop(tensorflow.keras.callbacks.Callback):
"""A callback to stop if we run out of time
Args:
minutes_allowed: Most time to allow training the model
by_batch: if True, try to stop after each batch, end of epoch otherwise
call_on_stopping: function or method to call when training is interrupted
"""
def __init__(self, minutes_allowed: int=350, check_after_batch: bool=False,
call_on_stopping: callable=None) -> None:
self.elapsed_allowed = timedelta(minutes = minutes_allowed)
self.call_on_stopping = call_on_stopping
self.check_after_batch = check_after_batch
# These are part of the Callback class
if check_after_batch:
self.on_batch_end = self.on_end_handler
else:
self.on_epoch_end = self.on_end_handler
return
def on_train_begin(self, logs: dict):
"""Called by keras at the start of training"""
self.start_time = datetime.now()
self.longest_elapsed = timedelta()
self.previous_end_time = self.start_time
self.total_elapsed = timedelta()
return
def on_end_handler(self, count: int, logs: dict):
"""called when an epoch or batch ends (depending on ```check_after_batch```)
Args:
count: the current batch or epoch
logs: the log for the batch or epoch
"""
now = datetime.now()
self.total_elapsed = now - self.start_time
this_elapsed = now - self.previous_end_time
self.last_time = now
if this_elapsed > self.longest_elapsed:
self.longest_elapsed = this_elapsed
if (self.elapsed_allowed - self.total_elapsed) < self.longest_elapsed:
# estimated time remaining will exceed our time-out
self.model.stop_training = True
print(f"TimedStop: Training out of time (Elapsed = {self.total_elapsed})")
print(f"TimedStop: Longest Epoch = {self.longest_elapsed}")
if self.call_on_stopping is not None:
self.call_on_stopping(self.model)
return
def __str__(self) -> str:
return (f"(TimedStop) - Minutes Allowed: {self.elapsed_allowed}, "
f"Check After Each Batch: {self.check_after_batch}, "
f"Call On Stopping: {self.call_on_stopping}")
Saving the Model
This gets used by the callbacks to save the model.
- Just the Weights
This saves it so you can re-use the model but it won't be trainable if you re-load it (so probably not a good idea for what I'm doing, which is just exploring).
file_name = "model_weights" def save_model_weights(model, name=file_name): """Save the models weights to disk Args: model: model whose weights to save name: base name for the file """ out_name = f"{name}.h5" print(f"Saving the Model Weights to {out_name}") model.save_weights(f"{out_name}") return
This stores everything so you can re-train it after loading the model.
def save_model(model: tensorflow.keras.models.Sequential, path: Path) -> None: """Save the model with the training state This allows you to load it and continue training Args: model: the model to save path: where to store the model """ model.save(str(path)) print(f"Saving the model to {path}") return
Building the Model
This is going to be a model with three convolutional layers feeding a dense layer with 512 neurons. We're going to resize the image to 150 x 150 x 3 so the first layer has to be set to expect that.
class Network:
"""The model to categorize the images
Args:
path: path to the training data
epochs: number of epochs to train
batch_size: size of the batches for each epoch
convolution_layers: layers of cnn/max-pooling
callbacks: things to stop the training
set_steps: whether to set the training steps-per-epoch
"""
def __init__(self, path: str, epochs: int=15,
batch_size: int=128, convolution_layers: int=3,
set_steps: bool=True,
callbacks: list=None) -> None:
self.path = path
self.epochs = epochs
self.batch_size = batch_size
self.convolution_layers = convolution_layers
self.set_steps = set_steps
self.callbacks = callbacks
self._data = None
self._model = None
self.history = None
return
@property
def data(self) -> Data:
"""The data generator builder"""
if self._data is None:
self._data = Data(self.path, batch_size=self.batch_size)
return self._data
@property
def model(self) -> tensorflow.keras.models.Sequential:
"""The neural network"""
if self._model is None:
self._model = tensorflow.keras.models.Sequential([
tensorflow.keras.layers.Conv2D(
32, (3,3), activation='relu',
input_shape=(150, 150, 3)),
tensorflow.keras.layers.MaxPooling2D(2,2)])
self._model.add(
tensorflow.keras.layers.Conv2D(
64, (3,3), activation='relu'))
self._model.add(
tensorflow.keras.layers.MaxPooling2D(2,2))
for layer in range(self.convolution_layers - 2):
self._model.add(tensorflow.keras.layers.Conv2D(
128, (3,3), activation='relu'))
self._model.add(tensorflow.keras.layers.MaxPooling2D(2,2))
for layer in [
tensorflow.keras.layers.Flatten(),
tensorflow.keras.layers.Dense(512, activation='relu'),
tensorflow.keras.layers.Dense(1, activation='sigmoid')]:
self._model.add(layer)
self._model.compile(optimizer=RMSprop(lr=0.001),
loss='binary_crossentropy',
metrics = ['acc'])
return self._model
def summary(self) -> None:
"""Prints the model summary"""
print(self.model.summary())
return
def train(self) -> None:
"""Trains the model"""
callbacks = self.callbacks if self.callbacks else []
arguments = dict(
generator=self.data.training_generator,
validation_data=self.data.validation_generator,
epochs = self.epochs,
callbacks = callbacks,
verbose=2,
)
if self.set_steps:
arguments["steps_per_epoch"] = int(
self.data.training_generator.samples/self.batch_size)
arguments["validation_steps"] = int(
self.data.validation_generator.samples/self.batch_size)
self.history = self.model.fit_generator(**arguments)
return
def __str__(self) -> str:
return (f"(Network) - \nPath: {self.path}\n Epochs: {self.epochs}\n "
f"Batch Size: {self.batch_size}\n Callbacks: {self.callbacks}\n"
f"Data: {self.data}\n"
f"Callbacks: {self.callbacks}")
Warning: I'm adapting the class definitions after running the fitting so don't run these again or the output will change.
network = Network(str(training_path))
network.summary()
Model: "sequential_1" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= conv2d_3 (Conv2D) (None, 148, 148, 16) 448 _________________________________________________________________ max_pooling2d_3 (MaxPooling2 (None, 74, 74, 16) 0 _________________________________________________________________ conv2d_4 (Conv2D) (None, 72, 72, 32) 4640 _________________________________________________________________ max_pooling2d_4 (MaxPooling2 (None, 36, 36, 32) 0 _________________________________________________________________ conv2d_5 (Conv2D) (None, 34, 34, 64) 18496 _________________________________________________________________ max_pooling2d_5 (MaxPooling2 (None, 17, 17, 64) 0 _________________________________________________________________ flatten_1 (Flatten) (None, 18496) 0 _________________________________________________________________ dense_2 (Dense) (None, 512) 9470464 _________________________________________________________________ dense_3 (Dense) (None, 1) 513 ================================================================= Total params: 9,494,561 Trainable params: 9,494,561 Non-trainable params: 0 _________________________________________________________________ None
Training the Model
network.train()
Found 20000 images belonging to 2 classes. Found 25000 images belonging to 2 classes. Epoch 1/15 20000/20000 - 3149s - loss: 0.2677 - acc: 0.8957 - val_loss: 0.2148 - val_acc: 0.9392 Epoch 2/15 20000/20000 - 3136s - loss: 0.2614 - acc: 0.9259 - val_loss: 0.2645 - val_acc: 0.9149 Epoch 3/15 20000/20000 - 3122s - loss: 0.2803 - acc: 0.9262 - val_loss: 0.2668 - val_acc: 0.9334 Epoch 4/15 20000/20000 - 3136s - loss: 0.2785 - acc: 0.9325 - val_loss: 0.3490 - val_acc: 0.9104 Epoch 5/15 20000/20000 - 3211s - loss: 0.2813 - acc: 0.9344 - val_loss: 0.2397 - val_acc: 0.9447 Epoch 6/15 20000/20000 - 3107s - loss: 0.3108 - acc: 0.9273 - val_loss: 0.3096 - val_acc: 0.9277 Epoch 7/15 20000/20000 - 3116s - loss: 0.3351 - acc: 0.9241 - val_loss: 0.2756 - val_acc: 0.9365 Epoch 8/15 20000/20000 - 3112s - loss: 0.3723 - acc: 0.9201 - val_loss: 0.5705 - val_acc: 0.9031 Epoch 9/15 20000/20000 - 3175s - loss: 0.3909 - acc: 0.9169 - val_loss: 0.3232 - val_acc: 0.8964 Epoch 10/15 20000/20000 - 3176s - loss: 0.3838 - acc: 0.9179 - val_loss: 0.4460 - val_acc: 0.8797 Epoch 11/15 20000/20000 - 3180s - loss: 0.4013 - acc: 0.9117 - val_loss: 1.5652 - val_acc: 0.8393 Epoch 12/15 20000/20000 - 3176s - loss: 0.3854 - acc: 0.9070 - val_loss: 0.2922 - val_acc: 0.9028 Epoch 13/15 20000/20000 - 3174s - loss: 0.3736 - acc: 0.9087 - val_loss: 0.3601 - val_acc: 0.8856 Epoch 14/15 20000/20000 - 3184s - loss: 0.4744 - acc: 0.9065 - val_loss: 0.5963 - val_acc: 0.7796 Epoch 15/15 20000/20000 - 3149s - loss: 0.4771 - acc: 0.8997 - val_loss: 0.3037 - val_acc: 0.8814
It looks like the model is starting to overfit, maybe we need a callback.
Take Two
I noticed that I accidentally used a batch-size of 20, which will make it converge more quickly per epoch, but make the epochs take longer, so besides adding the callback I'm going to change the batch-size.
callback = Stop()
network = Network(str(training_path), callbacks=[callback])
network.data.batch_size = 512
network.data._training_generator = None
network.data._validation_generator = None
TIMER.message = "Finished training the cats and dogs network"
with TIMER:
network.train()
2019-07-11 04:39:19,803 graeae.timers.timer start: Started: 2019-07-11 04:39:19.803252 I0711 04:39:19.803283 140573992724288 timer.py:70] Started: 2019-07-11 04:39:19.803252 Found 20000 images belonging to 2 classes. Found 5000 images belonging to 2 classes. Epoch 1/15 20000/20000 - 1765s - loss: 0.3099 - acc: 0.8796 - val_loss: 0.9553 - val_acc: 0.6900 Epoch 2/15 20000/20000 - 1764s - loss: 0.3434 - acc: 0.8976 - val_loss: 0.7495 - val_acc: 0.7854 Epoch 3/15 20000/20000 - 1765s - loss: 0.3718 - acc: 0.8923 - val_loss: 1.2745 - val_acc: 0.8064 Epoch 4/15 20000/20000 - 1774s - loss: 0.3892 - acc: 0.8904 - val_loss: 0.4991 - val_acc: 0.7906 Epoch 5/15 20000/20000 - 1775s - loss: 0.4290 - acc: 0.8786 - val_loss: 0.5698 - val_acc: 0.8338 Epoch 6/15 20000/20000 - 1774s - loss: 0.4889 - acc: 0.8847 - val_loss: 0.5719 - val_acc: 0.8078 Epoch 7/15 20000/20000 - 1774s - loss: 0.3992 - acc: 0.8883 - val_loss: 0.4647 - val_acc: 0.8258 Epoch 8/15 20000/20000 - 1774s - loss: 0.4458 - acc: 0.8853 - val_loss: 0.6354 - val_acc: 0.8098 Epoch 9/15 20000/20000 - 1774s - loss: 0.3810 - acc: 0.8856 - val_loss: 0.5611 - val_acc: 0.8198 Epoch 10/15 20000/20000 - 1773s - loss: 0.3894 - acc: 0.8886 - val_loss: 0.4861 - val_acc: 0.8476 Epoch 11/15 20000/20000 - 1771s - loss: 0.4517 - acc: 0.8972 - val_loss: 0.4623 - val_acc: 0.8420 Epoch 12/15 20000/20000 - 1755s - loss: 0.3626 - acc: 0.9030 - val_loss: 0.5481 - val_acc: 0.8310 Epoch 13/15 20000/20000 - 1753s - loss: 0.3574 - acc: 0.9054 - val_loss: 0.5138 - val_acc: 0.8430 Epoch 14/15 20000/20000 - 1753s - loss: 0.3658 - acc: 0.9132 - val_loss: 1.0924 - val_acc: 0.8164 Epoch 15/15 20000/20000 - 1753s - loss: 0.3331 - acc: 0.9145 - val_loss: 0.5148 - val_acc: 0.8312 2019-07-11 12:00:57,205 graeae.timers.timer end: Ended: 2019-07-11 12:00:57.205007 I0711 12:00:57.205034 140573992724288 timer.py:77] Ended: 2019-07-11 12:00:57.205007 2019-07-11 12:00:57,205 graeae.timers.timer end: Elapsed: 7:21:37.401755 I0711 12:00:57.205761 140573992724288 timer.py:78] Elapsed: 7:21:37.401755
So the loss and accuracy never matched what happened when we used the tiny batch size. On the other hand it took less than half the time. Maybe something in between.
Take Three
- Train Again
I'm going to use the default of 350 minutes that my
TimedStopclass has based on the time-limit that Kaggle kernels have. If it can meet the accuracy we want before this then it is trainable in one session using a kernel (I think, I haven't actually tried it yet).path = Path("~/models/dogs-vs-cats/cnn_weights_batch_size_128").expanduser() saver = partial(save_model_weights, name=str(path)) good_enough = Stop(on_interrupt=saver) out_of_time = TimedStop(on_interrupt=saver) network = Network(str(training_path), callbacks=[good_enough, out_of_time], batch_size=128) with TIMER: network.train()
2019-07-13 12:06:40,651 graeae.timers.timer start: Started: 2019-07-13 12:06:40.651269 I0713 12:06:40.651483 140400353847104 timer.py:70] Started: 2019-07-13 12:06:40.651269 Found 20000 images belonging to 2 classes. Found 5000 images belonging to 2 classes. Epoch 1/15 TimedStop: Training out of time (Elapsed = 3:19:17.730138) 20000/20000 - 11911s - loss: 0.0634 - acc: 0.9821 - val_loss: 4.4497 - val_acc: 0.8038 2019-07-13 15:25:59,863 graeae.timers.timer end: Ended: 2019-07-13 15:25:59.863265 I0713 15:25:59.863302 140400353847104 timer.py:77] Ended: 2019-07-13 15:25:59.863265 2019-07-13 15:25:59,864 graeae.timers.timer end: Elapsed: 3:19:19.211996 I0713 15:25:59.864300 140400353847104 timer.py:78] Elapsed: 3:19:19.211996
So, we have several problems here:
- It quit after just over 3 hours, not just under 6
- It only did one epoch
- It didn't come close to reaching the accuracy I wanted
The early timeout is probably because since it took over 3 hours it probably wouldn't have finished another epoch in the allotted time. Let's double-check my times.
print(f"Batch 20 First Epoch: {3136/3600:0.2f} Hours") print(f"Batch 512 First Epoch: {1765/3600: 0.2f} Hours") print(f"Batch 128 First Epoch: {11911/3600: 0.2f} Hours")
Batch 20 First Epoch: 0.87 Hours Batch 512 First Epoch: 0.49 Hours Batch 128 First Epoch: 3.31 Hours
So something went really wrong with this last run (and either my timer is wrong or the epoch times is wrong). What is it?
Take Four
Try one that times out early.
path = Path("~/models/dogs-vs-cats/cnn_weights_batch_size_256").expanduser()
saver = partial(save_model_weights, name=str(path))
good_enough = Stop(on_interrupt=saver)
out_of_time = TimedStop(1, on_interrupt=saver, by_batch=True)
network = Network(str(training_path),
callbacks=[good_enough, out_of_time],
batch_size=256)
print(str(network))
with TIMER:
network.train()
2019-07-13 16:56:05,428 graeae.timers.timer start: Started: 2019-07-13 16:56:05.428254 I0713 16:56:05.428468 140249436239680 timer.py:70] Started: 2019-07-13 16:56:05.428254 W0713 16:56:05.429713 140249436239680 deprecation.py:506] From /home/athena/.virtualenvs/In-Too-Deep/lib/python3.7/site-packages/tensorflow/python/ops/init_ops.py:1251: calling VarianceScaling.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version. Instructions for updating: Call initializer instance with the dtype argument instead of passing it to the constructor W0713 16:56:05.520384 140249436239680 deprecation.py:323] From /home/athena/.virtualenvs/In-Too-Deep/lib/python3.7/site-packages/tensorflow/python/ops/nn_impl.py:180: add_dispatch_support.<locals>.wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version. Instructions for updating: Use tf.where in 2.0, which has the same broadcast rule as np.where (Network) - Path: /home/athena/data/datasets/images/dogs-vs-cats/train, Epochs: 15, Batch Size: 256, Callbacks: [<__main__.Stop object at 0x7f8dcc0b8550>, <__main__.TimedStop object at 0x7f8dcc0b8048>],Data: (Data) - Path: /home/athena/data/datasets/images/dogs-vs-cats/train, Validation Split: 0.2,Batch Size: 256 Found 20000 images belonging to 2 classes. Found 5000 images belonging to 2 classes. Epoch 1/15 2019-07-13 16:56:36,641 graeae.timers.timer end: Ended: 2019-07-13 16:56:36.641438 I0713 16:56:36.641472 140249436239680 timer.py:77] Ended: 2019-07-13 16:56:36.641438 2019-07-13 16:56:36,643 graeae.timers.timer end: Elapsed: 0:00:31.213184 I0713 16:56:36.643917 140249436239680 timer.py:78] Elapsed: 0:00:31.213184 TimedStop: Training out of time (Elapsed = 0:00:30.441487) Saving the Model Weights to /home/athena/models/dogs-vs-cats/cnn_weights_batch_size_256.h5
Take Five
path = Path("~/models/dogs-vs-cats/cnn_weights_batch_size_256").expanduser()
saver = partial(save_model_weights, name=str(path))
good_enough = Stop(on_interrupt=saver)
out_of_time = TimedStop(on_interrupt=saver, by_batch=True)
network = Network(str(training_path),
callbacks=[good_enough, out_of_time],
batch_size=256)
print(str(network))
with TIMER:
network.train()
2019-07-13 16:59:47,492 graeae.timers.timer start: Started: 2019-07-13 16:59:47.491986 I0713 16:59:47.492009 140249436239680 timer.py:70] Started: 2019-07-13 16:59:47.491986 (Network) - Path: /home/athena/data/datasets/images/dogs-vs-cats/train, Epochs: 15, Batch Size: 256, Callbacks: [<__main__.Stop object at 0x7f8db84fa940>, <__main__.TimedStop object at 0x7f8db84fa908>],Data: (Data) - Path: /home/athena/data/datasets/images/dogs-vs-cats/train, Validation Split: 0.2,Batch Size: 256 Found 20000 images belonging to 2 classes. Found 5000 images belonging to 2 classes. Epoch 1/15 2019-07-13 19:54:49,091 graeae.timers.timer end: Ended: 2019-07-13 19:54:49.091389 I0713 19:54:49.091431 140249436239680 timer.py:77] Ended: 2019-07-13 19:54:49.091389 2019-07-13 19:54:49,093 graeae.timers.timer end: Elapsed: 2:55:01.599403 I0713 19:54:49.093391 140249436239680 timer.py:78] Elapsed: 2:55:01.599403 TimedStop: Training out of time (Elapsed = 2:55:00.848875) Saving the Model Weights to /home/athena/models/dogs-vs-cats/cnn_weights_batch_size_256.h5
Take Six
According to what I read, the smaller batch size is supposed to be faster and the larger batch-size slower, but that isn't what I saw with the 512 batch-size. Although I don't seem to see any pattern, really. Try this again.
path = Path("~/models/dogs-vs-cats/cnn_weights_batch_size_32").expanduser()
saver = partial(save_model_weights, name=str(path))
good_enough = Stop(call_on_stopping=saver, check_after_batch=True)
out_of_time = TimedStop(call_on_stopping=saver)
network = Network(str(training_path),
callbacks=[good_enough, out_of_time],
convolution_layers=5,
batch_size=32)
print(str(network))
with TIMER:
network.train()
2019-07-14 12:32:06,967 graeae.timers.timer start: Started: 2019-07-14 12:32:06.966983 I0714 12:32:06.967006 140261637994304 timer.py:70] Started: 2019-07-14 12:32:06.966983 (Network) - Path: /home/athena/data/datasets/images/dogs-vs-cats/train, Epochs: 15, Batch Size: 32, Callbacks: [<__main__.Stop object at 0x7f902c28e860>, <__main__.TimedStop object at 0x7f902c28e278>],Data: (Data) - Path: /home/athena/data/datasets/images/dogs-vs-cats/train, Validation Split: 0.2,Batch Size: 32,Callbacks: [<__main__.Stop object at 0x7f902c28e860>, <__main__.TimedStop object at 0x7f902c28e278>] Found 20000 images belonging to 2 classes. Found 5000 images belonging to 2 classes. Epoch 1/15 625/625 - 87s - loss: 0.6216 - acc: 0.6469 - val_loss: 0.5601 - val_acc: 0.6947 Epoch 2/15 625/625 - 85s - loss: 0.5002 - acc: 0.7568 - val_loss: 0.4532 - val_acc: 0.7833 Epoch 3/15 625/625 - 85s - loss: 0.4236 - acc: 0.8059 - val_loss: 0.3850 - val_acc: 0.8269 Epoch 4/15 625/625 - 85s - loss: 0.3618 - acc: 0.8400 - val_loss: 0.3532 - val_acc: 0.8397 Epoch 5/15 625/625 - 85s - loss: 0.3117 - acc: 0.8667 - val_loss: 0.4227 - val_acc: 0.8177 Epoch 6/15 625/625 - 85s - loss: 0.2683 - acc: 0.8845 - val_loss: 0.3806 - val_acc: 0.8438 Epoch 7/15 625/625 - 85s - loss: 0.2356 - acc: 0.9021 - val_loss: 0.3288 - val_acc: 0.8682 Epoch 8/15 625/625 - 85s - loss: 0.2065 - acc: 0.9143 - val_loss: 0.3912 - val_acc: 0.8678 Epoch 9/15 625/625 - 85s - loss: 0.1854 - acc: 0.9255 - val_loss: 0.3263 - val_acc: 0.8812 Epoch 10/15 625/625 - 85s - loss: 0.1629 - acc: 0.9359 - val_loss: 0.3515 - val_acc: 0.8768 Epoch 11/15 625/625 - 85s - loss: 0.1475 - acc: 0.9408 - val_loss: 0.4157 - val_acc: 0.8808 Epoch 12/15 625/625 - 85s - loss: 0.1342 - acc: 0.9486 - val_loss: 0.3752 - val_acc: 0.8740 Epoch 13/15 625/625 - 85s - loss: 0.1276 - acc: 0.9536 - val_loss: 0.4363 - val_acc: 0.8744 Epoch 14/15 625/625 - 85s - loss: 0.1233 - acc: 0.9542 - val_loss: 0.3375 - val_acc: 0.8714 Epoch 15/15 625/625 - 85s - loss: 0.1149 - acc: 0.9568 - val_loss: 0.3805 - val_acc: 0.8788 2019-07-14 12:53:20,998 graeae.timers.timer end: Ended: 2019-07-14 12:53:20.998164 I0714 12:53:20.998192 140261637994304 timer.py:77] Ended: 2019-07-14 12:53:20.998164 2019-07-14 12:53:20,999 graeae.timers.timer end: Elapsed: 0:21:14.031181 I0714 12:53:20.999267 140261637994304 timer.py:78] Elapsed: 0:21:14.031181
So I think I fixed the time problem, lets try upping the batch size.
path = Path("~/models/dogs-vs-cats/cnn_weights_batch_size_128").expanduser()
saver = partial(save_model_weights, name=str(path))
good_enough = Stop(call_on_stopping=saver, check_after_batch=True)
out_of_time = TimedStop(call_on_stopping=saver)
network = Network(str(training_path),
callbacks=[good_enough, out_of_time],
convolution_layers=5,
batch_size=128)
print(str(network))
with TIMER:
network.train()
2019-07-14 13:29:11,456 graeae.timers.timer start: Started: 2019-07-14 13:29:11.456042 I0714 13:29:11.456063 140261637994304 timer.py:70] Started: 2019-07-14 13:29:11.456042 (Network) - Path: /home/athena/data/datasets/images/dogs-vs-cats/train, Epochs: 15, Batch Size: 128, Callbacks: [<__main__.Stop object at 0x7f8fd911c9b0>, <__main__.TimedStop object at 0x7f902c09da20>],Data: (Data) - Path: /home/athena/data/datasets/images/dogs-vs-cats/train, Validation Split: 0.2,Batch Size: 128,Callbacks: [<__main__.Stop object at 0x7f8fd911c9b0>, <__main__.TimedStop object at 0x7f902c09da20>] Found 20000 images belonging to 2 classes. Found 5000 images belonging to 2 classes. Epoch 1/15 156/156 - 89s - loss: 0.6749 - acc: 0.5787 - val_loss: 0.6582 - val_acc: 0.6058 Epoch 2/15 156/156 - 89s - loss: 0.6266 - acc: 0.6545 - val_loss: 0.5754 - val_acc: 0.6967 Epoch 3/15 156/156 - 89s - loss: 0.5507 - acc: 0.7226 - val_loss: 0.5372 - val_acc: 0.7278 Epoch 4/15 156/156 - 89s - loss: 0.4921 - acc: 0.7646 - val_loss: 0.4557 - val_acc: 0.7889 Epoch 5/15 156/156 - 88s - loss: 0.4421 - acc: 0.7952 - val_loss: 0.5066 - val_acc: 0.7622 Epoch 6/15 156/156 - 87s - loss: 0.3993 - acc: 0.8194 - val_loss: 0.5141 - val_acc: 0.7684 Epoch 7/15 156/156 - 87s - loss: 0.3602 - acc: 0.8401 - val_loss: 0.4552 - val_acc: 0.7887 Epoch 8/15 156/156 - 87s - loss: 0.3194 - acc: 0.8634 - val_loss: 0.3735 - val_acc: 0.8401 Epoch 9/15 156/156 - 86s - loss: 0.2790 - acc: 0.8792 - val_loss: 0.3472 - val_acc: 0.8472 Epoch 10/15 156/156 - 86s - loss: 0.2460 - acc: 0.8964 - val_loss: 0.3934 - val_acc: 0.8281 Epoch 11/15 156/156 - 85s - loss: 0.2144 - acc: 0.9108 - val_loss: 0.3637 - val_acc: 0.8558 Epoch 12/15 156/156 - 85s - loss: 0.1857 - acc: 0.9249 - val_loss: 0.3656 - val_acc: 0.8516 Epoch 13/15 156/156 - 85s - loss: 0.1590 - acc: 0.9364 - val_loss: 0.3921 - val_acc: 0.8616 Epoch 14/15 156/156 - 84s - loss: 0.1400 - acc: 0.9449 - val_loss: 0.4903 - val_acc: 0.8297 Epoch 15/15 156/156 - 84s - loss: 0.1142 - acc: 0.9553 - val_loss: 0.4341 - val_acc: 0.8580 2019-07-14 13:50:52,575 graeae.timers.timer end: Ended: 2019-07-14 13:50:52.575319 I0714 13:50:52.575360 140261637994304 timer.py:77] Ended: 2019-07-14 13:50:52.575319 2019-07-14 13:50:52,576 graeae.timers.timer end: Elapsed: 0:21:41.119277 I0714 13:50:52.576494 140261637994304 timer.py:78] Elapsed: 0:21:41.119277
So it is now way faster but it doesn't reach the accuracy and validation we want. That's an incredible speedup, though. I guess I really wasn't using it correctly (you need to divide the number of sample by the batch size when setting the steps_per_epoch in the model.fit_generator call).
In the model.fit_generator:
steps_per_epoch=int(self.data.training_generator.samples/self.batch_size),
validation_steps=int(self.data.validation_generator.samples/self.batch_size),
Take Seven
path = Path("~/models/dogs-vs-cats/cnn_weights_batch_size_256").expanduser()
saver = partial(save_model_weights, name=str(path))
good_enough = Stop(call_on_stopping=saver,
minimum_accuracy=95)
out_of_time = TimedStop(call_on_stopping=saver)
network = Network(str(training_path),
callbacks=[good_enough, out_of_time],
convolution_layers=5,
batch_size=256)
print(str(network))
with TIMER:
network.train()
2019-07-14 14:51:25,349 graeae.timers.timer start: Started: 2019-07-14 14:51:25.349514 I0714 14:51:25.349538 140261637994304 timer.py:70] Started: 2019-07-14 14:51:25.349514 (Network) - Path: /home/athena/data/datasets/images/dogs-vs-cats/train, Epochs: 15, Batch Size: 256, Callbacks: [<__main__.Stop object at 0x7f90993397f0>, <__main__.TimedStop object at 0x7f9099339e80>],Data: (Data) - Path: /home/athena/data/datasets/images/dogs-vs-cats/train, Validation Split: 0.2,Batch Size: 256,Callbacks: [<__main__.Stop object at 0x7f90993397f0>, <__main__.TimedStop object at 0x7f9099339e80>] Found 20000 images belonging to 2 classes. Found 5000 images belonging to 2 classes. Epoch 1/15 78/78 - 89s - loss: 0.6720 - acc: 0.5733 - val_loss: 0.6325 - val_acc: 0.6295 Epoch 2/15 78/78 - 88s - loss: 0.6036 - acc: 0.6672 - val_loss: 0.6576 - val_acc: 0.6081 Epoch 3/15 78/78 - 87s - loss: 0.5626 - acc: 0.7084 - val_loss: 0.5236 - val_acc: 0.7358 Epoch 4/15 78/78 - 87s - loss: 0.5266 - acc: 0.7351 - val_loss: 0.4959 - val_acc: 0.7584 Epoch 5/15 78/78 - 85s - loss: 0.4957 - acc: 0.7556 - val_loss: 0.4607 - val_acc: 0.7767 Epoch 6/15 78/78 - 84s - loss: 0.4644 - acc: 0.7808 - val_loss: 0.4637 - val_acc: 0.7800 Epoch 7/15 78/78 - 85s - loss: 0.4296 - acc: 0.7976 - val_loss: 0.4066 - val_acc: 0.8100 Epoch 8/15 78/78 - 85s - loss: 0.3963 - acc: 0.8205 - val_loss: 0.4073 - val_acc: 0.8201 Epoch 9/15 78/78 - 83s - loss: 0.3736 - acc: 0.8294 - val_loss: 0.3809 - val_acc: 0.8248 Epoch 10/15 78/78 - 83s - loss: 0.3452 - acc: 0.8414 - val_loss: 0.4528 - val_acc: 0.7800 Epoch 11/15 78/78 - 81s - loss: 0.3128 - acc: 0.8624 - val_loss: 0.3582 - val_acc: 0.8452 Epoch 12/15 78/78 - 81s - loss: 0.2872 - acc: 0.8753 - val_loss: 0.4267 - val_acc: 0.8059 Epoch 13/15 78/78 - 80s - loss: 0.2662 - acc: 0.8864 - val_loss: 0.3332 - val_acc: 0.8557 Epoch 14/15 78/78 - 80s - loss: 0.2301 - acc: 0.9052 - val_loss: 0.3384 - val_acc: 0.8600 Epoch 15/15 78/78 - 79s - loss: 0.2084 - acc: 0.9122 - val_loss: 0.3857 - val_acc: 0.8298 2019-07-14 15:12:21,858 graeae.timers.timer end: Ended: 2019-07-14 15:12:21.857979 I0714 15:12:21.858018 140261637994304 timer.py:77] Ended: 2019-07-14 15:12:21.857979 2019-07-14 15:12:21,859 graeae.timers.timer end: Elapsed: 0:20:56.508465 I0714 15:12:21.859612 140261637994304 timer.py:78] Elapsed: 0:20:56.508465
So doubling the batch size doesn't change the time or the accuracy?
Take Eight
path = Path("~/models/dogs-vs-cats/cnn_weights_batch_size_512").expanduser()
saver = partial(save_model_weights, name=str(path))
good_enough = Stop(call_on_stopping=saver,
minimum_accuracy=95)
out_of_time = TimedStop(call_on_stopping=saver)
network = Network(str(training_path),
callbacks=[good_enough, out_of_time],
convolution_layers=5,
batch_size=512)
print(str(network))
with TIMER:
network.train()
2019-07-14 15:21:24,657 graeae.timers.timer start: Started: 2019-07-14 15:21:24.657613 I0714 15:21:24.657636 140261637994304 timer.py:70] Started: 2019-07-14 15:21:24.657613 (Network) - Path: /home/athena/data/datasets/images/dogs-vs-cats/train, Epochs: 15, Batch Size: 512, Callbacks: [<__main__.Stop object at 0x7f8fd8224128>, <__main__.TimedStop object at 0x7f90990c6e80>],Data: (Data) - Path: /home/athena/data/datasets/images/dogs-vs-cats/train, Validation Split: 0.2,Batch Size: 512,Callbacks: [<__main__.Stop object at 0x7f8fd8224128>, <__main__.TimedStop object at 0x7f90990c6e80>] Found 20000 images belonging to 2 classes. Found 5000 images belonging to 2 classes. Epoch 1/15 39/39 - 88s - loss: 0.6929 - acc: 0.5590 - val_loss: 0.6546 - val_acc: 0.6135 Epoch 2/15 39/39 - 86s - loss: 0.6552 - acc: 0.6109 - val_loss: 0.6258 - val_acc: 0.6523 Epoch 3/15 39/39 - 83s - loss: 0.6128 - acc: 0.6612 - val_loss: 0.5904 - val_acc: 0.6918 Epoch 4/15 39/39 - 83s - loss: 0.5830 - acc: 0.6929 - val_loss: 0.5612 - val_acc: 0.7198 Epoch 5/15 39/39 - 80s - loss: 0.5767 - acc: 0.6999 - val_loss: 0.5494 - val_acc: 0.7177 Epoch 6/15 39/39 - 80s - loss: 0.5416 - acc: 0.7262 - val_loss: 0.5036 - val_acc: 0.7591 Epoch 7/15 39/39 - 77s - loss: 0.5199 - acc: 0.7424 - val_loss: 0.4987 - val_acc: 0.7602 Epoch 8/15 39/39 - 76s - loss: 0.5040 - acc: 0.7501 - val_loss: 0.5018 - val_acc: 0.7565 Epoch 9/15 39/39 - 74s - loss: 0.4735 - acc: 0.7743 - val_loss: 0.5063 - val_acc: 0.7517 Epoch 10/15 39/39 - 74s - loss: 0.4635 - acc: 0.7778 - val_loss: 0.4747 - val_acc: 0.7682 Epoch 11/15 39/39 - 72s - loss: 0.4403 - acc: 0.7953 - val_loss: 0.4519 - val_acc: 0.7923 Epoch 12/15 39/39 - 72s - loss: 0.4175 - acc: 0.8014 - val_loss: 0.5312 - val_acc: 0.7385 Epoch 13/15 39/39 - 71s - loss: 0.4099 - acc: 0.8135 - val_loss: 0.4505 - val_acc: 0.7899 Epoch 14/15 39/39 - 71s - loss: 0.3855 - acc: 0.8246 - val_loss: 0.4089 - val_acc: 0.8121 Epoch 15/15 39/39 - 73s - loss: 0.3622 - acc: 0.8373 - val_loss: 0.4049 - val_acc: 0.8166 2019-07-14 15:40:46,189 graeae.timers.timer end: Ended: 2019-07-14 15:40:46.189544 I0714 15:40:46.189586 140261637994304 timer.py:77] Ended: 2019-07-14 15:40:46.189544 2019-07-14 15:40:46,191 graeae.timers.timer end: Elapsed: 0:19:21.531931 I0714 15:40:46.191108 140261637994304 timer.py:78] Elapsed: 0:19:21.531931
So now it looks like after a while the batch-size doesn't make a difference, and we don't get anywhere near the performance I got on that first run.
Take Nine
What if we cut down the number of convolutional layers?
path = Path("~/models/dogs-vs-cats/cnn_weights_batch_size_256_3").expanduser()
saver = partial(save_model_weights, name=str(path))
good_enough = Stop(call_on_stopping=saver,
minimum_accuracy=95)
out_of_time = TimedStop(call_on_stopping=saver)
network = Network(str(training_path),
callbacks=[good_enough, out_of_time],
convolution_layers=3,
batch_size=256)
print(str(network))
with TIMER:
network.train()
2019-07-14 15:48:37,805 graeae.timers.timer start: Started: 2019-07-14 15:48:37.805388 I0714 15:48:37.805425 140261637994304 timer.py:70] Started: 2019-07-14 15:48:37.805388 (Network) - Path: /home/athena/data/datasets/images/dogs-vs-cats/train, Epochs: 15, Batch Size: 256, Callbacks: [<__main__.Stop object at 0x7f8fb40cc668>, <__main__.TimedStop object at 0x7f8fb40ccb38>],Data: (Data) - Path: /home/athena/data/datasets/images/dogs-vs-cats/train, Validation Split: 0.2,Batch Size: 256,Callbacks: [<__main__.Stop object at 0x7f8fb40cc668>, <__main__.TimedStop object at 0x7f8fb40ccb38>] Found 20000 images belonging to 2 classes. Found 5000 images belonging to 2 classes. Epoch 1/15 78/78 - 90s - loss: 0.7905 - acc: 0.5876 - val_loss: 0.6102 - val_acc: 0.6686 Epoch 2/15 78/78 - 88s - loss: 0.6018 - acc: 0.6747 - val_loss: 0.4995 - val_acc: 0.7588 Epoch 3/15 78/78 - 88s - loss: 0.5166 - acc: 0.7430 - val_loss: 0.4653 - val_acc: 0.7815 Epoch 4/15 78/78 - 87s - loss: 0.4609 - acc: 0.7849 - val_loss: 0.4388 - val_acc: 0.7975 Epoch 5/15 78/78 - 86s - loss: 0.4251 - acc: 0.8010 - val_loss: 0.4319 - val_acc: 0.7973 Epoch 6/15 78/78 - 86s - loss: 0.3864 - acc: 0.8254 - val_loss: 0.4892 - val_acc: 0.7574 Epoch 7/15 78/78 - 85s - loss: 0.3516 - acc: 0.8420 - val_loss: 0.4605 - val_acc: 0.7958 Epoch 8/15 78/78 - 84s - loss: 0.3164 - acc: 0.8619 - val_loss: 0.4136 - val_acc: 0.8178 Epoch 9/15 78/78 - 84s - loss: 0.2688 - acc: 0.8877 - val_loss: 0.5011 - val_acc: 0.7611 Epoch 10/15 78/78 - 83s - loss: 0.2265 - acc: 0.9060 - val_loss: 0.5449 - val_acc: 0.7876 Epoch 11/15 78/78 - 82s - loss: 0.2066 - acc: 0.9149 - val_loss: 0.5982 - val_acc: 0.7741 Epoch 12/15 78/78 - 82s - loss: 0.1662 - acc: 0.9397 - val_loss: 0.5129 - val_acc: 0.8244 Epoch 13/15 78/78 - 81s - loss: 0.1327 - acc: 0.9505 - val_loss: 0.6415 - val_acc: 0.8014 Epoch 14/15 78/78 - 82s - loss: 0.1043 - acc: 0.9643 - val_loss: 0.6042 - val_acc: 0.8168 Epoch 15/15 78/78 - 80s - loss: 0.0813 - acc: 0.9746 - val_loss: 0.6658 - val_acc: 0.8201 2019-07-14 16:09:47,278 graeae.timers.timer end: Ended: 2019-07-14 16:09:47.278102 I0714 16:09:47.278141 140261637994304 timer.py:77] Ended: 2019-07-14 16:09:47.278102 2019-07-14 16:09:47,279 graeae.timers.timer end: Elapsed: 0:21:09.472714 I0714 16:09:47.279699 140261637994304 timer.py:78] Elapsed: 0:21:09.472714
Nope, it's almost as if I didn't do anything to change it.
Take Ten
What if I get rid of the steps per epoch?
path = Path("~/models/dogs-vs-cats/cnn_weights_batch_size_20_3").expanduser()
saver = partial(save_model_weights, name=str(path))
good_enough = Stop(call_on_stopping=saver,
minimum_accuracy=95)
out_of_time = TimedStop(call_on_stopping=saver)
network = Network(str(training_path),
callbacks=[good_enough, out_of_time],
convolution_layers=3,
set_steps = True,
batch_size=20)
print(str(network))
with TIMER:
network.train()
2019-07-14 19:08:19,353 graeae.timers.timer start: Started: 2019-07-14 19:08:19.353471 I0714 19:08:19.353494 140261637994304 timer.py:70] Started: 2019-07-14 19:08:19.353471 (Network) - Path: /home/athena/data/datasets/images/dogs-vs-cats/train, Epochs: 15, Batch Size: 20, Callbacks: [<__main__.Stop object at 0x7f8dfa4dd048>, <__main__.TimedStop object at 0x7f8dfa4dd470>],Data: (Data) - Path: /home/athena/data/datasets/images/dogs-vs-cats/train, Validation Split: 0.2,Batch Size: 20,Callbacks: [<__main__.Stop object at 0x7f8dfa4dd048>, <__main__.TimedStop object at 0x7f8dfa4dd470>] Found 20000 images belonging to 2 classes. Found 5000 images belonging to 2 classes. Epoch 1/15 1000/1000 - 155s - loss: 0.6403 - acc: 0.6418 - val_loss: 0.5868 - val_acc: 0.6744 Epoch 2/15 1000/1000 - 153s - loss: 0.5746 - acc: 0.7024 - val_loss: 0.5579 - val_acc: 0.7180 Epoch 3/15 1000/1000 - 152s - loss: 0.5397 - acc: 0.7315 - val_loss: 0.5599 - val_acc: 0.7248 Epoch 4/15 1000/1000 - 152s - loss: 0.5237 - acc: 0.7452 - val_loss: 0.4945 - val_acc: 0.7576 Epoch 5/15 1000/1000 - 152s - loss: 0.5110 - acc: 0.7520 - val_loss: 0.4821 - val_acc: 0.7710 Epoch 6/15 1000/1000 - 152s - loss: 0.5012 - acc: 0.7627 - val_loss: 0.7423 - val_acc: 0.7008 Epoch 7/15 1000/1000 - 152s - loss: 0.4913 - acc: 0.7668 - val_loss: 0.4916 - val_acc: 0.7732 Epoch 8/15 1000/1000 - 152s - loss: 0.4834 - acc: 0.7726 - val_loss: 0.5013 - val_acc: 0.7634 Epoch 9/15 1000/1000 - 152s - loss: 0.4765 - acc: 0.7810 - val_loss: 0.4503 - val_acc: 0.7930 Epoch 10/15 1000/1000 - 152s - loss: 0.4712 - acc: 0.7858 - val_loss: 0.4780 - val_acc: 0.7872 Epoch 11/15 1000/1000 - 152s - loss: 0.4663 - acc: 0.7882 - val_loss: 0.4645 - val_acc: 0.7810 Epoch 12/15 1000/1000 - 152s - loss: 0.4590 - acc: 0.7926 - val_loss: 0.4233 - val_acc: 0.8208 Epoch 13/15 1000/1000 - 152s - loss: 0.4566 - acc: 0.7958 - val_loss: 0.4494 - val_acc: 0.8020 Epoch 14/15 1000/1000 - 154s - loss: 0.4499 - acc: 0.7985 - val_loss: 0.4125 - val_acc: 0.8248 Epoch 15/15 1000/1000 - 157s - loss: 0.4453 - acc: 0.8014 - val_loss: 0.4875 - val_acc: 0.7510 2019-07-14 19:46:32,393 graeae.timers.timer end: Ended: 2019-07-14 19:46:32.393181 I0714 19:46:32.393228 140261637994304 timer.py:77] Ended: 2019-07-14 19:46:32.393181 2019-07-14 19:46:32,394 graeae.timers.timer end: Elapsed: 0:38:13.039710 I0714 19:46:32.394544 140261637994304 timer.py:78] Elapsed: 0:38:13.039710
Take Eleven
I updated the model to have twice as many neurons at each layes than it did before and added fill_mode="nearest" to the image generator. Let's see how it does.
path = Path("~/models/dogs-vs-cats/cnn_weights_batch_size_32_5_double").expanduser()
saver = partial(save_model_weights, name=str(path))
good_enough = Stop(call_on_stopping=saver,
minimum_accuracy=.95)
out_of_time = TimedStop(call_on_stopping=saver)
network = Network(str(training_path),
callbacks=[good_enough, out_of_time],
convolution_layers=5,
set_steps = True,
epochs = 100,
batch_size=32)
print(str(network))
with TIMER:
network.train()
2019-07-15 22:39:43,846 graeae.timers.timer start: Started: 2019-07-15 22:39:43.846462 I0715 22:39:43.846729 140093074827072 timer.py:70] Started: 2019-07-15 22:39:43.846462 (Network) - Path: /home/athena/data/datasets/images/dogs-vs-cats/train, Epochs: 100, Batch Size: 32, Callbacks: [<__main__.Stop object at 0x7f6964315f98>, <__main__.TimedStop object at 0x7f6964315f60>],Data: (Data) - Path: /home/athena/data/datasets/images/dogs-vs-cats/train, Validation Split: 0.2,Batch Size: 32,Callbacks: [<__main__.Stop object at 0x7f6964315f98>, <__main__.TimedStop object at 0x7f6964315f60>] Found 20000 images belonging to 2 classes. W0715 22:39:44.526424 140093074827072 deprecation.py:506] From /home/athena/.virtualenvs/In-Too-Deep/lib/python3.7/site-packages/tensorflow/python/ops/init_ops.py:1251: calling VarianceScaling.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version. Instructions for updating: Call initializer instance with the dtype argument instead of passing it to the constructor Found 5000 images belonging to 2 classes. W0715 22:39:44.941590 140093074827072 deprecation.py:323] From /home/athena/.virtualenvs/In-Too-Deep/lib/python3.7/site-packages/tensorflow/python/ops/nn_impl.py:180: add_dispatch_support.<locals>.wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version. Instructions for updating: Use tf.where in 2.0, which has the same broadcast rule as np.where Epoch 1/100 625/625 - 443s - loss: 0.6784 - acc: 0.5767 - val_loss: 0.6511 - val_acc: 0.6290 Epoch 2/100 625/625 - 153s - loss: 0.6261 - acc: 0.6617 - val_loss: 0.6095 - val_acc: 0.6833 Epoch 3/100 625/625 - 151s - loss: 0.5860 - acc: 0.6988 - val_loss: 0.5633 - val_acc: 0.7256 Epoch 4/100 625/625 - 151s - loss: 0.5576 - acc: 0.7165 - val_loss: 0.5596 - val_acc: 0.7079 Epoch 5/100 625/625 - 152s - loss: 0.5230 - acc: 0.7480 - val_loss: 0.4567 - val_acc: 0.7843 Epoch 6/100 625/625 - 151s - loss: 0.4899 - acc: 0.7704 - val_loss: 0.4682 - val_acc: 0.7778 Epoch 7/100 625/625 - 155s - loss: 0.4654 - acc: 0.7873 - val_loss: 0.4377 - val_acc: 0.8097 Epoch 8/100 625/625 - 152s - loss: 0.4426 - acc: 0.8019 - val_loss: 0.4038 - val_acc: 0.8249 Epoch 9/100 625/625 - 153s - loss: 0.4338 - acc: 0.8034 - val_loss: 0.3988 - val_acc: 0.8223 Epoch 10/100 625/625 - 152s - loss: 0.4278 - acc: 0.8113 - val_loss: 0.4619 - val_acc: 0.7979 Epoch 11/100 625/625 - 153s - loss: 0.4224 - acc: 0.8101 - val_loss: 0.4608 - val_acc: 0.7710 Epoch 12/100 625/625 - 153s - loss: 0.4223 - acc: 0.8140 - val_loss: 0.4496 - val_acc: 0.7726 Epoch 13/100 625/625 - 153s - loss: 0.4285 - acc: 0.8093 - val_loss: 0.4147 - val_acc: 0.8255 Epoch 14/100 625/625 - 156s - loss: 0.4224 - acc: 0.8070 - val_loss: 0.4451 - val_acc: 0.7885 Epoch 15/100 625/625 - 155s - loss: 0.4428 - acc: 0.8092 - val_loss: 0.4191 - val_acc: 0.8177 Epoch 16/100 625/625 - 156s - loss: 0.4267 - acc: 0.8089 - val_loss: 0.3791 - val_acc: 0.8127 Epoch 17/100 625/625 - 155s - loss: 0.4328 - acc: 0.8067 - val_loss: 0.6180 - val_acc: 0.7839 Epoch 18/100 625/625 - 156s - loss: 0.4361 - acc: 0.8083 - val_loss: 0.6032 - val_acc: 0.6785 Epoch 19/100 625/625 - 156s - loss: 0.4203 - acc: 0.8153 - val_loss: 0.3815 - val_acc: 0.8335 Epoch 20/100 625/625 - 156s - loss: 0.4403 - acc: 0.8116 - val_loss: 0.3653 - val_acc: 0.8393 Epoch 21/100 625/625 - 155s - loss: 0.4216 - acc: 0.8105 - val_loss: 0.4811 - val_acc: 0.7728 Epoch 22/100 625/625 - 155s - loss: 0.4423 - acc: 0.8049 - val_loss: 2.2071 - val_acc: 0.5913 Epoch 23/100 625/625 - 152s - loss: 0.4417 - acc: 0.8063 - val_loss: 0.3568 - val_acc: 0.8442 Epoch 24/100 625/625 - 151s - loss: 0.4353 - acc: 0.8105 - val_loss: 0.3727 - val_acc: 0.8405 Epoch 25/100 625/625 - 150s - loss: 0.4409 - acc: 0.8099 - val_loss: 0.4661 - val_acc: 0.7604 Epoch 26/100 625/625 - 151s - loss: 0.4467 - acc: 0.8021 - val_loss: 0.4138 - val_acc: 0.8227 Epoch 27/100 625/625 - 151s - loss: 0.4465 - acc: 0.8078 - val_loss: 0.3894 - val_acc: 0.8407 Epoch 28/100 625/625 - 156s - loss: 0.4269 - acc: 0.8126 - val_loss: 0.4504 - val_acc: 0.7901 Epoch 29/100 625/625 - 156s - loss: 0.4354 - acc: 0.8102 - val_loss: 0.3996 - val_acc: 0.8546 Epoch 30/100 625/625 - 156s - loss: 0.4677 - acc: 0.8122 - val_loss: 0.3758 - val_acc: 0.8508 Epoch 31/100 625/625 - 155s - loss: 0.4282 - acc: 0.8164 - val_loss: 0.4889 - val_acc: 0.7875 Epoch 32/100 625/625 - 156s - loss: 0.4306 - acc: 0.8108 - val_loss: 0.4583 - val_acc: 0.7562 Epoch 33/100 625/625 - 155s - loss: 0.4280 - acc: 0.8123 - val_loss: 0.4034 - val_acc: 0.7897 Epoch 34/100 625/625 - 157s - loss: 0.4316 - acc: 0.8058 - val_loss: 0.3685 - val_acc: 0.8462 Epoch 35/100 625/625 - 155s - loss: 0.4248 - acc: 0.8133 - val_loss: 0.5994 - val_acc: 0.8433 Epoch 36/100 625/625 - 156s - loss: 0.4450 - acc: 0.8112 - val_loss: 0.5043 - val_acc: 0.7099 Epoch 37/100 625/625 - 156s - loss: 0.4467 - acc: 0.8085 - val_loss: 0.3921 - val_acc: 0.8297 Epoch 38/100 625/625 - 156s - loss: 0.4377 - acc: 0.8106 - val_loss: 0.3842 - val_acc: 0.8403 Epoch 39/100 625/625 - 156s - loss: 0.4586 - acc: 0.8156 - val_loss: 0.6008 - val_acc: 0.6717 Epoch 40/100 625/625 - 156s - loss: 0.4234 - acc: 0.8180 - val_loss: 0.4325 - val_acc: 0.7943 Epoch 41/100 625/625 - 156s - loss: 0.4395 - acc: 0.8097 - val_loss: 0.3536 - val_acc: 0.8498 Epoch 42/100 625/625 - 156s - loss: 0.4453 - acc: 0.8062 - val_loss: 0.4309 - val_acc: 0.7971 Epoch 43/100 625/625 - 155s - loss: 0.4425 - acc: 0.8134 - val_loss: 0.3935 - val_acc: 0.8091 Epoch 44/100 625/625 - 156s - loss: 0.4301 - acc: 0.8153 - val_loss: 0.4786 - val_acc: 0.8207 Epoch 45/100 625/625 - 155s - loss: 0.4605 - acc: 0.8120 - val_loss: 0.3412 - val_acc: 0.8546 Epoch 46/100 625/625 - 156s - loss: 0.4261 - acc: 0.8221 - val_loss: 0.8955 - val_acc: 0.7194 Epoch 47/100 625/625 - 156s - loss: 0.4215 - acc: 0.8214 - val_loss: 0.4956 - val_acc: 0.8155 Epoch 48/100 625/625 - 156s - loss: 0.4270 - acc: 0.8144 - val_loss: 0.3912 - val_acc: 0.8331 Epoch 49/100 625/625 - 156s - loss: 0.4268 - acc: 0.8167 - val_loss: 0.3492 - val_acc: 0.8474 Epoch 50/100 625/625 - 155s - loss: 0.4275 - acc: 0.8135 - val_loss: 0.4503 - val_acc: 0.8383 Epoch 51/100 625/625 - 156s - loss: 0.4980 - acc: 0.8142 - val_loss: 0.4140 - val_acc: 0.8147 Epoch 52/100 625/625 - 155s - loss: 0.4528 - acc: 0.8221 - val_loss: 0.3570 - val_acc: 0.8456 Epoch 53/100 625/625 - 156s - loss: 0.4201 - acc: 0.8212 - val_loss: 0.5224 - val_acc: 0.6943 Epoch 54/100 625/625 - 153s - loss: 0.4399 - acc: 0.8113 - val_loss: 0.4587 - val_acc: 0.7782 Epoch 55/100 625/625 - 151s - loss: 0.4306 - acc: 0.8201 - val_loss: 0.3410 - val_acc: 0.8612 Epoch 56/100 625/625 - 150s - loss: 0.4986 - acc: 0.8180 - val_loss: 0.3842 - val_acc: 0.8345 Epoch 57/100 625/625 - 150s - loss: 0.4276 - acc: 0.8191 - val_loss: 0.4859 - val_acc: 0.7352 Epoch 58/100 625/625 - 150s - loss: 0.4344 - acc: 0.8084 - val_loss: 0.5267 - val_acc: 0.7242 Epoch 59/100 625/625 - 150s - loss: 0.4488 - acc: 0.8106 - val_loss: 0.3798 - val_acc: 0.8311 Epoch 60/100 625/625 - 150s - loss: 0.4445 - acc: 0.8111 - val_loss: 0.3818 - val_acc: 0.8367 Epoch 61/100 625/625 - 150s - loss: 0.5270 - acc: 0.8143 - val_loss: 0.3891 - val_acc: 0.8171 Epoch 62/100 625/625 - 150s - loss: 0.4444 - acc: 0.8131 - val_loss: 0.4622 - val_acc: 0.8061 Epoch 63/100 625/625 - 150s - loss: 0.4488 - acc: 0.8164 - val_loss: 0.4466 - val_acc: 0.7983 Epoch 64/100 625/625 - 150s - loss: 0.5321 - acc: 0.7947 - val_loss: 0.3542 - val_acc: 0.8546 Epoch 65/100 625/625 - 150s - loss: 0.4685 - acc: 0.7996 - val_loss: 0.3638 - val_acc: 0.8502 Epoch 66/100 625/625 - 151s - loss: 0.5376 - acc: 0.8097 - val_loss: 0.3572 - val_acc: 0.8464 Epoch 67/100 TimedStop: Training out of time (Elapsed = 2:56:51.937316) Saving the Model Weights to /home/athena/models/dogs-vs-cats/cnn_weights_batch_size_32_5_double.h5 625/625 - 150s - loss: 0.4454 - acc: 0.8030 - val_loss: 0.3283 - val_acc: 0.8650 2019-07-16 01:36:37,054 graeae.timers.timer end: Ended: 2019-07-16 01:36:37.054383 I0716 01:36:37.054409 140093074827072 timer.py:77] Ended: 2019-07-16 01:36:37.054383 2019-07-16 01:36:37,055 graeae.timers.timer end: Elapsed: 2:56:53.207921 I0716 01:36:37.055413 140093074827072 timer.py:78] Elapsed: 2:56:53.207921
So we didn't quite get there. Maybe more training on the same model? Is it really getting better? It did the best at epoch 55. Maybe 86 is as good as it gets, but might as well try.
path = Path("~/models/dogs-vs-cats/cnn_weights_batch_size_32_5_double").expanduser()
saver = partial(save_model_weights, name=str(path))
good_enough = Stop(call_on_stopping=saver,
minimum_accuracy=.95)
out_of_time = TimedStop(call_on_stopping=saver)
with TIMER:
network.train()
Take Twelve
path = Path("~/models/dogs-vs-cats/cnn_weights_batch_size_32_5_double").expanduser()
saver = partial(save_model, path=path)
good_enough = Stop(call_on_stopping=saver,
minimum_accuracy=95)
out_of_time = TimedStop(call_on_stopping=saver)
network = Network(str(training_path),
callbacks=[good_enough, out_of_time],
convolution_layers=5,
set_steps = True,
epochs = 100,
batch_size=32)
print(str(network))
with TIMER:
network.train()
2019-07-17 23:02:00,693 graeae.timers.timer start: Started: 2019-07-17 23:02:00.692917 I0717 23:02:00.693119 140065468053312 timer.py:70] Started: 2019-07-17 23:02:00.692917 (Network) - Path: /home/athena/data/datasets/images/dogs-vs-cats/train Epochs: 100 Batch Size: 32 Callbacks: [<__main__.Stop object at 0x7f62f68a6c18>, <__main__.TimedStop object at 0x7f62f68a6da0>] Data: (Data) - Path: /home/athena/data/datasets/images/dogs-vs-cats/train, Validation Split: 0.2,Batch Size: 32 Callbacks: [<__main__.Stop object at 0x7f62f68a6c18>, <__main__.TimedStop object at 0x7f62f68a6da0>] Found 20000 images belonging to 2 classes. W0717 23:02:01.406333 140065468053312 deprecation.py:506] From /home/athena/.virtualenvs/In-Too-Deep/lib/python3.7/site-packages/tensorflow/python/ops/init_ops.py:1251: calling VarianceScaling.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version. Instructions for updating: Call initializer instance with the dtype argument instead of passing it to the constructor Found 5000 images belonging to 2 classes. W0717 23:02:01.884152 140065468053312 deprecation.py:323] From /home/athena/.virtualenvs/In-Too-Deep/lib/python3.7/site-packages/tensorflow/python/ops/nn_impl.py:180: add_dispatch_support.<locals>.wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version. Instructions for updating: Use tf.where in 2.0, which has the same broadcast rule as np.where Epoch 1/100 625/625 - 533s - loss: 0.6770 - acc: 0.5823 - val_loss: 0.6773 - val_acc: 0.5877 Epoch 2/100 625/625 - 157s - loss: 0.6280 - acc: 0.6556 - val_loss: 0.5860 - val_acc: 0.6879 Epoch 3/100 625/625 - 156s - loss: 0.5955 - acc: 0.6870 - val_loss: 0.5364 - val_acc: 0.7322 Epoch 4/100 625/625 - 157s - loss: 0.5579 - acc: 0.7225 - val_loss: 0.5200 - val_acc: 0.7648 Epoch 5/100 625/625 - 157s - loss: 0.5208 - acc: 0.7455 - val_loss: 0.5350 - val_acc: 0.7410 Epoch 6/100 625/625 - 157s - loss: 0.4874 - acc: 0.7736 - val_loss: 0.4713 - val_acc: 0.7740 Epoch 7/100 625/625 - 157s - loss: 0.4630 - acc: 0.7847 - val_loss: 0.5257 - val_acc: 0.7530 Epoch 8/100 625/625 - 157s - loss: 0.4499 - acc: 0.7923 - val_loss: 0.5602 - val_acc: 0.7294 Epoch 9/100 625/625 - 157s - loss: 0.4299 - acc: 0.8048 - val_loss: 0.5801 - val_acc: 0.6901 Epoch 10/100 625/625 - 157s - loss: 0.4296 - acc: 0.8101 - val_loss: 0.5544 - val_acc: 0.7642 Epoch 11/100 625/625 - 156s - loss: 0.4328 - acc: 0.8066 - val_loss: 0.5598 - val_acc: 0.7354 Epoch 12/100 625/625 - 157s - loss: 0.4372 - acc: 0.8080 - val_loss: 0.3635 - val_acc: 0.8411 Epoch 13/100 625/625 - 156s - loss: 0.4376 - acc: 0.8092 - val_loss: 0.4418 - val_acc: 0.8003 Epoch 14/100 625/625 - 156s - loss: 0.4228 - acc: 0.8147 - val_loss: 0.3530 - val_acc: 0.8454 Epoch 15/100 625/625 - 156s - loss: 0.4181 - acc: 0.8167 - val_loss: 0.4420 - val_acc: 0.7879 Epoch 16/100 625/625 - 156s - loss: 0.4295 - acc: 0.8105 - val_loss: 0.3515 - val_acc: 0.8411 Epoch 17/100 625/625 - 156s - loss: 0.4400 - acc: 0.8087 - val_loss: 0.3885 - val_acc: 0.8323 Epoch 18/100 625/625 - 156s - loss: 0.4215 - acc: 0.8145 - val_loss: 0.3775 - val_acc: 0.8417 Epoch 19/100 625/625 - 156s - loss: 0.4323 - acc: 0.8159 - val_loss: 0.4359 - val_acc: 0.8065 Epoch 20/100 625/625 - 156s - loss: 0.4181 - acc: 0.8089 - val_loss: 0.5274 - val_acc: 0.8474 Epoch 21/100 625/625 - 156s - loss: 0.4392 - acc: 0.8116 - val_loss: 0.5858 - val_acc: 0.7163 Epoch 22/100 625/625 - 156s - loss: 0.4432 - acc: 0.8093 - val_loss: 0.5178 - val_acc: 0.7192 Epoch 23/100 625/625 - 156s - loss: 0.4629 - acc: 0.8091 - val_loss: 0.3340 - val_acc: 0.8604 Epoch 24/100 625/625 - 156s - loss: 0.4236 - acc: 0.8150 - val_loss: 0.3742 - val_acc: 0.8357 Epoch 25/100 625/625 - 156s - loss: 0.4257 - acc: 0.8152 - val_loss: 0.3582 - val_acc: 0.8367 Epoch 26/100 625/625 - 157s - loss: 0.4133 - acc: 0.8190 - val_loss: 0.3847 - val_acc: 0.8339 Epoch 27/100 625/625 - 156s - loss: 0.4298 - acc: 0.8136 - val_loss: 0.3627 - val_acc: 0.8446 Epoch 28/100 625/625 - 155s - loss: 0.4335 - acc: 0.8127 - val_loss: 0.4581 - val_acc: 0.8261 Epoch 29/100 625/625 - 156s - loss: 0.4234 - acc: 0.8123 - val_loss: 0.3819 - val_acc: 0.8333 Epoch 30/100 625/625 - 156s - loss: 0.4213 - acc: 0.8152 - val_loss: 0.3809 - val_acc: 0.8596 Epoch 31/100 625/625 - 156s - loss: 0.4399 - acc: 0.8029 - val_loss: 0.4879 - val_acc: 0.7662 Epoch 32/100 625/625 - 157s - loss: 0.4406 - acc: 0.8090 - val_loss: 0.3731 - val_acc: 0.8668 Epoch 33/100 625/625 - 156s - loss: 0.4336 - acc: 0.8162 - val_loss: 0.4306 - val_acc: 0.8389 Epoch 34/100 625/625 - 153s - loss: 0.4524 - acc: 0.8102 - val_loss: 0.3486 - val_acc: 0.8538 Epoch 35/100 625/625 - 151s - loss: 0.4306 - acc: 0.8127 - val_loss: 0.4819 - val_acc: 0.7923 Epoch 36/100 625/625 - 151s - loss: 0.4350 - acc: 0.8109 - val_loss: 0.3536 - val_acc: 0.8506 Epoch 37/100 625/625 - 151s - loss: 0.4480 - acc: 0.8073 - val_loss: 0.5374 - val_acc: 0.8165 Epoch 38/100 625/625 - 151s - loss: 0.4411 - acc: 0.8195 - val_loss: 1.1689 - val_acc: 0.6763 Epoch 39/100 625/625 - 151s - loss: 0.4652 - acc: 0.8189 - val_loss: 0.3614 - val_acc: 0.8395 Epoch 40/100 625/625 - 151s - loss: 0.4442 - acc: 0.8166 - val_loss: 0.3833 - val_acc: 0.8261 Epoch 41/100 625/625 - 151s - loss: 0.4259 - acc: 0.8128 - val_loss: 0.3824 - val_acc: 0.8349 Epoch 42/100 625/625 - 151s - loss: 0.4336 - acc: 0.8122 - val_loss: 0.3564 - val_acc: 0.8508 Epoch 43/100 625/625 - 151s - loss: 0.4268 - acc: 0.8151 - val_loss: 0.3299 - val_acc: 0.8636 Epoch 44/100 625/625 - 151s - loss: 0.4206 - acc: 0.8184 - val_loss: 0.3538 - val_acc: 0.8462 Epoch 45/100 625/625 - 151s - loss: 0.4318 - acc: 0.8173 - val_loss: 0.3854 - val_acc: 0.8482 Epoch 46/100 625/625 - 151s - loss: 0.4521 - acc: 0.8120 - val_loss: 0.3691 - val_acc: 0.8568 Epoch 47/100 625/625 - 150s - loss: 0.4561 - acc: 0.8090 - val_loss: 0.3810 - val_acc: 0.8474 Epoch 48/100 625/625 - 151s - loss: 0.4358 - acc: 0.8128 - val_loss: 0.4755 - val_acc: 0.8041 Epoch 49/100 625/625 - 151s - loss: 0.4296 - acc: 0.8132 - val_loss: 0.4490 - val_acc: 0.8618 Epoch 50/100 625/625 - 151s - loss: 0.4385 - acc: 0.8183 - val_loss: 0.3964 - val_acc: 0.8377 Epoch 51/100 625/625 - 150s - loss: 0.4368 - acc: 0.8164 - val_loss: 0.3463 - val_acc: 0.8592 Epoch 52/100 625/625 - 151s - loss: 0.4702 - acc: 0.8134 - val_loss: 0.3882 - val_acc: 0.8365 Epoch 53/100 625/625 - 151s - loss: 0.4555 - acc: 0.8025 - val_loss: 0.4516 - val_acc: 0.7558 Epoch 54/100 625/625 - 150s - loss: 0.4502 - acc: 0.8059 - val_loss: 0.4236 - val_acc: 0.8035 Epoch 55/100 625/625 - 151s - loss: 0.4829 - acc: 0.7970 - val_loss: 0.3800 - val_acc: 0.8349 Epoch 56/100 625/625 - 150s - loss: 0.4554 - acc: 0.8055 - val_loss: 0.4774 - val_acc: 0.8023 Epoch 57/100 625/625 - 151s - loss: 0.5059 - acc: 0.7813 - val_loss: 0.5145 - val_acc: 0.7332 Epoch 58/100 625/625 - 151s - loss: 0.4680 - acc: 0.8015 - val_loss: 0.3994 - val_acc: 0.8201 Epoch 59/100 625/625 - 151s - loss: 0.4938 - acc: 0.7962 - val_loss: 0.3682 - val_acc: 0.8421 Epoch 60/100 625/625 - 151s - loss: 0.4870 - acc: 0.7883 - val_loss: 0.3798 - val_acc: 0.8249 Epoch 61/100 625/625 - 151s - loss: 0.4785 - acc: 0.7902 - val_loss: 0.4609 - val_acc: 0.7967 Epoch 62/100 625/625 - 151s - loss: 0.4948 - acc: 0.7840 - val_loss: 0.4502 - val_acc: 0.7788 Epoch 63/100 625/625 - 150s - loss: 0.5021 - acc: 0.7847 - val_loss: 0.3829 - val_acc: 0.8307 Epoch 64/100 625/625 - 151s - loss: 0.4982 - acc: 0.7882 - val_loss: 0.6872 - val_acc: 0.6755 Epoch 65/100 625/625 - 151s - loss: 0.5514 - acc: 0.7707 - val_loss: 0.4682 - val_acc: 0.7804 Epoch 66/100 TimedStop: Training out of time (Elapsed = 2:55:40.163015) 625/625 - 151s - loss: 0.5037 - acc: 0.7713 - val_loss: 0.5198 - val_acc: 0.8091 2019-07-18 01:57:42,823 graeae.timers.timer end: Ended: 2019-07-18 01:57:42.823388 I0718 01:57:42.823415 140065468053312 timer.py:77] Ended: 2019-07-18 01:57:42.823388 2019-07-18 01:57:42,824 graeae.timers.timer end: Elapsed: 2:55:42.130471 I0718 01:57:42.824334 140065468053312 timer.py:78] Elapsed: 2:55:42.130471
best_accuracy = data.validation_accuracy.max()
best_loss = data.validation_loss.min()
accuracy_slice = data[data.validation_accuracy==best_accuracy]
loss_slice = data[data.validation_loss==best_loss]
accuracy_index = accuracy_slice.index[0]
loss_index = loss_slice.index[0]
print(f"Best Accuracy: {best_accuracy} "
f"(loss={accuracy_slice.validation_loss.iloc[0]})"
f" Epoch: {accuracy_index + 1}")
print(f"Best Loss: {best_loss} (accuracy="
f"{loss_slice.validation_accuracy.iloc[0]})"
f" Epoch: {loss_index + 1}")
Best Accuracy: 0.8668 (loss=0.3731) Epoch: 32 Best Loss: 0.3299 (accuracy=0.8636) Epoch: 43
data = pandas.read_csv("~/cats_vs_dogs.csv")
line_1 = holoviews.VLine(accuracy_index, label="Best Accuracy")
line_2 = holoviews.VLine(loss_index, label="Best Loss")
curves = [holoviews.Curve(data, ("index", "Epoch"), "training_loss",
label="Training Loss",),
holoviews.Curve(data, ("index", "Epoch"), "training_accuracy",
label="Training Accuracy").opts(tools=["hover"]),
holoviews.Curve(data, ("index", "Epoch"), "validation_loss",
label="Validation Loss",).opts(tools=["hover"]),
holoviews.Curve(data, ("index", "Epoch"), "validation_accuracy",
label="Validation Accuracy").opts(tools=["hover"]),
line_1, line_2]
plot = holoviews.Overlay(curves).opts(tools=["hover"], height=800, width=1000,
ylabel="Performance",
title="Training vs Validation")
Embed(plot=plot, file_name="training_validation_loss_12")()
There's more variance in the validation performance than I thought there would be. Although the best accuracy and loss come later, Epoch 22 (zero-based) has better loss (0.334) than the best accuracy's loss (0.37), and around the same accuracy (0.86) as the best loss' accuracy (0.864)
This next part won't work yet because I don't have the test data loaded and I need to save the predictions of the stored model.
predictions = model.predict(x_test)
loaded_model = kears.models.load_model(path)
new_predictions = loaded_model.predict(x_test)
numpy.assert.allclose(predictions, new_predictions, atol=1e-6)
Take Thirteen
data = pandas.read_csv("~/cats_vs_dogs.csv")
print(tabulate(data[data.validation_accuracy>=0.86],
headers="keys", tablefmt="orgtbl"))
| training_loss | training_accuracy | validation_loss | validation_accuracy | |
|---|---|---|---|---|
| 22 | 0.4629 | 0.8091 | 0.334 | 0.8604 |
| 31 | 0.4406 | 0.809 | 0.3731 | 0.8668 |
| 42 | 0.4268 | 0.8151 | 0.3299 | 0.8636 |
| 48 | 0.4296 | 0.8132 | 0.449 | 0.8618 |
path = Path("~/models/dogs-vs-cats/cnn_weights_batch_size_32_5_double").expanduser()
saver = partial(save_model, path=path)
good_enough = Stop(call_on_stopping=saver,
minimum_accuracy=0.86)
out_of_time = TimedStop(call_on_stopping=saver)
network = Network(str(training_path),
callbacks=[good_enough, out_of_time],
convolution_layers=5,
set_steps = True,
epochs = 100,
batch_size=64)
print(str(network))
with TIMER:
network.train()
2019-07-18 22:28:08,881 graeae.timers.timer start: Started: 2019-07-18 22:28:08.881418 I0718 22:28:08.881625 139703649425216 timer.py:70] Started: 2019-07-18 22:28:08.881418 (Network) - Path: /home/athena/data/datasets/images/dogs-vs-cats/train Epochs: 100 Batch Size: 64 Callbacks: [<__main__.Stop object at 0x7f0eb5d9f860>, <__main__.TimedStop object at 0x7f0eb5d9f828>] Data: (Data) - Path: /home/athena/data/datasets/images/dogs-vs-cats/train, Validation Split: 0.2,Batch Size: 64 Callbacks: [<__main__.Stop object at 0x7f0eb5d9f860>, <__main__.TimedStop object at 0x7f0eb5d9f828>] Found 20000 images belonging to 2 classes. Found 5000 images belonging to 2 classes. W0718 22:28:09.550659 139703649425216 deprecation.py:506] From /home/athena/.virtualenvs/In-Too-Deep/lib/python3.7/site-packages/tensorflow/python/ops/init_ops.py:1251: calling VarianceScaling.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version. Instructions for updating: Call initializer instance with the dtype argument instead of passing it to the constructor W0718 22:28:10.023711 139703649425216 deprecation.py:323] From /home/athena/.virtualenvs/In-Too-Deep/lib/python3.7/site-packages/tensorflow/python/ops/nn_impl.py:180: add_dispatch_support.<locals>.wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version. Instructions for updating: Use tf.where in 2.0, which has the same broadcast rule as np.where Epoch 1/100 312/312 - 523s - loss: 0.6780 - acc: 0.5673 - val_loss: 0.6886 - val_acc: 0.5978 Epoch 2/100 312/312 - 158s - loss: 0.6415 - acc: 0.6353 - val_loss: 0.5944 - val_acc: 0.6789 Epoch 3/100 312/312 - 154s - loss: 0.6050 - acc: 0.6724 - val_loss: 0.5635 - val_acc: 0.7155 Epoch 4/100 312/312 - 153s - loss: 0.5789 - acc: 0.7044 - val_loss: 0.6398 - val_acc: 0.6212 Epoch 5/100 312/312 - 152s - loss: 0.5501 - acc: 0.7231 - val_loss: 0.5117 - val_acc: 0.7480 Epoch 6/100 312/312 - 153s - loss: 0.5155 - acc: 0.7478 - val_loss: 0.4671 - val_acc: 0.7788 Epoch 7/100 312/312 - 157s - loss: 0.4841 - acc: 0.7725 - val_loss: 0.4228 - val_acc: 0.8041 Epoch 8/100 312/312 - 157s - loss: 0.4513 - acc: 0.7875 - val_loss: 0.4660 - val_acc: 0.7877 Epoch 9/100 312/312 - 154s - loss: 0.4275 - acc: 0.8033 - val_loss: 0.3893 - val_acc: 0.8225 Epoch 10/100 312/312 - 157s - loss: 0.4140 - acc: 0.8084 - val_loss: 0.3792 - val_acc: 0.8249 Epoch 11/100 312/312 - 155s - loss: 0.3907 - acc: 0.8258 - val_loss: 0.3678 - val_acc: 0.8299 Epoch 12/100 312/312 - 156s - loss: 0.3772 - acc: 0.8301 - val_loss: 0.4905 - val_acc: 0.7558 Epoch 13/100 312/312 - 155s - loss: 0.3702 - acc: 0.8354 - val_loss: 0.5838 - val_acc: 0.7200 Epoch 14/100 312/312 - 155s - loss: 0.3537 - acc: 0.8412 - val_loss: 0.4032 - val_acc: 0.8225 Epoch 15/100 312/312 - 154s - loss: 0.3501 - acc: 0.8448 - val_loss: 0.3675 - val_acc: 0.8530 Epoch 16/100 312/312 - 150s - loss: 0.3480 - acc: 0.8451 - val_loss: 0.2927 - val_acc: 0.8770 Epoch 17/100 312/312 - 152s - loss: 0.3381 - acc: 0.8518 - val_loss: 0.2863 - val_acc: 0.8746 Epoch 18/100 312/312 - 152s - loss: 0.3386 - acc: 0.8491 - val_loss: 0.3782 - val_acc: 0.8419 Epoch 19/100 312/312 - 153s - loss: 0.3389 - acc: 0.8527 - val_loss: 0.2987 - val_acc: 0.8682 Epoch 20/100 312/312 - 152s - loss: 0.3294 - acc: 0.8589 - val_loss: 0.3645 - val_acc: 0.8456 Epoch 21/100 312/312 - 154s - loss: 0.3239 - acc: 0.8584 - val_loss: 1.4912 - val_acc: 0.6442 Epoch 22/100 312/312 - 153s - loss: 0.3208 - acc: 0.8629 - val_loss: 0.3812 - val_acc: 0.8243 Epoch 23/100 312/312 - 154s - loss: 0.3253 - acc: 0.8642 - val_loss: 0.2765 - val_acc: 0.8918 Epoch 24/100 312/312 - 154s - loss: 0.3252 - acc: 0.8615 - val_loss: 0.2959 - val_acc: 0.8704 Epoch 25/100 312/312 - 154s - loss: 0.3158 - acc: 0.8639 - val_loss: 0.2870 - val_acc: 0.8804 Epoch 26/100 312/312 - 154s - loss: 0.3332 - acc: 0.8594 - val_loss: 0.2946 - val_acc: 0.8752 Epoch 27/100 312/312 - 154s - loss: 0.3346 - acc: 0.8546 - val_loss: 0.3017 - val_acc: 0.8754 Epoch 28/100 312/312 - 153s - loss: 0.3245 - acc: 0.8595 - val_loss: 0.3335 - val_acc: 0.8401 Epoch 29/100 312/312 - 153s - loss: 0.3180 - acc: 0.8626 - val_loss: 0.3673 - val_acc: 0.8237 Epoch 30/100 312/312 - 153s - loss: 0.3224 - acc: 0.8610 - val_loss: 0.2796 - val_acc: 0.8860 Epoch 31/100 312/312 - 153s - loss: 0.3227 - acc: 0.8613 - val_loss: 0.4363 - val_acc: 0.8173 Epoch 32/100 312/312 - 153s - loss: 0.3156 - acc: 0.8676 - val_loss: 0.5863 - val_acc: 0.8027 Epoch 33/100 312/312 - 154s - loss: 0.3222 - acc: 0.8623 - val_loss: 0.2824 - val_acc: 0.8870 Epoch 34/100 312/312 - 153s - loss: 0.3183 - acc: 0.8629 - val_loss: 0.2856 - val_acc: 0.8790 Epoch 35/100 312/312 - 153s - loss: 0.3152 - acc: 0.8647 - val_loss: 0.3953 - val_acc: 0.8267 Epoch 36/100 312/312 - 154s - loss: 0.3134 - acc: 0.8643 - val_loss: 0.2576 - val_acc: 0.8840 Epoch 37/100 312/312 - 154s - loss: 0.3296 - acc: 0.8595 - val_loss: 0.2680 - val_acc: 0.8838 Epoch 38/100 312/312 - 153s - loss: 0.3285 - acc: 0.8552 - val_loss: 0.3820 - val_acc: 0.8504 Epoch 39/100 312/312 - 154s - loss: 0.3098 - acc: 0.8653 - val_loss: 0.4445 - val_acc: 0.8089 Epoch 40/100 312/312 - 153s - loss: 0.3272 - acc: 0.8579 - val_loss: 0.3113 - val_acc: 0.8758 Epoch 41/100 312/312 - 153s - loss: 0.3382 - acc: 0.8570 - val_loss: 0.3215 - val_acc: 0.8640 Epoch 42/100 312/312 - 154s - loss: 0.3239 - acc: 0.8609 - val_loss: 0.2939 - val_acc: 0.8784 Epoch 43/100 312/312 - 154s - loss: 0.3302 - acc: 0.8582 - val_loss: 0.3737 - val_acc: 0.8431 Epoch 44/100 312/312 - 153s - loss: 0.3178 - acc: 0.8657 - val_loss: 0.2844 - val_acc: 0.8844 Epoch 45/100 312/312 - 153s - loss: 0.3260 - acc: 0.8601 - val_loss: 0.3615 - val_acc: 0.8333 Epoch 46/100 312/312 - 153s - loss: 0.3248 - acc: 0.8603 - val_loss: 0.3506 - val_acc: 0.8488 Epoch 47/100 312/312 - 153s - loss: 0.3260 - acc: 0.8616 - val_loss: 0.2585 - val_acc: 0.8940 Epoch 48/100 312/312 - 154s - loss: 0.3168 - acc: 0.8647 - val_loss: 0.2970 - val_acc: 0.8728 Epoch 49/100 312/312 - 154s - loss: 0.3418 - acc: 0.8630 - val_loss: 0.2806 - val_acc: 0.8852 Epoch 50/100 312/312 - 154s - loss: 0.3146 - acc: 0.8636 - val_loss: 0.2879 - val_acc: 0.8822 Epoch 51/100 312/312 - 153s - loss: 0.3185 - acc: 0.8639 - val_loss: 0.3118 - val_acc: 0.8668 Epoch 52/100 312/312 - 154s - loss: 0.3136 - acc: 0.8610 - val_loss: 0.2760 - val_acc: 0.8852 Epoch 53/100 312/312 - 153s - loss: 0.3079 - acc: 0.8704 - val_loss: 0.3753 - val_acc: 0.8329 Epoch 54/100 312/312 - 153s - loss: 0.3078 - acc: 0.8688 - val_loss: 0.2850 - val_acc: 0.8790 Epoch 55/100 312/312 - 153s - loss: 0.3123 - acc: 0.8658 - val_loss: 0.2774 - val_acc: 0.8884 Epoch 56/100 312/312 - 154s - loss: 0.3161 - acc: 0.8685 - val_loss: 0.3321 - val_acc: 0.8584 Epoch 57/100 312/312 - 153s - loss: 0.3084 - acc: 0.8662 - val_loss: 0.2941 - val_acc: 0.8710 Epoch 58/100 312/312 - 154s - loss: 0.3224 - acc: 0.8651 - val_loss: 0.2577 - val_acc: 0.8906 Epoch 59/100 312/312 - 154s - loss: 0.3105 - acc: 0.8639 - val_loss: 0.3438 - val_acc: 0.8722 Epoch 60/100 312/312 - 153s - loss: 0.3070 - acc: 0.8684 - val_loss: 0.2602 - val_acc: 0.8898 Epoch 61/100 312/312 - 153s - loss: 0.3140 - acc: 0.8658 - val_loss: 0.2677 - val_acc: 0.8882 Epoch 62/100 312/312 - 153s - loss: 0.3096 - acc: 0.8684 - val_loss: 0.2614 - val_acc: 0.8842 Epoch 63/100 312/312 - 153s - loss: 0.3081 - acc: 0.8686 - val_loss: 0.2908 - val_acc: 0.8740 Epoch 64/100 312/312 - 153s - loss: 0.3134 - acc: 0.8668 - val_loss: 0.4757 - val_acc: 0.7314 Epoch 65/100 312/312 - 154s - loss: 0.3085 - acc: 0.8699 - val_loss: 0.3414 - val_acc: 0.8538 Epoch 66/100 TimedStop: Training out of time (Elapsed = 2:55:50.573494) 312/312 - 154s - loss: 0.3208 - acc: 0.8635 - val_loss: 0.3494 - val_acc: 0.8399 2019-07-19 01:24:01,375 graeae.timers.timer end: Ended: 2019-07-19 01:24:01.375685 I0719 01:24:01.375718 139703649425216 timer.py:77] Ended: 2019-07-19 01:24:01.375685 2019-07-19 01:24:01,376 graeae.timers.timer end: Elapsed: 2:55:52.494267 I0719 01:24:01.376679 139703649425216 timer.py:78] Elapsed: 2:55:52.494267
Weirdly, we got up to 89 % validation accuracy this time but it didn't stop. Looking at the stop I had the maximum loss set to 24%, which is probably why it didn't stop, but it looks like I should be able to get better accuracy anyway.
data = pandas.read_csv("~/cats_vs_dogs_2.csv")
print(tabulate(data[data.validation_accuracy>=0.88],
headers="keys", tablefmt="orgtbl"))
| training_loss | training_accuracy | validation_loss | validation_accuracy | |
|---|---|---|---|---|
| 22 | 0.3253 | 0.8642 | 0.2765 | 0.8918 |
| 24 | 0.3158 | 0.8639 | 0.287 | 0.8804 |
| 29 | 0.3224 | 0.861 | 0.2796 | 0.886 |
| 32 | 0.3222 | 0.8623 | 0.2824 | 0.887 |
| 35 | 0.3134 | 0.8643 | 0.2576 | 0.884 |
| 36 | 0.3296 | 0.8595 | 0.268 | 0.8838 |
| 43 | 0.3178 | 0.8657 | 0.2844 | 0.8844 |
| 46 | 0.326 | 0.8616 | 0.2585 | 0.894 |
| 48 | 0.3418 | 0.863 | 0.2806 | 0.8852 |
| 49 | 0.3146 | 0.8636 | 0.2879 | 0.8822 |
| 51 | 0.3136 | 0.861 | 0.276 | 0.8852 |
| 54 | 0.3123 | 0.8658 | 0.2774 | 0.8884 |
| 57 | 0.3224 | 0.8651 | 0.2577 | 0.8906 |
| 59 | 0.307 | 0.8684 | 0.2602 | 0.8898 |
| 60 | 0.314 | 0.8658 | 0.2677 | 0.8882 |
| 61 | 0.3096 | 0.8684 | 0.2614 | 0.8842 |
By increasing the batch size we got better faster. So, maybe it isn't done learning yet. Or maybe a larger batch size would be useful. It doesn't seem to be improving, by much, though.
best_accuracy = data.validation_accuracy.max()
best_loss = data.validation_loss.min()
accuracy_index = data[data.validation_accuracy==best_accuracy].index[0]
loss_index = data[data.validation_loss==best_loss].index[0]
print(f"Highest Accuracy: {best_accuracy} Epoch: {accuracy_index + 1}")
print(f"Lowest Loss: {best_loss} Epoch: {loss_index}")
Highest Accuracy: 0.894 Epoch: 47 Lowest Loss: 0.2576 Epoch: 35
line_1 = holoviews.VLine(accuracy_index, label="Best Accuracy Epoch")
line_2 = holoviews.VLine(loss_index, label="Best Loss Epoch")
accuracy_line = holoviews.HLine(best_accuracy, label="Highest Accuracy")
loss_line = holoviews.HLine(best_loss, label="lowest Loss")
curves = [holoviews.Curve(data, ("index", "Epoch"), "training_loss",
label="Training Loss",),
holoviews.Curve(data, ("index", "Epoch"), "training_accuracy",
label="Training Accuracy").opts(tools=["hover"]),
holoviews.Curve(data, ("index", "Epoch"), "validation_loss",
label="Validation Loss",).opts(tools=["hover"]),
holoviews.Curve(data, ("index", "Epoch"), "validation_accuracy",
label="Validation Accuracy").opts(tools=["hover"]),
line_1, line_2, accuracy_line, loss_line]
plot = holoviews.Overlay(curves).opts(tools=["hover"], height=800, width=1000,
ylabel="Performance",
title="Training vs Validation")
Embed(plot=plot, file_name="training_validation_loss_13")()
It looks like it might be plateauing. Only one way to really find out, I guess - more training.
- Take Thirteen Point Two
I'll re-train it without using the Stop condition to see if it gets better than I was allowing it to get.
saver = partial(save_model, path=path) good_enough = Stop(call_on_stopping=saver, minimum_accuracy=0.90) out_of_time = TimedStop(call_on_stopping=saver) network = Network(str(training_path), callbacks=[out_of_time], convolution_layers=5, set_steps = True, epochs = 100, batch_size=64) network._model = tensorflow.keras.models.load_model(str(path)) with TIMER: network.train()
W0720 10:27:38.686592 139935525873472 deprecation.py:506] From /home/athena/.virtualenvs/In-Too-Deep/lib/python3.7/site-packages/tensorflow/python/ops/init_ops.py:97: calling GlorotUniform.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version. Instructions for updating: Call initializer instance with the dtype argument instead of passing it to the constructor W0720 10:27:38.688725 139935525873472 deprecation.py:506] From /home/athena/.virtualenvs/In-Too-Deep/lib/python3.7/site-packages/tensorflow/python/ops/init_ops.py:1251: calling VarianceScaling.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version. Instructions for updating: Call initializer instance with the dtype argument instead of passing it to the constructor W0720 10:27:38.690803 139935525873472 deprecation.py:506] From /home/athena/.virtualenvs/In-Too-Deep/lib/python3.7/site-packages/tensorflow/python/ops/init_ops.py:97: calling Zeros.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version. Instructions for updating: Call initializer instance with the dtype argument instead of passing it to the constructor W0720 10:27:53.533317 139935525873472 deprecation.py:323] From /home/athena/.virtualenvs/In-Too-Deep/lib/python3.7/site-packages/tensorflow/python/ops/nn_impl.py:180: add_dispatch_support.<locals>.wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version. Instructions for updating: Use tf.where in 2.0, which has the same broadcast rule as np.where 2019-07-20 10:27:54,312 graeae.timers.timer start: Started: 2019-07-20 10:27:54.312000 I0720 10:27:54.312226 139935525873472 timer.py:70] Started: 2019-07-20 10:27:54.312000 Found 20000 images belonging to 2 classes. Found 5000 images belonging to 2 classes. Epoch 1/100 312/312 - 448s - loss: 0.3622 - acc: 0.8679 - val_loss: 0.2536 - val_acc: 0.8898 Epoch 2/100 312/312 - 154s - loss: 0.3059 - acc: 0.8716 - val_loss: 0.2927 - val_acc: 0.8932 Epoch 3/100 312/312 - 154s - loss: 0.3164 - acc: 0.8691 - val_loss: 0.3203 - val_acc: 0.8620 Epoch 4/100 312/312 - 154s - loss: 0.3190 - acc: 0.8620 - val_loss: 1.0261 - val_acc: 0.6781 Epoch 5/100 312/312 - 153s - loss: 0.3308 - acc: 0.8662 - val_loss: 0.2791 - val_acc: 0.8736 Epoch 6/100 312/312 - 152s - loss: 0.3234 - acc: 0.8633 - val_loss: 0.2434 - val_acc: 0.9054 Epoch 7/100 312/312 - 151s - loss: 0.3040 - acc: 0.8702 - val_loss: 0.2895 - val_acc: 0.8828 Epoch 8/100 312/312 - 150s - loss: 0.3063 - acc: 0.8714 - val_loss: 0.2788 - val_acc: 0.8744 Epoch 9/100 312/312 - 150s - loss: 0.3043 - acc: 0.8717 - val_loss: 0.4412 - val_acc: 0.8011 Epoch 10/100 312/312 - 150s - loss: 0.3059 - acc: 0.8690 - val_loss: 0.3050 - val_acc: 0.8784 Epoch 11/100 312/312 - 150s - loss: 0.3368 - acc: 0.8633 - val_loss: 0.2813 - val_acc: 0.8864 Epoch 12/100 312/312 - 150s - loss: 0.3282 - acc: 0.8633 - val_loss: 0.4125 - val_acc: 0.8209 Epoch 13/100 312/312 - 150s - loss: 0.3091 - acc: 0.8701 - val_loss: 0.3506 - val_acc: 0.8842 Epoch 14/100 312/312 - 149s - loss: 0.3369 - acc: 0.8585 - val_loss: 0.2702 - val_acc: 0.8908 Epoch 15/100 312/312 - 150s - loss: 0.3253 - acc: 0.8624 - val_loss: 0.2360 - val_acc: 0.8996 Epoch 16/100 312/312 - 149s - loss: 0.3206 - acc: 0.8622 - val_loss: 0.4118 - val_acc: 0.8454 Epoch 17/100 312/312 - 149s - loss: 0.3396 - acc: 0.8629 - val_loss: 0.4940 - val_acc: 0.7899 Epoch 18/100 312/312 - 149s - loss: 0.3190 - acc: 0.8632 - val_loss: 0.2928 - val_acc: 0.8762 Epoch 19/100 312/312 - 150s - loss: 0.3154 - acc: 0.8658 - val_loss: 0.2806 - val_acc: 0.8914 Epoch 20/100 312/312 - 151s - loss: 0.3241 - acc: 0.8632 - val_loss: 0.2797 - val_acc: 0.8778 Epoch 21/100 312/312 - 150s - loss: 0.4349 - acc: 0.8620 - val_loss: 0.2824 - val_acc: 0.8836 Epoch 22/100 312/312 - 150s - loss: 0.3255 - acc: 0.8626 - val_loss: 0.3107 - val_acc: 0.8736 Epoch 23/100 312/312 - 150s - loss: 0.3139 - acc: 0.8678 - val_loss: 0.2830 - val_acc: 0.8796 Epoch 24/100 312/312 - 151s - loss: 0.3231 - acc: 0.8641 - val_loss: 0.3103 - val_acc: 0.8874 Epoch 25/100 312/312 - 151s - loss: 0.3222 - acc: 0.8647 - val_loss: 0.7359 - val_acc: 0.6821 Epoch 26/100 312/312 - 151s - loss: 0.3140 - acc: 0.8657 - val_loss: 0.5905 - val_acc: 0.8235 Epoch 27/100 312/312 - 150s - loss: 0.3431 - acc: 0.8684 - val_loss: 0.3469 - val_acc: 0.8776 Epoch 28/100 312/312 - 149s - loss: 0.3124 - acc: 0.8681 - val_loss: 0.3550 - val_acc: 0.8444 Epoch 29/100 312/312 - 149s - loss: 0.3159 - acc: 0.8682 - val_loss: 0.3107 - val_acc: 0.8642 Epoch 30/100 312/312 - 149s - loss: 0.3177 - acc: 0.8621 - val_loss: 0.2972 - val_acc: 0.8700 Epoch 31/100 312/312 - 149s - loss: 0.3370 - acc: 0.8636 - val_loss: 0.2857 - val_acc: 0.8876 Epoch 32/100 312/312 - 149s - loss: 0.3476 - acc: 0.8602 - val_loss: 0.2762 - val_acc: 0.8866 Epoch 33/100 312/312 - 150s - loss: 0.3383 - acc: 0.8600 - val_loss: 0.3066 - val_acc: 0.8874 Epoch 34/100 312/312 - 150s - loss: 0.3316 - acc: 0.8681 - val_loss: 0.4850 - val_acc: 0.7244 Epoch 35/100 312/312 - 149s - loss: 0.3158 - acc: 0.8642 - val_loss: 0.2958 - val_acc: 0.8748 Epoch 36/100 312/312 - 151s - loss: 0.3285 - acc: 0.8597 - val_loss: 0.2739 - val_acc: 0.8762 Epoch 37/100 312/312 - 150s - loss: 0.3239 - acc: 0.8622 - val_loss: 0.3081 - val_acc: 0.8714 Epoch 38/100 312/312 - 150s - loss: 0.3277 - acc: 0.8601 - val_loss: 0.3068 - val_acc: 0.8882 Epoch 39/100 312/312 - 152s - loss: 0.3228 - acc: 0.8601 - val_loss: 0.3480 - val_acc: 0.8546 Epoch 40/100 312/312 - 152s - loss: 0.3631 - acc: 0.8633 - val_loss: 0.2939 - val_acc: 0.8906 Epoch 41/100 312/312 - 152s - loss: 0.3313 - acc: 0.8592 - val_loss: 0.2674 - val_acc: 0.8918 Epoch 42/100 312/312 - 152s - loss: 0.4150 - acc: 0.8648 - val_loss: 0.2846 - val_acc: 0.8796 Epoch 43/100 312/312 - 154s - loss: 0.3161 - acc: 0.8665 - val_loss: 0.2665 - val_acc: 0.8858 Epoch 44/100 312/312 - 153s - loss: 0.3330 - acc: 0.8627 - val_loss: 0.2977 - val_acc: 0.8726 Epoch 45/100 312/312 - 149s - loss: 0.3250 - acc: 0.8668 - val_loss: 0.2746 - val_acc: 0.8796 Epoch 46/100 312/312 - 153s - loss: 0.3091 - acc: 0.8715 - val_loss: 0.2864 - val_acc: 0.8924 Epoch 47/100 312/312 - 151s - loss: 0.3288 - acc: 0.8622 - val_loss: 0.2662 - val_acc: 0.8886 Epoch 48/100 312/312 - 149s - loss: 0.3594 - acc: 0.8589 - val_loss: 0.2818 - val_acc: 0.8842 Epoch 49/100 312/312 - 148s - loss: 0.3275 - acc: 0.8604 - val_loss: 0.4913 - val_acc: 0.7610 Epoch 50/100 312/312 - 149s - loss: 0.3611 - acc: 0.8586 - val_loss: 0.3517 - val_acc: 0.8438 Epoch 51/100 312/312 - 148s - loss: 0.3413 - acc: 0.8568 - val_loss: 0.3549 - val_acc: 0.8602 Epoch 52/100 312/312 - 148s - loss: 0.3275 - acc: 0.8627 - val_loss: 0.2567 - val_acc: 0.8926 Epoch 53/100 312/312 - 148s - loss: 0.3679 - acc: 0.8592 - val_loss: 0.3676 - val_acc: 0.8554 Epoch 54/100 312/312 - 148s - loss: 0.3332 - acc: 0.8580 - val_loss: 0.2862 - val_acc: 0.8754 Epoch 55/100 312/312 - 148s - loss: 0.3254 - acc: 0.8644 - val_loss: 11.4265 - val_acc: 0.5905 Epoch 56/100 312/312 - 148s - loss: 0.3735 - acc: 0.8634 - val_loss: 0.3241 - val_acc: 0.8728 Epoch 57/100 312/312 - 148s - loss: 0.3401 - acc: 0.8588 - val_loss: 0.2946 - val_acc: 0.8746 Epoch 58/100 312/312 - 149s - loss: 0.4813 - acc: 0.8624 - val_loss: 0.3596 - val_acc: 0.8452 Epoch 59/100 312/312 - 148s - loss: 0.3279 - acc: 0.8633 - val_loss: 0.2789 - val_acc: 0.8734 Epoch 60/100 312/312 - 147s - loss: 0.3375 - acc: 0.8591 - val_loss: 0.3680 - val_acc: 0.8331 Epoch 61/100 312/312 - 148s - loss: 0.3359 - acc: 0.8585 - val_loss: 0.3290 - val_acc: 0.8802 Epoch 62/100 312/312 - 148s - loss: 0.3259 - acc: 0.8633 - val_loss: 0.3308 - val_acc: 0.8676 Epoch 63/100 312/312 - 149s - loss: 0.3290 - acc: 0.8598 - val_loss: 0.2710 - val_acc: 0.8850 Epoch 64/100 312/312 - 148s - loss: 0.3383 - acc: 0.8507 - val_loss: 0.2740 - val_acc: 0.8838 Epoch 65/100 312/312 - 149s - loss: 0.3287 - acc: 0.8627 - val_loss: 0.3301 - val_acc: 0.8540 Epoch 66/100 312/312 - 149s - loss: 1.9072 - acc: 0.8593 - val_loss: 0.3180 - val_acc: 0.8602 Epoch 67/100 312/312 - 148s - loss: 0.3265 - acc: 0.8597 - val_loss: 0.2972 - val_acc: 0.8708 Epoch 68/100 312/312 - 150s - loss: 0.3515 - acc: 0.8613 - val_loss: 0.4104 - val_acc: 0.8444 Epoch 69/100 TimedStop: Training out of time (Elapsed = 2:57:23.471615) TimedStop: Longest Epoch = 2:57:23.471615 312/312 - 148s - loss: 0.3617 - acc: 0.8605 - val_loss: 0.6406 - val_acc: 0.8221 2019-07-20 13:25:18,577 graeae.timers.timer end: Ended: 2019-07-20 13:25:18.577775 I0720 13:25:18.577804 139935525873472 timer.py:77] Ended: 2019-07-20 13:25:18.577775 2019-07-20 13:25:18,578 graeae.timers.timer end: Elapsed: 2:57:24.265775 I0720 13:25:18.578630 139935525873472 timer.py:78] Elapsed: 2:57:24.265775
data = pandas.read_csv("~/cats_vs_dogs_3.csv") print(tabulate(data[data.validation_accuracy>=0.89], headers="keys", tablefmt="orgtbl"))
training_loss training_accuracy validation_loss validation_accuracy 1 0.3059 0.8716 0.2927 0.8932 5 0.3234 0.8633 0.2434 0.9054 13 0.3369 0.8585 0.2702 0.8908 14 0.3253 0.8624 0.236 0.8996 18 0.3154 0.8658 0.2806 0.8914 39 0.3631 0.8633 0.2939 0.8906 40 0.3313 0.8592 0.2674 0.8918 45 0.3091 0.8715 0.2864 0.8924 51 0.3275 0.8627 0.2567 0.8926 best_accuracy = data.validation_accuracy.max() best_loss = data.validation_loss.min() accuracy_index = data[data.validation_accuracy==best_accuracy].index[0] loss_index = data[data.validation_loss==best_loss].index[0] print(f"Highest Accuracy: {best_accuracy} Epoch: {accuracy_index + 1}") print(f"Lowest Loss: {best_loss} Epoch: {loss_index}")
Highest Accuracy: 0.9054 Epoch: 6 Lowest Loss: 0.236 Epoch: 14
line_1 = holoviews.VLine(accuracy_index, label="Best Accuracy Epoch") line_2 = holoviews.VLine(loss_index, label="Best Loss Epoch") accuracy_line = holoviews.HLine(best_accuracy, label="Highest Accuracy") loss_line = holoviews.HLine(best_loss, label="lowest Loss") curves = [holoviews.Curve(data, ("index", "Epoch"), "training_loss", label="Training Loss",), holoviews.Curve(data, ("index", "Epoch"), "training_accuracy", label="Training Accuracy").opts(tools=["hover"]), holoviews.Curve(data, ("index", "Epoch"), "validation_loss", label="Validation Loss",).opts(tools=["hover"]), holoviews.Curve(data, ("index", "Epoch"), "validation_accuracy", label="Validation Accuracy").opts(tools=["hover"]), line_1, line_2, accuracy_line, loss_line] plot = holoviews.Overlay(curves).opts(tools=["hover"], height=800, width=1000, ylabel="Performance", title="Training vs Validation") Embed(plot=plot, file_name="training_validation_loss_13")()
So, I'm not sure what to make of this. It looks like it did improve a little, but that it peaked early on, and yet kept rising back up to a high level of validation accuracy. Is it overfitting or not? I guess I'll train it some more and find out.
with TIMER: network.train()
2019-07-20 13:37:51,641 graeae.timers.timer start: Started: 2019-07-20 13:37:51.641628 I0720 13:37:51.641655 139935525873472 timer.py:70] Started: 2019-07-20 13:37:51.641628 Epoch 1/100 312/312 - 152s - loss: 0.3367 - acc: 0.8564 - val_loss: 0.4919 - val_acc: 0.7398 Epoch 2/100 312/312 - 152s - loss: 0.3354 - acc: 0.8538 - val_loss: 0.4213 - val_acc: 0.7724 Epoch 3/100 312/312 - 151s - loss: 0.3568 - acc: 0.8515 - val_loss: 0.4105 - val_acc: 0.8249 Epoch 4/100 312/312 - 152s - loss: 0.3413 - acc: 0.8557 - val_loss: 0.2920 - val_acc: 0.8850 Epoch 5/100 312/312 - 154s - loss: 0.3542 - acc: 0.8528 - val_loss: 0.2883 - val_acc: 0.8846 Epoch 6/100 312/312 - 153s - loss: 0.3494 - acc: 0.8558 - val_loss: 0.4051 - val_acc: 0.7983 Epoch 7/100 312/312 - 151s - loss: 0.3653 - acc: 0.8487 - val_loss: 0.3081 - val_acc: 0.8710 Epoch 8/100 312/312 - 152s - loss: 0.3699 - acc: 0.8447 - val_loss: 0.2843 - val_acc: 0.8882 Epoch 9/100 312/312 - 151s - loss: 0.8043 - acc: 0.8519 - val_loss: 0.3204 - val_acc: 0.8518 Epoch 10/100 312/312 - 151s - loss: 0.3429 - acc: 0.8528 - val_loss: 0.2779 - val_acc: 0.8796 Epoch 11/100 312/312 - 150s - loss: 0.3486 - acc: 0.8572 - val_loss: 0.3647 - val_acc: 0.8435 Epoch 12/100 312/312 - 149s - loss: 0.3324 - acc: 0.8572 - val_loss: 0.3560 - val_acc: 0.8602 Epoch 13/100 312/312 - 149s - loss: 0.3465 - acc: 0.8529 - val_loss: 0.3772 - val_acc: 0.8584 Epoch 14/100 312/312 - 148s - loss: 0.4045 - acc: 0.8446 - val_loss: 0.4425 - val_acc: 0.7360 Epoch 15/100 312/312 - 150s - loss: 6.0508 - acc: 0.8242 - val_loss: 0.3422 - val_acc: 0.8460 Epoch 16/100 312/312 - 150s - loss: 0.3486 - acc: 0.8495 - val_loss: 0.4366 - val_acc: 0.7712 Epoch 17/100 312/312 - 149s - loss: 0.3509 - acc: 0.8523 - val_loss: 0.2882 - val_acc: 0.8730 Epoch 18/100 312/312 - 149s - loss: 0.3509 - acc: 0.8513 - val_loss: 0.3403 - val_acc: 0.8508 Epoch 19/100 312/312 - 148s - loss: 0.3557 - acc: 0.8486 - val_loss: 0.2864 - val_acc: 0.8886 Epoch 20/100 312/312 - 148s - loss: 0.3596 - acc: 0.8491 - val_loss: 0.3974 - val_acc: 0.7899 Epoch 21/100 312/312 - 149s - loss: 0.3630 - acc: 0.8465 - val_loss: 0.3558 - val_acc: 0.8482 Epoch 22/100 312/312 - 149s - loss: 0.3745 - acc: 0.8441 - val_loss: 0.2867 - val_acc: 0.8870 Epoch 23/100 312/312 - 148s - loss: 0.3749 - acc: 0.8490 - val_loss: 0.2815 - val_acc: 0.8806 Epoch 24/100 312/312 - 149s - loss: 0.5479 - acc: 0.8500 - val_loss: 0.3382 - val_acc: 0.8830 Epoch 25/100 312/312 - 148s - loss: 0.3600 - acc: 0.8485 - val_loss: 0.5508 - val_acc: 0.6703 Epoch 26/100 312/312 - 148s - loss: 0.3494 - acc: 0.8533 - val_loss: 0.4929 - val_acc: 0.7678 Epoch 27/100 312/312 - 148s - loss: 0.3487 - acc: 0.8493 - val_loss: 0.3008 - val_acc: 0.8728 Epoch 28/100 312/312 - 148s - loss: 0.3657 - acc: 0.8483 - val_loss: 0.3355 - val_acc: 0.8668 Epoch 29/100 312/312 - 148s - loss: 0.3730 - acc: 0.8422 - val_loss: 0.7513 - val_acc: 0.7348 Epoch 30/100 312/312 - 148s - loss: 0.3703 - acc: 0.8477 - val_loss: 0.3497 - val_acc: 0.8558 Epoch 31/100 312/312 - 148s - loss: 0.3626 - acc: 0.8453 - val_loss: 0.2778 - val_acc: 0.8940 Epoch 32/100 312/312 - 148s - loss: 0.3627 - acc: 0.8470 - val_loss: 0.2933 - val_acc: 0.8768 Epoch 33/100 312/312 - 148s - loss: 0.4131 - acc: 0.8341 - val_loss: 0.3067 - val_acc: 0.8698 Epoch 34/100 312/312 - 149s - loss: 0.3766 - acc: 0.8491 - val_loss: 0.5967 - val_acc: 0.6657 Epoch 35/100 312/312 - 148s - loss: 0.5109 - acc: 0.8372 - val_loss: 0.3495 - val_acc: 0.8440 Epoch 36/100 312/312 - 148s - loss: 0.3736 - acc: 0.8472 - val_loss: 0.7757 - val_acc: 0.7460 Epoch 37/100 312/312 - 149s - loss: 0.3557 - acc: 0.8509 - val_loss: 0.3210 - val_acc: 0.8726 Epoch 38/100 312/312 - 149s - loss: 0.3841 - acc: 0.8386 - val_loss: 0.2899 - val_acc: 0.8640 Epoch 39/100 312/312 - 149s - loss: 0.4163 - acc: 0.8462 - val_loss: 0.4341 - val_acc: 0.7977 Epoch 40/100 312/312 - 148s - loss: 0.3674 - acc: 0.8446 - val_loss: 0.5823 - val_acc: 0.8526 Epoch 41/100 312/312 - 148s - loss: 0.3612 - acc: 0.8510 - val_loss: 0.2640 - val_acc: 0.8804 Epoch 42/100 312/312 - 149s - loss: 0.3752 - acc: 0.8491 - val_loss: 0.4750 - val_acc: 0.7726 Epoch 43/100 312/312 - 149s - loss: 0.3563 - acc: 0.8424 - val_loss: 0.3072 - val_acc: 0.8810 Epoch 44/100 312/312 - 149s - loss: 0.3587 - acc: 0.8474 - val_loss: 0.3906 - val_acc: 0.8866 Epoch 45/100 312/312 - 149s - loss: 0.3615 - acc: 0.8461 - val_loss: 0.5745 - val_acc: 0.7776 Epoch 46/100 312/312 - 149s - loss: 0.4017 - acc: 0.8384 - val_loss: 0.3786 - val_acc: 0.8722 Epoch 47/100 312/312 - 149s - loss: 0.3969 - acc: 0.8383 - val_loss: 0.4942 - val_acc: 0.7286 Epoch 48/100 312/312 - 148s - loss: 0.3628 - acc: 0.8395 - val_loss: 0.3358 - val_acc: 0.8606 Epoch 49/100 312/312 - 149s - loss: 0.3804 - acc: 0.8353 - val_loss: 0.3131 - val_acc: 0.8698 Epoch 50/100 312/312 - 150s - loss: 0.3997 - acc: 0.8378 - val_loss: 0.4310 - val_acc: 0.7873 Epoch 51/100 312/312 - 152s - loss: 0.3905 - acc: 0.8364 - val_loss: 0.3262 - val_acc: 0.8750 Epoch 52/100 312/312 - 151s - loss: 0.4109 - acc: 0.8449 - val_loss: 0.7448 - val_acc: 0.7512 Epoch 53/100 312/312 - 150s - loss: 0.3737 - acc: 0.8426 - val_loss: 0.3926 - val_acc: 0.8317 Epoch 54/100 312/312 - 148s - loss: 0.3573 - acc: 0.8420 - val_loss: 0.3395 - val_acc: 0.8516 Epoch 55/100 312/312 - 149s - loss: 0.3886 - acc: 0.8314 - val_loss: 0.3221 - val_acc: 0.8423 Epoch 56/100 312/312 - 149s - loss: 0.3991 - acc: 0.8325 - val_loss: 0.4920 - val_acc: 0.8243 Epoch 57/100 312/312 - 149s - loss: 0.3845 - acc: 0.8332 - val_loss: 0.3332 - val_acc: 0.8518 Epoch 58/100 312/312 - 149s - loss: 0.3764 - acc: 0.8336 - val_loss: 0.5819 - val_acc: 0.7029 Epoch 59/100 312/312 - 149s - loss: 0.4083 - acc: 0.8326 - val_loss: 0.3206 - val_acc: 0.8494 Epoch 60/100 312/312 - 149s - loss: 0.4010 - acc: 0.8372 - val_loss: 1.2613 - val_acc: 0.6791 Epoch 61/100 312/312 - 149s - loss: 0.4093 - acc: 0.8316 - val_loss: 0.4272 - val_acc: 0.7905 Epoch 62/100 312/312 - 149s - loss: 0.4389 - acc: 0.8319 - val_loss: 0.3596 - val_acc: 0.8486 Epoch 63/100 312/312 - 148s - loss: 0.4066 - acc: 0.8385 - val_loss: 0.3253 - val_acc: 0.8774 Epoch 64/100 312/312 - 148s - loss: 0.4000 - acc: 0.8325 - val_loss: 0.3190 - val_acc: 0.8920 Epoch 65/100 312/312 - 149s - loss: 0.3924 - acc: 0.8355 - val_loss: 0.5483 - val_acc: 0.6829 Epoch 66/100 312/312 - 150s - loss: 0.4344 - acc: 0.8272 - val_loss: 0.4520 - val_acc: 0.7971 Epoch 67/100 312/312 - 150s - loss: 0.3973 - acc: 0.8338 - val_loss: 0.2962 - val_acc: 0.8770 Epoch 68/100 312/312 - 150s - loss: 0.3868 - acc: 0.8299 - val_loss: 0.3322 - val_acc: 0.8510 Epoch 69/100 312/312 - 150s - loss: 0.4051 - acc: 0.8243 - val_loss: 0.4731 - val_acc: 0.8295 Epoch 70/100 312/312 - 150s - loss: 0.4241 - acc: 0.8228 - val_loss: 0.3180 - val_acc: 0.8682 Epoch 71/100 TimedStop: Training out of time (Elapsed = 2:56:41.032776) TimedStop: Longest Epoch = 2:56:41.032776 312/312 - 149s - loss: 0.4185 - acc: 0.8227 - val_loss: 0.3691 - val_acc: 0.8041 2019-07-20 16:34:32,739 graeae.timers.timer end: Ended: 2019-07-20 16:34:32.739264 I0720 16:34:32.739288 139935525873472 timer.py:77] Ended: 2019-07-20 16:34:32.739264 2019-07-20 16:34:32,740 graeae.timers.timer end: Elapsed: 2:56:41.097636 I0720 16:34:32.740128 139935525873472 timer.py:78] Elapsed: 2:56:41.097636
data = pandas.read_csv("~/cats_vs_dogs_4.csv") print(tabulate(data[data.validation_accuracy>=0.89], headers="keys", tablefmt="orgtbl"))
training_loss training_accuracy validation_loss validation_accuracy 30 0.3626 0.8453 0.2778 0.894 63 0.4 0.8325 0.319 0.892 So, we don't see a lot of degradation, but we also don't see much improvement.
best_accuracy = data.validation_accuracy.max() best_loss = data.validation_loss.min() accuracy_index = data[data.validation_accuracy==best_accuracy].index[0] loss_index = data[data.validation_loss==best_loss].index[0] print(f"Highest Accuracy: {best_accuracy} Epoch: {accuracy_index + 1}") print(f"Lowest Loss: {best_loss} Epoch: {loss_index}")
Highest Accuracy: 0.894 Epoch: 31 Lowest Loss: 0.264 Epoch: 40
line_1 = holoviews.VLine(accuracy_index, label="Best Accuracy Epoch") line_2 = holoviews.VLine(loss_index, label="Best Loss Epoch") accuracy_line = holoviews.HLine(best_accuracy, label="Highest Accuracy") loss_line = holoviews.HLine(best_loss, label="lowest Loss") curves = [holoviews.Curve(data, ("index", "Epoch"), "training_loss", label="Training Loss",), holoviews.Curve(data, ("index", "Epoch"), "training_accuracy", label="Training Accuracy").opts(tools=["hover"]), holoviews.Curve(data, ("index", "Epoch"), "validation_loss", label="Validation Loss",).opts(tools=["hover"]), holoviews.Curve(data, ("index", "Epoch"), "validation_accuracy", label="Validation Accuracy").opts(tools=["hover"]), line_1, line_2, accuracy_line, loss_line] plot = holoviews.Overlay(curves).opts(tools=["hover"], height=800, width=1000, ylabel="Performance", title="Training vs Validation") Embed(plot=plot, file_name="training_validation_loss_13")()
Take Fourteen
Okay, I'm going to say tha 90 % is our target.
saver = partial(save_model, path=path)
good_enough = Stop(call_on_stopping=saver,
minimum_accuracy=0.9)
network = Network(str(training_path),
callbacks=[good_enough],
convolution_layers=5,
set_steps = True,
epochs = 100,
batch_size=128)
print(str(network))
with TIMER:
network.train()
2019-07-20 18:21:55,292 graeae.timers.timer start: Started: 2019-07-20 18:21:55.292917 I0720 18:21:55.292948 139935525873472 timer.py:70] Started: 2019-07-20 18:21:55.292917 (Network) - Path: /home/athena/data/datasets/images/dogs-vs-cats/train Epochs: 100 Batch Size: 128 Callbacks: [<__main__.Stop object at 0x7f440c04d518>] Data: (Data) - Path: /home/athena/data/datasets/images/dogs-vs-cats/train, Validation Split: 0.2,Batch Size: 128 Callbacks: [<__main__.Stop object at 0x7f440c04d518>] Found 20000 images belonging to 2 classes. Found 5000 images belonging to 2 classes. Epoch 1/100 156/156 - 158s - loss: 0.7231 - acc: 0.5330 - val_loss: 0.6808 - val_acc: 0.5960 Epoch 2/100 156/156 - 158s - loss: 0.6639 - acc: 0.6077 - val_loss: 0.6349 - val_acc: 0.6396 Epoch 3/100 156/156 - 157s - loss: 0.6362 - acc: 0.6355 - val_loss: 0.6278 - val_acc: 0.6306 Epoch 4/100 156/156 - 158s - loss: 0.6144 - acc: 0.6669 - val_loss: 0.6162 - val_acc: 0.6522 Epoch 5/100 156/156 - 157s - loss: 0.5877 - acc: 0.6917 - val_loss: 0.5351 - val_acc: 0.7424 Epoch 6/100 156/156 - 152s - loss: 0.5736 - acc: 0.7032 - val_loss: 0.5579 - val_acc: 0.7125 Epoch 7/100 156/156 - 152s - loss: 0.5476 - acc: 0.7265 - val_loss: 0.5104 - val_acc: 0.7510 Epoch 8/100 156/156 - 151s - loss: 0.5268 - acc: 0.7372 - val_loss: 0.4760 - val_acc: 0.7692 Epoch 9/100 156/156 - 150s - loss: 0.5057 - acc: 0.7574 - val_loss: 0.4546 - val_acc: 0.7800 Epoch 10/100 156/156 - 149s - loss: 0.4862 - acc: 0.7667 - val_loss: 0.4432 - val_acc: 0.7987 Epoch 11/100 156/156 - 149s - loss: 0.4697 - acc: 0.7787 - val_loss: 0.4413 - val_acc: 0.7941 Epoch 12/100 156/156 - 151s - loss: 0.4419 - acc: 0.7924 - val_loss: 0.5044 - val_acc: 0.7454 Epoch 13/100 156/156 - 150s - loss: 0.4204 - acc: 0.8046 - val_loss: 0.4150 - val_acc: 0.8059 Epoch 14/100 156/156 - 150s - loss: 0.4038 - acc: 0.8155 - val_loss: 0.3776 - val_acc: 0.8245 Epoch 15/100 156/156 - 149s - loss: 0.3849 - acc: 0.8244 - val_loss: 0.4047 - val_acc: 0.8033 Epoch 16/100 156/156 - 149s - loss: 0.3745 - acc: 0.8311 - val_loss: 0.6245 - val_acc: 0.6635 Epoch 17/100 156/156 - 150s - loss: 0.3556 - acc: 0.8410 - val_loss: 0.3274 - val_acc: 0.8544 Epoch 18/100 156/156 - 150s - loss: 0.3434 - acc: 0.8447 - val_loss: 0.3143 - val_acc: 0.8606 Epoch 19/100 156/156 - 150s - loss: 0.3356 - acc: 0.8473 - val_loss: 0.3181 - val_acc: 0.8600 Epoch 20/100 156/156 - 148s - loss: 0.3295 - acc: 0.8551 - val_loss: 0.3042 - val_acc: 0.8630 Epoch 21/100 156/156 - 150s - loss: 0.3165 - acc: 0.8612 - val_loss: 0.2961 - val_acc: 0.8668 Epoch 22/100 156/156 - 149s - loss: 0.3060 - acc: 0.8650 - val_loss: 0.3530 - val_acc: 0.8484 Epoch 23/100 156/156 - 148s - loss: 0.3062 - acc: 0.8659 - val_loss: 0.3327 - val_acc: 0.8494 Epoch 24/100 156/156 - 150s - loss: 0.2942 - acc: 0.8725 - val_loss: 0.2533 - val_acc: 0.8944 Epoch 25/100 156/156 - 150s - loss: 0.2897 - acc: 0.8712 - val_loss: 0.3440 - val_acc: 0.8458 Epoch 26/100 156/156 - 149s - loss: 0.2829 - acc: 0.8769 - val_loss: 0.2627 - val_acc: 0.8842 Epoch 27/100 156/156 - 150s - loss: 0.2691 - acc: 0.8836 - val_loss: 0.2857 - val_acc: 0.8840 Epoch 28/100 156/156 - 149s - loss: 0.2814 - acc: 0.8780 - val_loss: 0.2717 - val_acc: 0.8800 Epoch 29/100 156/156 - 150s - loss: 0.2632 - acc: 0.8826 - val_loss: 0.3521 - val_acc: 0.8584 Epoch 30/100 156/156 - 150s - loss: 0.2652 - acc: 0.8868 - val_loss: 0.2736 - val_acc: 0.8872 Epoch 31/100 156/156 - 148s - loss: 0.2591 - acc: 0.8875 - val_loss: 0.2842 - val_acc: 0.8782 Epoch 32/100 156/156 - 150s - loss: 0.2569 - acc: 0.8851 - val_loss: 0.2657 - val_acc: 0.9014 Epoch 33/100 156/156 - 149s - loss: 0.2623 - acc: 0.8866 - val_loss: 0.2539 - val_acc: 0.8894 Epoch 34/100 156/156 - 150s - loss: 0.2564 - acc: 0.8893 - val_loss: 0.3088 - val_acc: 0.8606 Epoch 35/100 156/156 - 151s - loss: 0.2496 - acc: 0.8924 - val_loss: 0.2415 - val_acc: 0.8996 Epoch 36/100 156/156 - 147s - loss: 0.2459 - acc: 0.8918 - val_loss: 0.2661 - val_acc: 0.8816 Epoch 37/100 156/156 - 147s - loss: 0.2473 - acc: 0.8939 - val_loss: 0.2842 - val_acc: 0.8826 Epoch 38/100 156/156 - 146s - loss: 0.2418 - acc: 0.8948 - val_loss: 0.2875 - val_acc: 0.8864 Epoch 39/100 156/156 - 145s - loss: 0.2502 - acc: 0.8924 - val_loss: 0.2174 - val_acc: 0.9089 Epoch 40/100 156/156 - 146s - loss: 0.2416 - acc: 0.8966 - val_loss: 0.2251 - val_acc: 0.9058 Epoch 41/100 156/156 - 147s - loss: 0.2489 - acc: 0.8934 - val_loss: 0.2548 - val_acc: 0.8958 Epoch 42/100 156/156 - 150s - loss: 0.2341 - acc: 0.9011 - val_loss: 0.2150 - val_acc: 0.9065 Epoch 43/100 156/156 - 151s - loss: 0.2400 - acc: 0.9011 - val_loss: 0.2103 - val_acc: 0.9131 Epoch 44/100 156/156 - 150s - loss: 0.2340 - acc: 0.8990 - val_loss: 0.6287 - val_acc: 0.7945 Epoch 45/100 156/156 - 150s - loss: 0.2359 - acc: 0.8981 - val_loss: 0.2213 - val_acc: 0.9131 Epoch 46/100 156/156 - 150s - loss: 0.2277 - acc: 0.9034 - val_loss: 0.2491 - val_acc: 0.8920 Epoch 47/100 156/156 - 151s - loss: 0.2433 - acc: 0.8946 - val_loss: 0.2540 - val_acc: 0.8990 Epoch 48/100 156/156 - 145s - loss: 0.2354 - acc: 0.9000 - val_loss: 0.3256 - val_acc: 0.8464 Epoch 49/100 156/156 - 146s - loss: 0.2374 - acc: 0.8994 - val_loss: 0.2516 - val_acc: 0.8932 Epoch 50/100 156/156 - 145s - loss: 0.2335 - acc: 0.8997 - val_loss: 0.2138 - val_acc: 0.9127 Epoch 51/100 156/156 - 145s - loss: 0.2283 - acc: 0.9040 - val_loss: 0.2800 - val_acc: 0.9087 Epoch 52/100 156/156 - 146s - loss: 0.2367 - acc: 0.8992 - val_loss: 0.2279 - val_acc: 0.9113 Epoch 53/100 156/156 - 146s - loss: 0.2316 - acc: 0.9008 - val_loss: 0.2157 - val_acc: 0.9123 Epoch 54/100 156/156 - 146s - loss: 0.2236 - acc: 0.9059 - val_loss: 0.2653 - val_acc: 0.8946 Epoch 55/100 156/156 - 145s - loss: 0.2348 - acc: 0.8993 - val_loss: 0.2310 - val_acc: 0.9077 Epoch 56/100 156/156 - 144s - loss: 0.2383 - acc: 0.8982 - val_loss: 0.3550 - val_acc: 0.8544 Epoch 57/100 156/156 - 147s - loss: 0.2292 - acc: 0.9028 - val_loss: 0.2356 - val_acc: 0.9095 Epoch 58/100 156/156 - 146s - loss: 0.2315 - acc: 0.9042 - val_loss: 0.2088 - val_acc: 0.9163 Epoch 59/100 156/156 - 146s - loss: 0.2256 - acc: 0.9037 - val_loss: 0.4032 - val_acc: 0.8149 Epoch 60/100 156/156 - 145s - loss: 0.2302 - acc: 0.9024 - val_loss: 0.2611 - val_acc: 0.8918 Epoch 61/100 156/156 - 144s - loss: 0.2227 - acc: 0.9048 - val_loss: 0.2068 - val_acc: 0.9103 Epoch 62/100 156/156 - 145s - loss: 0.2353 - acc: 0.8988 - val_loss: 0.2446 - val_acc: 0.8862 Epoch 63/100 156/156 - 146s - loss: 0.2313 - acc: 0.9008 - val_loss: 0.2295 - val_acc: 0.9119 Epoch 64/100 156/156 - 144s - loss: 0.2237 - acc: 0.9042 - val_loss: 0.2193 - val_acc: 0.9179 Epoch 65/100 156/156 - 145s - loss: 0.2292 - acc: 0.9055 - val_loss: 0.2427 - val_acc: 0.8960 Epoch 66/100 156/156 - 145s - loss: 0.2353 - acc: 0.9015 - val_loss: 0.4461 - val_acc: 0.8035 Epoch 67/100 156/156 - 145s - loss: 0.2205 - acc: 0.9074 - val_loss: 0.2056 - val_acc: 0.9107 Epoch 68/100 156/156 - 145s - loss: 0.2257 - acc: 0.9035 - val_loss: 0.2528 - val_acc: 0.8846 Epoch 69/100 156/156 - 146s - loss: 0.2275 - acc: 0.9053 - val_loss: 0.2302 - val_acc: 0.9034 Epoch 70/100 156/156 - 144s - loss: 0.2261 - acc: 0.9032 - val_loss: 0.2248 - val_acc: 0.9107 Epoch 71/100 156/156 - 145s - loss: 0.2280 - acc: 0.9005 - val_loss: 0.2289 - val_acc: 0.9060 Epoch 72/100 156/156 - 145s - loss: 0.2276 - acc: 0.9070 - val_loss: 0.2604 - val_acc: 0.8960 Epoch 73/100 156/156 - 145s - loss: 0.2312 - acc: 0.9032 - val_loss: 0.2774 - val_acc: 0.8874 Epoch 74/100 156/156 - 145s - loss: 0.2249 - acc: 0.9062 - val_loss: 0.2025 - val_acc: 0.9173 Epoch 75/100 156/156 - 145s - loss: 0.2332 - acc: 0.9044 - val_loss: 0.2055 - val_acc: 0.9151 Epoch 76/100 156/156 - 145s - loss: 0.2238 - acc: 0.9065 - val_loss: 0.2933 - val_acc: 0.8954 Epoch 77/100 156/156 - 145s - loss: 0.2312 - acc: 0.9021 - val_loss: 0.2556 - val_acc: 0.8940 Epoch 78/100 156/156 - 145s - loss: 0.2186 - acc: 0.9070 - val_loss: 0.1914 - val_acc: 0.9217 Epoch 79/100 156/156 - 146s - loss: 0.2315 - acc: 0.9055 - val_loss: 0.2192 - val_acc: 0.9087 Epoch 80/100 156/156 - 145s - loss: 0.2264 - acc: 0.9039 - val_loss: 0.2494 - val_acc: 0.8862 Epoch 81/100 156/156 - 145s - loss: 0.2209 - acc: 0.9086 - val_loss: 0.2026 - val_acc: 0.9185 Epoch 82/100 156/156 - 145s - loss: 0.2356 - acc: 0.9026 - val_loss: 0.2011 - val_acc: 0.9163 Epoch 83/100 156/156 - 144s - loss: 0.2236 - acc: 0.9059 - val_loss: 0.1996 - val_acc: 0.9229 Epoch 84/100 156/156 - 145s - loss: 0.2277 - acc: 0.9023 - val_loss: 0.2071 - val_acc: 0.9157 Epoch 85/100 156/156 - 145s - loss: 0.2365 - acc: 0.9053 - val_loss: 0.2343 - val_acc: 0.9115 Epoch 86/100 156/156 - 145s - loss: 0.2237 - acc: 0.9026 - val_loss: 0.2111 - val_acc: 0.9187 Epoch 87/100 156/156 - 144s - loss: 0.2285 - acc: 0.9022 - val_loss: 0.1987 - val_acc: 0.9167 Epoch 88/100 156/156 - 146s - loss: 0.2390 - acc: 0.8996 - val_loss: 0.2169 - val_acc: 0.9069 Epoch 89/100 156/156 - 144s - loss: 0.2478 - acc: 0.9030 - val_loss: 0.2021 - val_acc: 0.9195 Epoch 90/100 156/156 - 146s - loss: 0.2221 - acc: 0.9037 - val_loss: 0.3306 - val_acc: 0.8758 Epoch 91/100 156/156 - 146s - loss: 0.2342 - acc: 0.9005 - val_loss: 0.3883 - val_acc: 0.8183 Epoch 92/100 156/156 - 145s - loss: 0.2286 - acc: 0.9052 - val_loss: 0.2461 - val_acc: 0.8920 Epoch 93/100 156/156 - 146s - loss: 0.2260 - acc: 0.9060 - val_loss: 0.2327 - val_acc: 0.9155 Epoch 94/100 156/156 - 144s - loss: 0.2391 - acc: 0.9017 - val_loss: 0.2106 - val_acc: 0.9151 Epoch 95/100 156/156 - 145s - loss: 0.2320 - acc: 0.9008 - val_loss: 0.2006 - val_acc: 0.9217 Epoch 96/100 156/156 - 146s - loss: 0.2305 - acc: 0.9044 - val_loss: 0.2095 - val_acc: 0.9129 Epoch 97/100 156/156 - 145s - loss: 0.2312 - acc: 0.9029 - val_loss: 0.2112 - val_acc: 0.9123 Epoch 98/100 156/156 - 145s - loss: 0.2214 - acc: 0.9055 - val_loss: 0.1963 - val_acc: 0.9203 Epoch 99/100 156/156 - 144s - loss: 0.2364 - acc: 0.9029 - val_loss: 0.2055 - val_acc: 0.9085 Epoch 100/100 156/156 - 146s - loss: 0.2198 - acc: 0.9074 - val_loss: 0.2404 - val_acc: 0.9006 2019-07-20 22:27:48,713 graeae.timers.timer end: Ended: 2019-07-20 22:27:48.713962 I0720 22:27:48.713997 139935525873472 timer.py:77] Ended: 2019-07-20 22:27:48.713962 2019-07-20 22:27:48,714 graeae.timers.timer end: Elapsed: 4:05:53.421045 I0720 22:27:48.714990 139935525873472 timer.py:78] Elapsed: 4:05:53.421045
So, two things to notice. One is that it did even better than I thought it would, the other is that it didn't stop. The fact that it didn't stop is beacuse I was assiging Stop.on_epoch_end to Stop.on_epoch_end instead of assigning Stop.on_end_handler (so Stop.on_end_handler was never called). Hopefully I fixed it.
Take Fifteen
I originally set this up with a batch-size of 256 and no callbacks and then had to kill it because it hadn't stopped by the next morning when I had to go to work. That's probably something to be dealt with when training a real model, but in this case I'm just doing an exercise, so I'll try it with 256 and the have it time-out after 8 hours.
network = Network(str(training_path),
convolution_layers=5,
set_steps = True,
epochs = 100,
batch_size=256)
print(str(network))
with TIMER:
network.train()
2019-07-22 22:29:32,865 graeae.timers.timer start: Started: 2019-07-22 22:29:32.865426 I0722 22:29:32.865622 140052197230400 timer.py:70] Started: 2019-07-22 22:29:32.865426 (Network) - Path: /home/athena/data/datasets/images/dogs-vs-cats/train Epochs: 100 Batch Size: 256 Callbacks: None Data: (Data) - Path: /home/athena/data/datasets/images/dogs-vs-cats/train, Validation Split: 0.2,Batch Size: 256 Callbacks: None Found 20000 images belonging to 2 classes. Found 5000 images belonging to 2 classes. W0722 22:29:33.512133 140052197230400 deprecation.py:506] From /home/athena/.virtualenvs/In-Too-Deep/lib/python3.7/site-packages/tensorflow/python/ops/init_ops.py:1251: calling VarianceScaling.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version. Instructions for updating: Call initializer instance with the dtype argument instead of passing it to the constructor W0722 22:29:33.928766 140052197230400 deprecation.py:323] From /home/athena/.virtualenvs/In-Too-Deep/lib/python3.7/site-packages/tensorflow/python/ops/nn_impl.py:180: add_dispatch_support.<locals>.wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version. Instructions for updating: Use tf.where in 2.0, which has the same broadcast rule as np.where Epoch 1/100 78/78 - 429s - loss: 0.6963 - acc: 0.5277 - val_loss: 0.6838 - val_acc: 0.6053 Epoch 2/100 78/78 - 152s - loss: 0.6823 - acc: 0.5664 - val_loss: 0.7476 - val_acc: 0.5107 Epoch 3/100 78/78 - 152s - loss: 0.6827 - acc: 0.6109 - val_loss: 0.6231 - val_acc: 0.6567 Epoch 4/100 78/78 - 154s - loss: 0.6475 - acc: 0.6241 - val_loss: 0.6528 - val_acc: 0.6182 Epoch 5/100 78/78 - 155s - loss: 0.6304 - acc: 0.6465 - val_loss: 0.5851 - val_acc: 0.6846 Epoch 6/100 78/78 - 155s - loss: 0.6161 - acc: 0.6607 - val_loss: 0.5997 - val_acc: 0.6780 Epoch 7/100 78/78 - 149s - loss: 0.5978 - acc: 0.6828 - val_loss: 0.5866 - val_acc: 0.6748 Epoch 8/100 78/78 - 150s - loss: 0.5883 - acc: 0.6894 - val_loss: 0.5663 - val_acc: 0.7079 Epoch 9/100 78/78 - 149s - loss: 0.5803 - acc: 0.6977 - val_loss: 0.6254 - val_acc: 0.6332 Epoch 10/100 78/78 - 147s - loss: 0.5617 - acc: 0.7119 - val_loss: 0.5867 - val_acc: 0.6865 Epoch 11/100 78/78 - 146s - loss: 0.5559 - acc: 0.7217 - val_loss: 0.5812 - val_acc: 0.6959 Epoch 12/100 78/78 - 141s - loss: 0.5521 - acc: 0.7208 - val_loss: 0.5157 - val_acc: 0.7605 Epoch 13/100 78/78 - 143s - loss: 0.5440 - acc: 0.7283 - val_loss: 0.5218 - val_acc: 0.7541 Epoch 14/100 78/78 - 143s - loss: 0.5225 - acc: 0.7407 - val_loss: 0.4998 - val_acc: 0.7593 Epoch 15/100 78/78 - 142s - loss: 0.5044 - acc: 0.7538 - val_loss: 0.4718 - val_acc: 0.7695 Epoch 16/100 78/78 - 141s - loss: 0.4946 - acc: 0.7607 - val_loss: 0.5743 - val_acc: 0.6912 Epoch 17/100 78/78 - 141s - loss: 0.4643 - acc: 0.7780 - val_loss: 0.4124 - val_acc: 0.8156 Epoch 18/100 78/78 - 142s - loss: 0.4480 - acc: 0.7894 - val_loss: 0.4762 - val_acc: 0.7593 Epoch 19/100 78/78 - 143s - loss: 0.4426 - acc: 0.7880 - val_loss: 0.4288 - val_acc: 0.8074 Epoch 20/100 78/78 - 141s - loss: 0.4271 - acc: 0.8006 - val_loss: 0.3893 - val_acc: 0.8248 Epoch 21/100 78/78 - 142s - loss: 0.4111 - acc: 0.8073 - val_loss: 0.3602 - val_acc: 0.8355 Epoch 22/100 78/78 - 144s - loss: 0.4066 - acc: 0.8094 - val_loss: 0.3546 - val_acc: 0.8403 Epoch 23/100 78/78 - 143s - loss: 0.3927 - acc: 0.8160 - val_loss: 0.3536 - val_acc: 0.8368 Epoch 24/100 78/78 - 138s - loss: 0.3839 - acc: 0.8228 - val_loss: 0.3409 - val_acc: 0.8481 Epoch 25/100 78/78 - 141s - loss: 0.3704 - acc: 0.8290 - val_loss: 0.3364 - val_acc: 0.8514 Epoch 26/100 78/78 - 141s - loss: 0.3633 - acc: 0.8362 - val_loss: 0.3329 - val_acc: 0.8577 Epoch 27/100 78/78 - 142s - loss: 0.3434 - acc: 0.8447 - val_loss: 0.3131 - val_acc: 0.8670 Epoch 28/100 78/78 - 142s - loss: 0.3481 - acc: 0.8401 - val_loss: 0.3515 - val_acc: 0.8433 Epoch 29/100 78/78 - 142s - loss: 0.3225 - acc: 0.8555 - val_loss: 0.4498 - val_acc: 0.8049 Epoch 30/100 78/78 - 142s - loss: 0.3280 - acc: 0.8482 - val_loss: 0.3338 - val_acc: 0.8475 Epoch 31/100 78/78 - 144s - loss: 0.3252 - acc: 0.8534 - val_loss: 0.3694 - val_acc: 0.8339 Epoch 32/100 78/78 - 142s - loss: 0.3232 - acc: 0.8524 - val_loss: 0.3124 - val_acc: 0.8571 Epoch 33/100 78/78 - 142s - loss: 0.3011 - acc: 0.8619 - val_loss: 0.3094 - val_acc: 0.8657 Epoch 34/100 78/78 - 143s - loss: 0.2908 - acc: 0.8702 - val_loss: 0.3324 - val_acc: 0.8540 Epoch 35/100 78/78 - 140s - loss: 0.2943 - acc: 0.8697 - val_loss: 0.2880 - val_acc: 0.8717 Epoch 36/100 78/78 - 142s - loss: 0.2848 - acc: 0.8707 - val_loss: 0.2671 - val_acc: 0.8894 Epoch 37/100 78/78 - 143s - loss: 0.2783 - acc: 0.8774 - val_loss: 0.2870 - val_acc: 0.8756 Epoch 38/100 78/78 - 141s - loss: 0.2876 - acc: 0.8750 - val_loss: 0.2666 - val_acc: 0.8855 Epoch 39/100 78/78 - 139s - loss: 0.2729 - acc: 0.8790 - val_loss: 0.2557 - val_acc: 0.8914 Epoch 40/100 78/78 - 137s - loss: 0.2667 - acc: 0.8809 - val_loss: 0.2424 - val_acc: 0.8929 Epoch 41/100 78/78 - 137s - loss: 0.2626 - acc: 0.8824 - val_loss: 0.2746 - val_acc: 0.8820 Epoch 42/100 78/78 - 135s - loss: 0.2515 - acc: 0.8925 - val_loss: 0.2525 - val_acc: 0.8912 Epoch 43/100 78/78 - 137s - loss: 0.2487 - acc: 0.8905 - val_loss: 0.2516 - val_acc: 0.8956 Epoch 44/100 78/78 - 137s - loss: 0.2544 - acc: 0.8856 - val_loss: 0.2444 - val_acc: 0.8914 Epoch 45/100 78/78 - 138s - loss: 0.2484 - acc: 0.8899 - val_loss: 0.2727 - val_acc: 0.8785 Epoch 46/100 78/78 - 135s - loss: 0.2447 - acc: 0.8919 - val_loss: 0.2385 - val_acc: 0.9032 Epoch 47/100 78/78 - 138s - loss: 0.2349 - acc: 0.8973 - val_loss: 0.2296 - val_acc: 0.8999 Epoch 48/100 78/78 - 137s - loss: 0.2324 - acc: 0.8966 - val_loss: 0.2772 - val_acc: 0.8843 Epoch 49/100 78/78 - 136s - loss: 0.2383 - acc: 0.8948 - val_loss: 0.2698 - val_acc: 0.8820 Epoch 50/100 78/78 - 138s - loss: 0.2293 - acc: 0.9000 - val_loss: 0.2763 - val_acc: 0.8789 Epoch 51/100 78/78 - 137s - loss: 0.2300 - acc: 0.9006 - val_loss: 0.2231 - val_acc: 0.9060 Epoch 52/100 78/78 - 135s - loss: 0.2236 - acc: 0.9029 - val_loss: 0.2321 - val_acc: 0.9021 Epoch 53/100 78/78 - 137s - loss: 0.2256 - acc: 0.9012 - val_loss: 0.2229 - val_acc: 0.9075 Epoch 54/100 78/78 - 138s - loss: 0.2147 - acc: 0.9079 - val_loss: 0.2330 - val_acc: 0.9040 Epoch 55/100 78/78 - 137s - loss: 0.2192 - acc: 0.9066 - val_loss: 0.2390 - val_acc: 0.9040 Epoch 56/100 78/78 - 138s - loss: 0.2138 - acc: 0.9068 - val_loss: 0.2234 - val_acc: 0.9077 Epoch 57/100 78/78 - 137s - loss: 0.2115 - acc: 0.9078 - val_loss: 0.2734 - val_acc: 0.8810 Epoch 58/100 78/78 - 136s - loss: 0.2312 - acc: 0.8987 - val_loss: 0.2320 - val_acc: 0.9038 Epoch 59/100 78/78 - 138s - loss: 0.2054 - acc: 0.9129 - val_loss: 0.2188 - val_acc: 0.9120 Epoch 60/100 78/78 - 137s - loss: 0.2215 - acc: 0.9047 - val_loss: 0.2329 - val_acc: 0.9019 Epoch 61/100 78/78 - 137s - loss: 0.2088 - acc: 0.9087 - val_loss: 0.2357 - val_acc: 0.8937 Epoch 62/100 78/78 - 135s - loss: 0.2052 - acc: 0.9113 - val_loss: 0.2478 - val_acc: 0.8890 Epoch 63/100 78/78 - 138s - loss: 0.2013 - acc: 0.9138 - val_loss: 0.2079 - val_acc: 0.9134 Epoch 64/100 78/78 - 137s - loss: 0.2042 - acc: 0.9135 - val_loss: 0.2106 - val_acc: 0.9141 Epoch 65/100 78/78 - 137s - loss: 0.2036 - acc: 0.9122 - val_loss: 0.2266 - val_acc: 0.9054 Epoch 66/100 78/78 - 137s - loss: 0.1954 - acc: 0.9158 - val_loss: 0.2151 - val_acc: 0.9087 Epoch 67/100 78/78 - 137s - loss: 0.2013 - acc: 0.9125 - val_loss: 0.2062 - val_acc: 0.9116 Epoch 68/100 78/78 - 137s - loss: 0.1969 - acc: 0.9140 - val_loss: 0.2053 - val_acc: 0.9122 Epoch 69/100 78/78 - 137s - loss: 0.1944 - acc: 0.9152 - val_loss: 0.2159 - val_acc: 0.9112 Epoch 70/100 78/78 - 137s - loss: 0.1899 - acc: 0.9172 - val_loss: 0.2509 - val_acc: 0.8927 Epoch 71/100 78/78 - 137s - loss: 0.1966 - acc: 0.9187 - val_loss: 0.2110 - val_acc: 0.9128 Epoch 72/100 78/78 - 137s - loss: 0.1847 - acc: 0.9215 - val_loss: 0.2299 - val_acc: 0.9093 Epoch 73/100 78/78 - 137s - loss: 0.1838 - acc: 0.9227 - val_loss: 0.1841 - val_acc: 0.9278 Epoch 74/100 78/78 - 137s - loss: 0.1966 - acc: 0.9176 - val_loss: 0.2924 - val_acc: 0.8742 Epoch 75/100 78/78 - 138s - loss: 0.1842 - acc: 0.9220 - val_loss: 0.1932 - val_acc: 0.9223 Epoch 76/100 78/78 - 137s - loss: 0.1844 - acc: 0.9210 - val_loss: 0.2169 - val_acc: 0.9198 Epoch 77/100 78/78 - 137s - loss: 0.1906 - acc: 0.9219 - val_loss: 0.2597 - val_acc: 0.8818 Epoch 78/100 78/78 - 137s - loss: 0.1735 - acc: 0.9255 - val_loss: 0.1994 - val_acc: 0.9204 Epoch 79/100 78/78 - 137s - loss: 0.1826 - acc: 0.9230 - val_loss: 0.1862 - val_acc: 0.9245 Epoch 80/100 78/78 - 137s - loss: 0.1847 - acc: 0.9239 - val_loss: 0.2384 - val_acc: 0.9122 Epoch 81/100 78/78 - 152s - loss: 0.1843 - acc: 0.9226 - val_loss: 0.2039 - val_acc: 0.9151 Epoch 82/100 78/78 - 151s - loss: 0.1778 - acc: 0.9225 - val_loss: 0.2089 - val_acc: 0.9192 Epoch 83/100 78/78 - 150s - loss: 0.1765 - acc: 0.9225 - val_loss: 0.1998 - val_acc: 0.9134 Epoch 84/100 78/78 - 148s - loss: 0.1780 - acc: 0.9238 - val_loss: 0.1837 - val_acc: 0.9266 Epoch 85/100 78/78 - 147s - loss: 0.1739 - acc: 0.9259 - val_loss: 0.2011 - val_acc: 0.9204 Epoch 86/100 78/78 - 146s - loss: 0.1718 - acc: 0.9271 - val_loss: 0.1885 - val_acc: 0.9307 Epoch 87/100 78/78 - 144s - loss: 0.1775 - acc: 0.9242 - val_loss: 0.2126 - val_acc: 0.9213 Epoch 88/100 78/78 - 142s - loss: 0.1732 - acc: 0.9265 - val_loss: 0.1929 - val_acc: 0.9204 Epoch 89/100 78/78 - 141s - loss: 0.1773 - acc: 0.9242 - val_loss: 0.2050 - val_acc: 0.9243 Epoch 90/100 78/78 - 140s - loss: 0.1745 - acc: 0.9278 - val_loss: 0.2168 - val_acc: 0.9122 Epoch 91/100 78/78 - 139s - loss: 0.1802 - acc: 0.9214 - val_loss: 0.2263 - val_acc: 0.9079 Epoch 92/100 78/78 - 139s - loss: 0.1642 - acc: 0.9320 - val_loss: 0.2503 - val_acc: 0.9087 Epoch 93/100 78/78 - 136s - loss: 0.1705 - acc: 0.9285 - val_loss: 0.2184 - val_acc: 0.9126 Epoch 94/100 78/78 - 136s - loss: 0.1714 - acc: 0.9253 - val_loss: 0.2365 - val_acc: 0.9130 Epoch 95/100 78/78 - 136s - loss: 0.1678 - acc: 0.9305 - val_loss: 0.1882 - val_acc: 0.9231 Epoch 96/100 78/78 - 137s - loss: 0.1923 - acc: 0.9212 - val_loss: 0.1931 - val_acc: 0.9211 Epoch 97/100 78/78 - 136s - loss: 0.1574 - acc: 0.9335 - val_loss: 0.2011 - val_acc: 0.9215 Epoch 98/100 78/78 - 136s - loss: 0.1673 - acc: 0.9280 - val_loss: 0.2561 - val_acc: 0.9032 Epoch 99/100 78/78 - 137s - loss: 0.1670 - acc: 0.9304 - val_loss: 0.2018 - val_acc: 0.9180 Epoch 100/100 78/78 - 137s - loss: 0.1663 - acc: 0.9292 - val_loss: 0.2245 - val_acc: 0.9065 2019-07-23 02:29:24,212 graeae.timers.timer end: Ended: 2019-07-23 02:29:24.212763 I0723 02:29:24.212822 140052197230400 timer.py:77] Ended: 2019-07-23 02:29:24.212763 2019-07-23 02:29:24,214 graeae.timers.timer end: Elapsed: 3:59:51.347337 I0723 02:29:24.214272 140052197230400 timer.py:78] Elapsed: 3:59:51.347337
print(f"About {(4 * 60)/100} minutes per epoch")
About 2.4 minutes per epoch
So with a batch size of 256 it doesn't take an unreasonable amount of time to train.
data = pandas.read_csv("~/cats_vs_dogs_5.csv")
curves = [holoviews.Curve(data, ("index", "Epoch"), "training_loss",
label="Training Loss",),
holoviews.Curve(data, ("index", "Epoch"), "training_accuracy",
label="Training Accuracy").opts(tools=["hover"]),
holoviews.Curve(data, ("index", "Epoch"), "validation_loss",
label="Validation Loss",).opts(tools=["hover"]),
holoviews.Curve(data, ("index", "Epoch"), "validation_accuracy",
label="Validation Accuracy").opts(tools=["hover"]),
]
plot = holoviews.Overlay(curves).opts(tools=["hover"], height=800, width=1000,
ylabel="Performance",
title="Training vs Validation")
Embed(plot=plot, file_name="training_validation_loss_5")()
best_validation(data)
Best Accuracy: 0.9307 (loss=0.1885) Epoch: 85 Best Loss: 0.1837 (accuracy=0.9266, Epoch: 83)
The model is doing pretty good already. Unfortunately I didn't save the model before turning off my machine…
Take Sixteen
network = Network(str(training_path),
convolution_layers=5,
set_steps = True,
epochs = 100,
batch_size=256)
print(str(network))
with TIMER:
network.train()
save_model(network.model, MODELS/"layers_5_batch_size_256.h5")
2019-07-27 11:17:39,023 graeae.timers.timer start: Started: 2019-07-27 11:17:39.023595 I0727 11:17:39.023817 140052539918144 timer.py:70] Started: 2019-07-27 11:17:39.023595 (Network) - Path: /home/athena/data/datasets/images/dogs-vs-cats/train Epochs: 100 Batch Size: 256 Callbacks: None Data: (Data) - Path: /home/athena/data/datasets/images/dogs-vs-cats/train, Validation Split: 0.2,Batch Size: 256 Callbacks: None Found 20000 images belonging to 2 classes. W0727 11:17:39.640755 140052539918144 deprecation.py:506] From /home/athena/.virtualenvs/In-Too-Deep/lib/python3.7/site-packages/tensorflow/python/ops/init_ops.py:1251: calling VarianceScaling.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version. Instructions for updating: Call initializer instance with the dtype argument instead of passing it to the constructor Found 5000 images belonging to 2 classes. W0727 11:17:39.796763 140052539918144 deprecation.py:323] From /home/athena/.virtualenvs/In-Too-Deep/lib/python3.7/site-packages/tensorflow/python/ops/nn_impl.py:180: add_dispatch_support.<locals>.wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version. Instructions for updating: Use tf.where in 2.0, which has the same broadcast rule as np.where Epoch 1/100 78/78 - 368s - loss: 0.6996 - acc: 0.5326 - val_loss: 0.6777 - val_acc: 0.5983 Epoch 2/100 78/78 - 235s - loss: 0.6823 - acc: 0.5713 - val_loss: 0.6554 - val_acc: 0.6205 Epoch 3/100 78/78 - 229s - loss: 0.6718 - acc: 0.5961 - val_loss: 0.6645 - val_acc: 0.5913 Epoch 4/100 78/78 - 229s - loss: 0.6458 - acc: 0.6334 - val_loss: 0.5993 - val_acc: 0.6822 Epoch 5/100 78/78 - 227s - loss: 0.6395 - acc: 0.6479 - val_loss: 0.5940 - val_acc: 0.6984 Epoch 6/100 78/78 - 234s - loss: 0.6139 - acc: 0.6639 - val_loss: 0.6270 - val_acc: 0.6340 Epoch 7/100 78/78 - 228s - loss: 0.6043 - acc: 0.6742 - val_loss: 0.5533 - val_acc: 0.7231 Epoch 8/100 78/78 - 228s - loss: 0.5862 - acc: 0.6912 - val_loss: 0.5496 - val_acc: 0.7243 Epoch 9/100 78/78 - 227s - loss: 0.5698 - acc: 0.7089 - val_loss: 0.5726 - val_acc: 0.7007 Epoch 10/100 78/78 - 230s - loss: 0.5673 - acc: 0.7010 - val_loss: 0.5106 - val_acc: 0.7432 Epoch 11/100 78/78 - 226s - loss: 0.5416 - acc: 0.7245 - val_loss: 0.5233 - val_acc: 0.7387 Epoch 12/100 78/78 - 230s - loss: 0.5411 - acc: 0.7306 - val_loss: 0.5489 - val_acc: 0.7132 Epoch 13/100 78/78 - 230s - loss: 0.5301 - acc: 0.7332 - val_loss: 0.5358 - val_acc: 0.7163 Epoch 14/100 78/78 - 225s - loss: 0.5200 - acc: 0.7432 - val_loss: 0.4981 - val_acc: 0.7463 Epoch 15/100 78/78 - 225s - loss: 0.5004 - acc: 0.7521 - val_loss: 0.4841 - val_acc: 0.7718 Epoch 16/100 78/78 - 226s - loss: 0.4934 - acc: 0.7590 - val_loss: 0.4679 - val_acc: 0.7640 Epoch 17/100 78/78 - 223s - loss: 0.4709 - acc: 0.7733 - val_loss: 0.6309 - val_acc: 0.7130 Epoch 18/100 78/78 - 224s - loss: 0.4722 - acc: 0.7707 - val_loss: 0.4204 - val_acc: 0.7983 Epoch 19/100 78/78 - 226s - loss: 0.4553 - acc: 0.7832 - val_loss: 0.4041 - val_acc: 0.8129 Epoch 20/100 78/78 - 222s - loss: 0.4325 - acc: 0.7968 - val_loss: 0.3785 - val_acc: 0.8326 Epoch 21/100 78/78 - 226s - loss: 0.4227 - acc: 0.8004 - val_loss: 0.5409 - val_acc: 0.7358 Epoch 22/100 78/78 - 226s - loss: 0.4090 - acc: 0.8063 - val_loss: 0.3555 - val_acc: 0.8438 Epoch 23/100 78/78 - 228s - loss: 0.3903 - acc: 0.8179 - val_loss: 0.3571 - val_acc: 0.8429 Epoch 24/100 78/78 - 226s - loss: 0.3745 - acc: 0.8307 - val_loss: 0.3516 - val_acc: 0.8421 Epoch 25/100 78/78 - 227s - loss: 0.3728 - acc: 0.8304 - val_loss: 0.3703 - val_acc: 0.8285 Epoch 26/100 78/78 - 228s - loss: 0.3584 - acc: 0.8344 - val_loss: 0.3641 - val_acc: 0.8392 Epoch 27/100 78/78 - 227s - loss: 0.3481 - acc: 0.8400 - val_loss: 0.3430 - val_acc: 0.8386 Epoch 28/100 78/78 - 228s - loss: 0.3265 - acc: 0.8497 - val_loss: 0.3399 - val_acc: 0.8470 Epoch 29/100 78/78 - 227s - loss: 0.3356 - acc: 0.8459 - val_loss: 0.3095 - val_acc: 0.8629 Epoch 30/100 78/78 - 226s - loss: 0.3167 - acc: 0.8592 - val_loss: 0.3169 - val_acc: 0.8536 Epoch 31/100 78/78 - 227s - loss: 0.3155 - acc: 0.8599 - val_loss: 0.2829 - val_acc: 0.8820 Epoch 32/100 78/78 - 235s - loss: 0.2984 - acc: 0.8664 - val_loss: 0.2902 - val_acc: 0.8688 Epoch 33/100 78/78 - 236s - loss: 0.3002 - acc: 0.8642 - val_loss: 0.3434 - val_acc: 0.8586 Epoch 34/100 78/78 - 238s - loss: 0.3082 - acc: 0.8629 - val_loss: 0.3796 - val_acc: 0.8271 Epoch 35/100 78/78 - 236s - loss: 0.2828 - acc: 0.8756 - val_loss: 0.2850 - val_acc: 0.8719 Epoch 36/100 78/78 - 235s - loss: 0.2877 - acc: 0.8716 - val_loss: 0.2975 - val_acc: 0.8664 Epoch 37/100 78/78 - 238s - loss: 0.2839 - acc: 0.8723 - val_loss: 0.2544 - val_acc: 0.8910 Epoch 38/100 78/78 - 230s - loss: 0.2749 - acc: 0.8801 - val_loss: 0.2558 - val_acc: 0.8845 Epoch 39/100 78/78 - 228s - loss: 0.2603 - acc: 0.8852 - val_loss: 0.2621 - val_acc: 0.8797 Epoch 40/100 78/78 - 258s - loss: 0.2664 - acc: 0.8841 - val_loss: 0.2513 - val_acc: 0.8937 Epoch 41/100 78/78 - 255s - loss: 0.2548 - acc: 0.8865 - val_loss: 0.2523 - val_acc: 0.8929 Epoch 42/100 78/78 - 255s - loss: 0.2561 - acc: 0.8866 - val_loss: 0.2484 - val_acc: 0.8962 Epoch 43/100 78/78 - 247s - loss: 0.2575 - acc: 0.8870 - val_loss: 0.2452 - val_acc: 0.8960 Epoch 44/100 78/78 - 247s - loss: 0.2562 - acc: 0.8882 - val_loss: 0.3566 - val_acc: 0.8390 Epoch 45/100 78/78 - 251s - loss: 0.2493 - acc: 0.8903 - val_loss: 0.2950 - val_acc: 0.8731 Epoch 46/100 78/78 - 262s - loss: 0.2386 - acc: 0.8957 - val_loss: 0.2778 - val_acc: 0.8806 Epoch 47/100 78/78 - 226s - loss: 0.2519 - acc: 0.8948 - val_loss: 0.2276 - val_acc: 0.9046 Epoch 48/100 78/78 - 234s - loss: 0.2386 - acc: 0.8963 - val_loss: 0.3851 - val_acc: 0.8037 Epoch 49/100 78/78 - 271s - loss: 0.2299 - acc: 0.8997 - val_loss: 0.2298 - val_acc: 0.9015 Epoch 50/100 78/78 - 242s - loss: 0.2283 - acc: 0.9016 - val_loss: 0.2885 - val_acc: 0.8748 Epoch 51/100 78/78 - 234s - loss: 0.2226 - acc: 0.9026 - val_loss: 0.2814 - val_acc: 0.8771 Epoch 52/100 78/78 - 263s - loss: 0.2303 - acc: 0.9002 - val_loss: 0.2829 - val_acc: 0.8781 Epoch 53/100 78/78 - 253s - loss: 0.2259 - acc: 0.9031 - val_loss: 0.2312 - val_acc: 0.9042 Epoch 54/100 78/78 - 241s - loss: 0.2177 - acc: 0.9052 - val_loss: 0.2423 - val_acc: 0.8986 Epoch 55/100 78/78 - 251s - loss: 0.2285 - acc: 0.9001 - val_loss: 0.2126 - val_acc: 0.9124 Epoch 56/100 78/78 - 257s - loss: 0.2200 - acc: 0.9040 - val_loss: 0.2224 - val_acc: 0.9062 Epoch 57/100 78/78 - 249s - loss: 0.2151 - acc: 0.9066 - val_loss: 0.2279 - val_acc: 0.9067 Epoch 58/100 78/78 - 256s - loss: 0.2106 - acc: 0.9087 - val_loss: 0.2813 - val_acc: 0.8851 Epoch 59/100 78/78 - 255s - loss: 0.2091 - acc: 0.9098 - val_loss: 0.2058 - val_acc: 0.9128 Epoch 60/100 78/78 - 258s - loss: 0.2108 - acc: 0.9108 - val_loss: 0.2587 - val_acc: 0.8949 Epoch 61/100 78/78 - 257s - loss: 0.2065 - acc: 0.9126 - val_loss: 0.2341 - val_acc: 0.8972 Epoch 62/100 78/78 - 259s - loss: 0.2032 - acc: 0.9121 - val_loss: 0.1973 - val_acc: 0.9137 Epoch 63/100 78/78 - 257s - loss: 0.2032 - acc: 0.9127 - val_loss: 0.1991 - val_acc: 0.9190 Epoch 64/100 78/78 - 256s - loss: 0.2000 - acc: 0.9143 - val_loss: 0.2605 - val_acc: 0.8888 Epoch 65/100 78/78 - 258s - loss: 0.2008 - acc: 0.9140 - val_loss: 0.2061 - val_acc: 0.9186 Epoch 66/100 78/78 - 253s - loss: 0.1934 - acc: 0.9175 - val_loss: 0.1978 - val_acc: 0.9192 Epoch 67/100 78/78 - 237s - loss: 0.1996 - acc: 0.9138 - val_loss: 0.1912 - val_acc: 0.9186 Epoch 68/100 78/78 - 229s - loss: 0.1976 - acc: 0.9150 - val_loss: 0.2157 - val_acc: 0.9134 Epoch 69/100 78/78 - 236s - loss: 0.1909 - acc: 0.9179 - val_loss: 0.1942 - val_acc: 0.9204 Epoch 70/100 78/78 - 249s - loss: 0.1929 - acc: 0.9187 - val_loss: 0.2266 - val_acc: 0.9132 Epoch 71/100 78/78 - 265s - loss: 0.1883 - acc: 0.9174 - val_loss: 0.2101 - val_acc: 0.9126 Epoch 72/100 78/78 - 280s - loss: 0.1924 - acc: 0.9165 - val_loss: 0.2090 - val_acc: 0.9118 Epoch 73/100 78/78 - 284s - loss: 0.1932 - acc: 0.9177 - val_loss: 0.1864 - val_acc: 0.9250 Epoch 74/100 78/78 - 281s - loss: 0.1797 - acc: 0.9220 - val_loss: 0.2070 - val_acc: 0.9198 Epoch 75/100 78/78 - 261s - loss: 0.1843 - acc: 0.9212 - val_loss: 0.2201 - val_acc: 0.9052 Epoch 76/100 78/78 - 233s - loss: 0.1933 - acc: 0.9199 - val_loss: 0.1908 - val_acc: 0.9180 Epoch 77/100 78/78 - 232s - loss: 0.1858 - acc: 0.9214 - val_loss: 0.3415 - val_acc: 0.8300 Epoch 78/100 78/78 - 227s - loss: 0.1841 - acc: 0.9193 - val_loss: 0.2657 - val_acc: 0.9023 Epoch 79/100 78/78 - 226s - loss: 0.1886 - acc: 0.9203 - val_loss: 0.1911 - val_acc: 0.9268 Epoch 80/100 78/78 - 237s - loss: 0.1872 - acc: 0.9219 - val_loss: 0.1882 - val_acc: 0.9245 Epoch 81/100 78/78 - 260s - loss: 0.1839 - acc: 0.9220 - val_loss: 0.2338 - val_acc: 0.9040 Epoch 82/100 78/78 - 262s - loss: 0.1866 - acc: 0.9218 - val_loss: 0.2314 - val_acc: 0.8986 Epoch 83/100 78/78 - 261s - loss: 0.1761 - acc: 0.9257 - val_loss: 0.2218 - val_acc: 0.9038 Epoch 84/100 78/78 - 263s - loss: 0.1791 - acc: 0.9236 - val_loss: 0.3364 - val_acc: 0.8466 Epoch 85/100 78/78 - 262s - loss: 0.1814 - acc: 0.9217 - val_loss: 0.1917 - val_acc: 0.9229 Epoch 86/100 78/78 - 262s - loss: 0.1838 - acc: 0.9213 - val_loss: 0.1783 - val_acc: 0.9264 Epoch 87/100 78/78 - 264s - loss: 0.1790 - acc: 0.9269 - val_loss: 0.2196 - val_acc: 0.9110 Epoch 88/100 78/78 - 264s - loss: 0.1730 - acc: 0.9285 - val_loss: 0.1876 - val_acc: 0.9280 Epoch 89/100 78/78 - 230s - loss: 0.1782 - acc: 0.9227 - val_loss: 0.2387 - val_acc: 0.9046 Epoch 90/100 78/78 - 229s - loss: 0.1719 - acc: 0.9255 - val_loss: 0.2171 - val_acc: 0.9258 Epoch 91/100 78/78 - 232s - loss: 0.1743 - acc: 0.9279 - val_loss: 0.2246 - val_acc: 0.9134 Epoch 92/100 78/78 - 229s - loss: 0.1688 - acc: 0.9287 - val_loss: 0.2299 - val_acc: 0.9141 Epoch 93/100 78/78 - 227s - loss: 0.1763 - acc: 0.9224 - val_loss: 0.1850 - val_acc: 0.9264 Epoch 94/100 78/78 - 230s - loss: 0.1810 - acc: 0.9222 - val_loss: 0.1880 - val_acc: 0.9260 Epoch 95/100 78/78 - 229s - loss: 0.1788 - acc: 0.9243 - val_loss: 0.1884 - val_acc: 0.9141 Epoch 96/100 78/78 - 231s - loss: 0.1688 - acc: 0.9302 - val_loss: 0.2187 - val_acc: 0.9100 Epoch 97/100 78/78 - 228s - loss: 0.1721 - acc: 0.9265 - val_loss: 0.2140 - val_acc: 0.9102 Epoch 98/100 78/78 - 231s - loss: 0.1677 - acc: 0.9312 - val_loss: 0.1884 - val_acc: 0.9278 Epoch 99/100 78/78 - 228s - loss: 0.1691 - acc: 0.9275 - val_loss: 0.1891 - val_acc: 0.9229 Epoch 100/100 78/78 - 229s - loss: 0.1684 - acc: 0.9265 - val_loss: 0.1866 - val_acc: 0.9200 2019-07-27 18:00:56,707 graeae.timers.timer end: Ended: 2019-07-27 18:00:56.707395 I0727 18:00:56.707435 140052539918144 timer.py:77] Ended: 2019-07-27 18:00:56.707395 2019-07-27 18:00:56,708 graeae.timers.timer end: Elapsed: 6:43:17.683800 I0727 18:00:56.708784 140052539918144 timer.py:78] Elapsed: 6:43:17.683800 Saving the model to /home/athena/models/dogs-vs-cats/layers_5_batch_size_256.h5
data = pandas.read_csv("~/cats_vs_dogs_6.csv")
curves = [holoviews.Curve(data, ("index", "Epoch"), "training_loss",
label="Training Loss",),
holoviews.Curve(data, ("index", "Epoch"), "training_accuracy",
label="Training Accuracy").opts(tools=["hover"]),
holoviews.Curve(data, ("index", "Epoch"), "validation_loss",
label="Validation Loss",).opts(tools=["hover"]),
holoviews.Curve(data, ("index", "Epoch"), "validation_accuracy",
label="Validation Accuracy").opts(tools=["hover"]),
]
plot = holoviews.Overlay(curves).opts(tools=["hover"], height=800, width=1000,
ylabel="Performance",
title="Training vs Validation")
Embed(plot=plot, file_name="training_validation_loss_5")()
best_validation(data)
Best Accuracy: 0.928 (loss=0.1876) Epoch: 87 Best Loss: 0.1783 (accuracy=0.9264, Epoch: 85)
- Another Training Session
model_path = MODELS/"layers_5_batch_size_256.h5" network = Network(str(training_path), convolution_layers=5, set_steps = True, epochs = 100, batch_size=256) model = tensorflow.keras.models.load_model(str(model_path)) network._model = model with TIMER: network.train() network.model.save(str(model_path))
W0728 10:40:02.467012 140643102242624 deprecation.py:506] From /home/athena/.virtualenvs/In-Too-Deep/lib/python3.7/site-packages/tensorflow/python/ops/init_ops.py:97: calling GlorotUniform.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version. Instructions for updating: Call initializer instance with the dtype argument instead of passing it to the constructor W0728 10:40:02.469725 140643102242624 deprecation.py:506] From /home/athena/.virtualenvs/In-Too-Deep/lib/python3.7/site-packages/tensorflow/python/ops/init_ops.py:1251: calling VarianceScaling.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version. Instructions for updating: Call initializer instance with the dtype argument instead of passing it to the constructor W0728 10:40:02.471851 140643102242624 deprecation.py:506] From /home/athena/.virtualenvs/In-Too-Deep/lib/python3.7/site-packages/tensorflow/python/ops/init_ops.py:97: calling Zeros.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version. Instructions for updating: Call initializer instance with the dtype argument instead of passing it to the constructor W0728 10:40:16.260661 140643102242624 deprecation.py:323] From /home/athena/.virtualenvs/In-Too-Deep/lib/python3.7/site-packages/tensorflow/python/ops/nn_impl.py:180: add_dispatch_support.<locals>.wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version. Instructions for updating: Use tf.where in 2.0, which has the same broadcast rule as np.where 2019-07-28 10:40:16,840 graeae.timers.timer start: Started: 2019-07-28 10:40:16.840941 I0728 10:40:16.840974 140643102242624 timer.py:70] Started: 2019-07-28 10:40:16.840941 Found 20000 images belonging to 2 classes. Found 5000 images belonging to 2 classes. Epoch 1/100 78/78 - 422s - loss: 0.1641 - acc: 0.9317 - val_loss: 0.2265 - val_acc: 0.9157 Epoch 2/100 78/78 - 151s - loss: 0.1770 - acc: 0.9264 - val_loss: 0.1841 - val_acc: 0.9235 Epoch 3/100 78/78 - 149s - loss: 0.1696 - acc: 0.9282 - val_loss: 0.3041 - val_acc: 0.8935 Epoch 4/100 78/78 - 148s - loss: 0.1678 - acc: 0.9290 - val_loss: 0.1862 - val_acc: 0.9211 Epoch 5/100 78/78 - 146s - loss: 0.1722 - acc: 0.9272 - val_loss: 0.2598 - val_acc: 0.8750 Epoch 6/100 78/78 - 145s - loss: 0.1701 - acc: 0.9299 - val_loss: 0.1904 - val_acc: 0.9235 Epoch 7/100 78/78 - 144s - loss: 0.1605 - acc: 0.9331 - val_loss: 0.2940 - val_acc: 0.8616 Epoch 8/100 78/78 - 143s - loss: 0.1789 - acc: 0.9242 - val_loss: 0.2029 - val_acc: 0.9192 Epoch 9/100 78/78 - 143s - loss: 0.1604 - acc: 0.9313 - val_loss: 0.2291 - val_acc: 0.9097 Epoch 10/100 78/78 - 143s - loss: 0.1688 - acc: 0.9291 - val_loss: 0.1833 - val_acc: 0.9280 Epoch 11/100 78/78 - 139s - loss: 0.1717 - acc: 0.9291 - val_loss: 0.2187 - val_acc: 0.9110 Epoch 12/100 78/78 - 138s - loss: 0.1703 - acc: 0.9293 - val_loss: 0.2419 - val_acc: 0.9108 Epoch 13/100 78/78 - 139s - loss: 0.1663 - acc: 0.9276 - val_loss: 0.2721 - val_acc: 0.8812 Epoch 14/100 78/78 - 138s - loss: 0.1744 - acc: 0.9272 - val_loss: 0.2325 - val_acc: 0.9062 Epoch 15/100 78/78 - 137s - loss: 0.1662 - acc: 0.9297 - val_loss: 0.2134 - val_acc: 0.9089 Epoch 16/100 78/78 - 137s - loss: 0.1745 - acc: 0.9301 - val_loss: 0.1962 - val_acc: 0.9169 Epoch 17/100 78/78 - 137s - loss: 0.1670 - acc: 0.9288 - val_loss: 0.1876 - val_acc: 0.9276 Epoch 18/100 78/78 - 136s - loss: 0.1725 - acc: 0.9277 - val_loss: 0.2187 - val_acc: 0.9270 Epoch 19/100 78/78 - 138s - loss: 0.1638 - acc: 0.9304 - val_loss: 0.2063 - val_acc: 0.9102 Epoch 20/100 78/78 - 137s - loss: 0.1696 - acc: 0.9276 - val_loss: 0.1685 - val_acc: 0.9346 Epoch 21/100 78/78 - 137s - loss: 0.1664 - acc: 0.9296 - val_loss: 0.4021 - val_acc: 0.7954 Epoch 22/100 78/78 - 136s - loss: 0.1626 - acc: 0.9306 - val_loss: 0.1839 - val_acc: 0.9317 Epoch 23/100 78/78 - 136s - loss: 0.1783 - acc: 0.9309 - val_loss: 0.1931 - val_acc: 0.9237 Epoch 24/100 78/78 - 136s - loss: 0.1615 - acc: 0.9309 - val_loss: 0.2030 - val_acc: 0.9153 Epoch 25/100 78/78 - 138s - loss: 0.1595 - acc: 0.9318 - val_loss: 0.2336 - val_acc: 0.9091 Epoch 26/100 78/78 - 136s - loss: 0.1671 - acc: 0.9299 - val_loss: 0.1836 - val_acc: 0.9342 Epoch 27/100 78/78 - 136s - loss: 0.1716 - acc: 0.9287 - val_loss: 0.1740 - val_acc: 0.9289 Epoch 28/100 78/78 - 137s - loss: 0.1597 - acc: 0.9327 - val_loss: 0.1679 - val_acc: 0.9324 Epoch 29/100 78/78 - 141s - loss: 0.1617 - acc: 0.9313 - val_loss: 0.1904 - val_acc: 0.9326 Epoch 30/100 78/78 - 138s - loss: 0.1638 - acc: 0.9335 - val_loss: 0.2675 - val_acc: 0.8966 Epoch 31/100 78/78 - 138s - loss: 0.1716 - acc: 0.9284 - val_loss: 0.1997 - val_acc: 0.9141 Epoch 32/100 78/78 - 139s - loss: 0.1579 - acc: 0.9338 - val_loss: 0.2569 - val_acc: 0.8857 Epoch 33/100 78/78 - 139s - loss: 0.1613 - acc: 0.9324 - val_loss: 0.2751 - val_acc: 0.8921 Epoch 34/100 78/78 - 136s - loss: 0.1663 - acc: 0.9295 - val_loss: 0.3323 - val_acc: 0.8645 Epoch 35/100 78/78 - 139s - loss: 0.1756 - acc: 0.9301 - val_loss: 0.2702 - val_acc: 0.8945 Epoch 36/100 78/78 - 137s - loss: 0.1630 - acc: 0.9326 - val_loss: 0.2327 - val_acc: 0.9291 Epoch 37/100 78/78 - 141s - loss: 0.1636 - acc: 0.9319 - val_loss: 0.1659 - val_acc: 0.9346 Epoch 38/100 78/78 - 138s - loss: 0.1639 - acc: 0.9317 - val_loss: 0.1837 - val_acc: 0.9270 Epoch 39/100 78/78 - 139s - loss: 0.1585 - acc: 0.9334 - val_loss: 0.2631 - val_acc: 0.8863 Epoch 40/100 78/78 - 138s - loss: 0.1640 - acc: 0.9301 - val_loss: 0.2654 - val_acc: 0.9081 Epoch 41/100 78/78 - 140s - loss: 0.1575 - acc: 0.9341 - val_loss: 0.2170 - val_acc: 0.9200 Epoch 42/100 78/78 - 138s - loss: 0.1636 - acc: 0.9338 - val_loss: 0.2877 - val_acc: 0.8785 Epoch 43/100 78/78 - 137s - loss: 0.1598 - acc: 0.9329 - val_loss: 0.4987 - val_acc: 0.8964 Epoch 44/100 78/78 - 137s - loss: 0.1695 - acc: 0.9275 - val_loss: 0.1844 - val_acc: 0.9338 Epoch 45/100 78/78 - 139s - loss: 0.1654 - acc: 0.9313 - val_loss: 0.2525 - val_acc: 0.9042 Epoch 46/100 78/78 - 139s - loss: 0.1660 - acc: 0.9308 - val_loss: 0.1734 - val_acc: 0.9322 Epoch 47/100 78/78 - 137s - loss: 0.1608 - acc: 0.9345 - val_loss: 0.2090 - val_acc: 0.9289 Epoch 48/100 78/78 - 138s - loss: 0.1694 - acc: 0.9310 - val_loss: 0.2494 - val_acc: 0.8869 Epoch 49/100 78/78 - 138s - loss: 0.1603 - acc: 0.9353 - val_loss: 0.2201 - val_acc: 0.9085 Epoch 50/100 78/78 - 135s - loss: 0.1665 - acc: 0.9301 - val_loss: 0.2500 - val_acc: 0.9069 Epoch 51/100 78/78 - 139s - loss: 0.1531 - acc: 0.9353 - val_loss: 0.1769 - val_acc: 0.9346 Epoch 52/100 78/78 - 142s - loss: 0.1642 - acc: 0.9339 - val_loss: 0.1953 - val_acc: 0.9254 Epoch 53/100 78/78 - 140s - loss: 0.1620 - acc: 0.9327 - val_loss: 0.1592 - val_acc: 0.9354 Epoch 54/100 78/78 - 137s - loss: 0.1533 - acc: 0.9370 - val_loss: 0.1649 - val_acc: 0.9348 Epoch 55/100 78/78 - 136s - loss: 0.1652 - acc: 0.9323 - val_loss: 0.1732 - val_acc: 0.9287 Epoch 56/100 78/78 - 137s - loss: 0.1527 - acc: 0.9369 - val_loss: 0.3824 - val_acc: 0.8201 Epoch 57/100 78/78 - 136s - loss: 0.1618 - acc: 0.9312 - val_loss: 0.2443 - val_acc: 0.8943 Epoch 58/100 78/78 - 137s - loss: 0.1553 - acc: 0.9366 - val_loss: 0.1598 - val_acc: 0.9369 Epoch 59/100 78/78 - 136s - loss: 0.1550 - acc: 0.9332 - val_loss: 0.2080 - val_acc: 0.9097 Epoch 60/100 78/78 - 136s - loss: 0.1597 - acc: 0.9351 - val_loss: 0.1973 - val_acc: 0.9336 Epoch 61/100 78/78 - 136s - loss: 0.1565 - acc: 0.9349 - val_loss: 0.1700 - val_acc: 0.9342 Epoch 62/100 78/78 - 137s - loss: 0.1571 - acc: 0.9332 - val_loss: 0.1830 - val_acc: 0.9309 Epoch 63/100 78/78 - 138s - loss: 0.1575 - acc: 0.9349 - val_loss: 0.2413 - val_acc: 0.9252 Epoch 64/100 78/78 - 139s - loss: 0.1607 - acc: 0.9320 - val_loss: 0.1989 - val_acc: 0.9225 Epoch 65/100 78/78 - 143s - loss: 0.2276 - acc: 0.9272 - val_loss: 0.1754 - val_acc: 0.9309 Epoch 66/100 78/78 - 139s - loss: 0.1510 - acc: 0.9374 - val_loss: 0.1707 - val_acc: 0.9330 Epoch 67/100 78/78 - 144s - loss: 0.1578 - acc: 0.9321 - val_loss: 0.2149 - val_acc: 0.9192 Epoch 68/100 78/78 - 141s - loss: 0.1557 - acc: 0.9345 - val_loss: 0.2180 - val_acc: 0.9149 Epoch 69/100 78/78 - 142s - loss: 0.1721 - acc: 0.9329 - val_loss: 0.1878 - val_acc: 0.9223 Epoch 70/100 78/78 - 142s - loss: 0.1603 - acc: 0.9339 - val_loss: 0.2094 - val_acc: 0.9262 Epoch 71/100 78/78 - 143s - loss: 0.1593 - acc: 0.9328 - val_loss: 0.1864 - val_acc: 0.9245 Epoch 72/100 78/78 - 145s - loss: 0.1526 - acc: 0.9357 - val_loss: 0.3221 - val_acc: 0.8830 Epoch 73/100 78/78 - 145s - loss: 0.1633 - acc: 0.9311 - val_loss: 0.2339 - val_acc: 0.9202 Epoch 74/100 78/78 - 144s - loss: 0.1605 - acc: 0.9316 - val_loss: 0.2211 - val_acc: 0.9272 Epoch 75/100 78/78 - 143s - loss: 0.1428 - acc: 0.9407 - val_loss: 0.1861 - val_acc: 0.9270 Epoch 76/100 78/78 - 139s - loss: 0.1517 - acc: 0.9373 - val_loss: 0.1944 - val_acc: 0.9344 Epoch 77/100 78/78 - 139s - loss: 0.1684 - acc: 0.9312 - val_loss: 0.1645 - val_acc: 0.9332 Epoch 78/100 78/78 - 138s - loss: 0.1628 - acc: 0.9346 - val_loss: 0.2015 - val_acc: 0.9229 Epoch 79/100 78/78 - 136s - loss: 0.1520 - acc: 0.9387 - val_loss: 0.1919 - val_acc: 0.9272 Epoch 80/100 78/78 - 136s - loss: 0.1570 - acc: 0.9357 - val_loss: 0.2247 - val_acc: 0.9305 Epoch 81/100 78/78 - 152s - loss: 0.1580 - acc: 0.9358 - val_loss: 0.1721 - val_acc: 0.9309 Epoch 82/100 78/78 - 150s - loss: 0.1471 - acc: 0.9374 - val_loss: 0.1836 - val_acc: 0.9317 Epoch 83/100 78/78 - 149s - loss: 0.1565 - acc: 0.9334 - val_loss: 0.2132 - val_acc: 0.9217 Epoch 84/100 78/78 - 148s - loss: 0.1466 - acc: 0.9372 - val_loss: 0.1786 - val_acc: 0.9276 Epoch 85/100 78/78 - 147s - loss: 0.1577 - acc: 0.9346 - val_loss: 0.2986 - val_acc: 0.9050 Epoch 86/100 78/78 - 147s - loss: 0.1551 - acc: 0.9359 - val_loss: 0.1581 - val_acc: 0.9400 Epoch 87/100 78/78 - 144s - loss: 0.1620 - acc: 0.9324 - val_loss: 0.1769 - val_acc: 0.9363 Epoch 88/100 78/78 - 143s - loss: 0.1593 - acc: 0.9355 - val_loss: 0.2341 - val_acc: 0.9023 Epoch 89/100 78/78 - 141s - loss: 0.1557 - acc: 0.9346 - val_loss: 0.3245 - val_acc: 0.8935 Epoch 90/100 78/78 - 140s - loss: 0.1582 - acc: 0.9346 - val_loss: 0.2618 - val_acc: 0.9044 Epoch 91/100 78/78 - 139s - loss: 0.1514 - acc: 0.9388 - val_loss: 0.3941 - val_acc: 0.8302 Epoch 92/100 78/78 - 137s - loss: 0.1688 - acc: 0.9331 - val_loss: 0.1791 - val_acc: 0.9317 Epoch 93/100 78/78 - 138s - loss: 0.1533 - acc: 0.9365 - val_loss: 0.2477 - val_acc: 0.9169 Epoch 94/100 78/78 - 136s - loss: 0.1485 - acc: 0.9381 - val_loss: 0.2371 - val_acc: 0.9354 Epoch 95/100 78/78 - 136s - loss: 0.1656 - acc: 0.9321 - val_loss: 0.1838 - val_acc: 0.9221 Epoch 96/100 78/78 - 136s - loss: 0.1544 - acc: 0.9346 - val_loss: 0.1894 - val_acc: 0.9344 Epoch 97/100 78/78 - 138s - loss: 0.1529 - acc: 0.9378 - val_loss: 0.2735 - val_acc: 0.9192 Epoch 98/100 78/78 - 135s - loss: 0.1557 - acc: 0.9370 - val_loss: 0.1895 - val_acc: 0.9309 Epoch 99/100 78/78 - 137s - loss: 0.1609 - acc: 0.9349 - val_loss: 0.1683 - val_acc: 0.9375 Epoch 100/100 78/78 - 137s - loss: 0.1512 - acc: 0.9368 - val_loss: 0.5123 - val_acc: 0.8454 2019-07-28 14:37:54,241 graeae.timers.timer end: Ended: 2019-07-28 14:37:54.241406 I0728 14:37:54.241441 140643102242624 timer.py:77] Ended: 2019-07-28 14:37:54.241406 2019-07-28 14:37:54,242 graeae.timers.timer end: Elapsed: 3:57:37.400465 I0728 14:37:54.242489 140643102242624 timer.py:78] Elapsed: 3:57:37.400465
data = pandas.DataFrame(network.model.history.history).rename( columns={ "loss": "Training Loss", "acc": "Training Accuracy", "val_loss": "Validation Loss", "val_acc": "Validation Accuracy" }) best_validation(data.rename(columns={ "Validation Loss": "validation_loss", "Validation Accuracy": "validation_accuracy"}))
Best Accuracy: 0.94 (loss=0.16) Epoch: 85 Best Loss: 0.16 (accuracy=0.94, Epoch: 85)
line = holoviews.VLine(85) plot = data.hvplot(x="index", value_label="Epoch").opts( tools=["hover"], title="Accuracy and Loss For 100 Epochs", width=1000, height=800, ) * line Embed(plot=plot, file_name="five_layers_batch_size_256")()
By re-training I managed to squeeze out another 1 percent. I might be able to get another percentage point improvement, but maybe at this point it's good enough.
Using the Model
test_path = Path("~/data/datasets/images/dogs-vs-cats/test1/").expanduser()
test_image = random.choice(list(test_path.iterdir()))
plot = holoviews.RGB.load_image(str(test_image)).opts(width=800, height=800)
Embed(plot=plot, file_name="test_image")()
image = keras.preprocessing.image.load_img(test_image, target_size=(150, 150))
x = keras.preprocessing.image.img_to_array(image)
x = numpy.expand_dims(x, axis=0)
predictions = network.model.predict(x)
print(predictions[0])
classification = "dog" if predictions[0] else "cat"
print(f"{test_image.name} is a {classification}")
[1.] 8595.jpg is a dog
If you run this more than once you'll get different images, but it seems to have a bias towards predicting dogs.
Visualizing Intermediate Representations
Let's define a new Model that will take an image as input, and will output intermediate representations for all layers in the previous model after the first.
successive_outputs = [layer.output for layer in model.layers[1:]]
visualization_model = tensorflow.keras.models.Model(
inputs = model.input,
outputs = successive_outputs)
Now prepare a random input image of a cat or dog from the training set.
image_path = random.choice(list(testing_path.iterdir()))
image = load_img(image_path, target_size=(150, 150)) # this is a PIL image
x = img_to_array(image)
print(f"X ({type(x)}): {x.shape}")
x = x.reshape((1,) + x.shape)
print(f"X re-shape: {x.shape}")
# Rescale by 1/255
x /= 255.0
successive_feature_maps = visualization_model.predict(x)
# These are the names of the layers, so can have them as part of our plot
layer_names = [layer.name for layer in model.layers]
X (<class 'numpy.ndarray'>): (150, 150, 3) X re-shape: (1, 150, 150, 3)
Let's run our image through our network, thus obtaining all intermediate representations for this image.
# -----------------------------------------------------------------------
# Now let's display our representations
# -----------------------------------------------------------------------
for layer_name, feature_map in zip(layer_names, successive_feature_maps):
if len(feature_map.shape) == 4:
#-------------------------------------------
# Just do this for the conv / maxpool layers, not the fully-connected layers
#-------------------------------------------
n_features = feature_map.shape[-1] # number of features in the feature map
size = feature_map.shape[ 1] # feature map shape (1, size, size, n_features)
# We will tile our images in this matrix
display_grid = numpy.zeros((size, size * n_features))
#-------------------------------------------------
# Postprocess the feature to be visually palatable
#-------------------------------------------------
for i in range(n_features):
x = feature_map[0, :, :, i]
x -= x.mean()
x /= x.std ()
x *= 64
x += 128
x = numpy.clip(x, 0, 255).astype('uint8')
display_grid[:, i * size : (i + 1) * size] = x # Tile each filter into a horizontal grid
#-----------------
# Display the grid
#-----------------
scale = 40. / n_features
pyplot.figure(figsize=(scale * n_features, scale))
pyplot.title (layer_name)
pyplot.grid (False )
pyplot.imshow(display_grid, aspect='auto', cmap='viridis' )
