Tips, Tricks and Other Notes

As the tensors go through the model you should check the shapes to make sure they are correct (or at least what you expect).

• Make sure you are clearing the gradients in the training loop with optimizer.zero_grad()
• In the validation loop, set the network to evaluation mode with model.eval() and then back to training mode with model.train

If you see an error saying pytorch Expected an object of type torch.FloatTensor but found type torch.cuda.FloatTensor then it means something is trying to be run on the CPU but something else wants to use the GPU. Make sure you called .to(device) on the model and all your tensors (including the data).