PyTorch

Things to Note:

• In places where BackPropagation is not needed (such as when using test data or in practical applications), it should be done within torch.no_grad().

  • Also be cautious when using a Pretrained Model, set the parts that you don’t want to modify to false.

• If you are using Dropout, turn it off for test data or in practical applications using model.eval().

  • When training, revert back to the original state using model.train().
  • Alternatively, you can use “when” to make it easier.

#udacity_intro_to_deep_learning_with_pytorch

• A few things to check if your network isn’t training appropriately
  • Make sure you’re clearing the gradients in the training loop with optimizer.zero_grad(). If you’re doing a validation loop, be sure to set the network to evaluation mode with model.eval(), then back to training mode with model.train().
• Well, it’s the same as what was mentioned above.