Training Model - Deep Learning and Neural Networks with Python and Pytorch p.4

In this deep learning with Python and Pytorch tutorial, we'll be actually training this neural network by learning how to iterate over our data, pass to the model, calculate loss from the result, and then do backpropagation to slowly fit our model to the data.

#pytorch #deeplearning #machinelearning

[fuba]

So.. You are testing on your training set, would explain the 97+% accuracy?

[Darmokxx]

"model.train()" and "model.eval()" activates and deactivates Dropout and BatchNorm, so it is quite important. "with torch.no_grad()" only deactivates gradient calculations, but doesn't turn off Dropout and BatchNorm. Your model accuracy will therefore be lower if you don't use model.eval() when evaluating the model.

[vibhuvaibhav]

Please please do a RNN tutorial as well. It's amazing to learn from you! Thanks a lot for being there for us, lots of love!

[fireheart__]

These are probably the finest tutorials I have come across. Thank you so much sentdex!!

[kingeng]

thanks for the tutorial:
you can use this line of code instead of second for loop to get num of corrects:
```
correct +=
total += len(y)
```

[wktodd]

Still here, an old dog learning new tricks :-) (I wrote my first computer program in 1968 - waiting a week for the result to be posted back to me, I still think I should optimise loops for speed and use ints where ever possible ;-) )

[alexnick]

Hey, I just wanted to tell you that you do the best tutorials! You know exactly what are the problems with 95% of all tutorials (like only using preprocessed datasets). I tried to use my own images and no tutorial helped me. Your work is just amazing!

[arindam]

Just finished all the 8 videos in this playlist. Loved it. Hope you make more of these pytorch videos.

[r.alexander]

Thanks for the video first of all.
Furthermore, I think Adam is an optimizer which uses both Momentum and RMSprop.
RMSprop in itself already does some form of decreasing of the learning rate.
Why the need for another (7:35) learning rate decay method? Isnt this decay strong enough by itself (especially when using correct parameters) ?
Would love your thoughts on this, thank you.

[SACAS]

Dude, thanks for the class, being honest it took me three days to watch the three videos and I'm spealingspanish so I coudn´t understand the mindest part but I watch other videos and at last I understood almost all, by the way i proved the testset and I get
Accuracy: 97.1%

[alissondamasceno]

You're the best python teacher ever bro!

[tudor]

I recommend using this approach for counting correctly identified samples: correct += torch.sum(torch.argmax(output, dim=1) == y)

[WandererOfWorlds]

If you want to run your tutorial on the GPU:

Before the train loop:
device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
net.to(device)
In the train loop:
X, Y = data
X = X.to(device)
Y = Y.to(device)

MNIST is such a small set that it takes longer to constantly copy the data to the GPU than the actual training but it is the easiest way to validate GPU training based on your current code.

[maitreyaverma]

Amazing work! I really like your "engineering" approach to every new thing.

[neroandjohn]

hello, can someone explain why did we write torch.argmax(i) == y[idx] at 22:06?

[girish]

your tutorials are great, logical and rational

[HarshitSingh-tgyv]

17:36 loss.backward() is inside the for loop of batch which means we are back propagating the loss of only one particular batch at a time. Shouldn't we sum the loss for all batches and then back propagate?

[DrNaserRazavi]

You are wrong about model.train() and model.eval(). The model.eval() deactivates some layers which are specific to training mode such as dropout layers and bachnorm layers.

[Skinz]

This tutorial serie is insane. Thanks a lot

[martinmartin]

Its worth mentioning that if you use overcapacity networks (which have much more parameters that would actually needed) you can circumvent the problem of getting stuck in local minima when using smaller learning rates as the model has enough "knobs" which it can use to wiggle aroung the local loss minima. Actually, this is the main idea behind deep learning.