PyTorch Tutorial 15 - Transfer Learning

New Tutorial series about Deep Learning with PyTorch!

In this part we will learn about transfer learning and how this can be implemented in PyTorch.

We will learn:
- What is Transfer Learning
- Use the pretrained ResNet-18 model
- Apply transfer learning to classify ants and bees
- Exchange the last fully connected layer
- Try 2 methods: Finetune the whole network or train only the last layer
- Evaluate the results

📚 Get my FREE NumPy Handbook:

📓 Notebooks available on Patreon:

Part 15: Transfer Learning

You can download the dataset here:

If you enjoyed this video, please subscribe to the channel!

Official website:

Part 01:

More about Transfer Learning:

Code for this tutorial series:

You can find me here:

#Python #DeepLearning #Pytorch

----------------------------------------------------------------------------------------------------------
* This is a sponsored link. By clicking on it you will not have any additional costs, instead you will support me and my project. Thank you so much for the support! 🙏

[ross]

For those new to transfer learning: ideally we would like to freeze all of the layers other than the newly added head layer, and train for n epochs, then unfreeze the preceding layers, and train the entire network using a sliced learning rate, where the parameters of the later layers are updated faster than the parameters of the earlier layers. This is how libraries like fastai handle transfer learning out of the box.

[georgesebastian]

Best PyTorch tutorial series available on Internet ! Could you please make a video on "Training models using Tensorboard" as well?

[안경환-xt]

i'm from Korea. it was wonderful lectures ever!! thank you.

[shelfinyoung]

The best pytorch tutorial I've ever seen，very foudmamental and good for pytorh beginer

[TheOraware]

Thanks for your teaching, normally when we transform train data then we use the same transformation on validation data, but here i am seeing train data transformation is and validation data has transforms.Resize(256). Same with Flip with train data but not on validation data, i am confused?

[cibinjohnjoseph]

Thanks for creating this playlist. It has been really helpful. Well done, and keep going...

[pranjalsaxena]

Finally, it was a great journey with you bro. :) Thanks a ton

[VjayVenugopal]

for thos who dont know, how these standard_dev and mean are calculated before handedly

1) apply the to_tensor() transformation to the dataset
2) load the dataloader
3) keep the batch size as one for the train dataset

data = iter(train_data_loader)
data = next(data)
print(data[0].mean(), data[0].std())

[nougatschnitte]

Hey, appreciate the tutorials, but you keep saying that you explained everything for the training loop in the previous tutorials. However I find it differs significantly with what you covered sofar. E.g. an explaination of what use it is to set the model in training or eval mode is completely missing. I am wondering what that is for, since we didnt need it in the previous tutorials.
Also the line with best_model_wts = leaves a question mark with me. Could you clear that up ? :)

[bluebox]

I didn't know about this concept before and was super impressed. That is so damn amazing! Nice explanation as always :D

[fabienmathieu]

Thank you for this really nice, simple and efficient course which has become my reference for learning PyTorch and practice deep learning.

[alanzhu]

just wonder why the mean and std are np.array with three elements?

[camus]

Awesome job, thanks!
An LSTMs Pytorch tutorial would be welcome...

How to deal with tabular data and the transfer learning? I have a model that has been trained, and suddenly, the number of features will be increased, so now, how to deal with trained model and various head part of features (the first, dynamic layer of dense neural network) ? Thank you.

[javeriaehsan]

how you calculated mean and std? I have to transfer learning the alexnet for mnist data but how to calculate mean and std for that

[abderrahmanebououden]

thanks so much, bro, it is really good explanation and helpful

[darthdaenerys]

Thanks for sharing. Can you make a video on GANs

[noamills]

The link in the description that says "More about Transfer Learning" is actually the link to the source code that Patrick copied and provided minimal revisions to (including deleting the comments specifying the original author and licensing info). The original code has a BSD license which requires attribution. Saying "more information here" in the description isn't the same as saying "source code from here". Please provide appropriate attribution in your videos and on your github.

[ahnafmunir]

I couldn't find any previous video in this playlist that discusses dividing the data into training and evaluation sets. Did I miss anything?

[rs]

How can I use pretrained weights like vgg16 in fcn architecture.
Is this correct?
self.conv1_1 = vgg16.features[0]

Please help