Callable Neural Networks - Linear Layers in Depth

In this post, we'll be examining how and why we call PyTorch networks and layers. We'll also be diving into the inner workings of linear layers, the math and the code!

🕒🦎 VIDEO SECTIONS 🦎🕒

00:30 Help deeplizard add video timestamps - See example in the description
12:51 Collective Intelligence and the DEEPLIZARD HIVEMIND

💥🦎 DEEPLIZARD COMMUNITY RESOURCES 🦎💥

👋 Hey, we're Chris and Mandy, the creators of deeplizard!

👉 Check out the website for more learning material:

💻 ENROLL TO GET DOWNLOAD ACCESS TO CODE FILES

🧠 Support collective intelligence, join the deeplizard hivemind:

🧠 Use code DEEPLIZARD at checkout to receive 15% off your first Neurohacker order
👉 Use your receipt from Neurohacker to get a discount on deeplizard courses

👀 CHECK OUT OUR VLOG:

❤️🦎 Special thanks to the following polymaths of the deeplizard hivemind:
Tammy
Mano Prime
Ling Li

🚀 Boost collective intelligence by sharing this video on social media!

👀 Follow deeplizard:

🎓 Deep Learning with deeplizard:

🎓 Other Courses:

🛒 Check out products deeplizard recommends on Amazon:

🎵 deeplizard uses music by Kevin MacLeod

❤️ Please use the knowledge gained from deeplizard content for good, not evil.

[deeplizard]

Check out the corresponding blog and other resources for this video at:

[MrDonald]

Please don't give up on doing videos, this youtube channel is too good to stop producing content :p <3

[tingnews]

Thank you for your guys hard work.


Before I watch what I thought I will learn
1、How linear works
2、How pytorch implement


What I learned
1、Linear is mapping input dimension to output dimension
2、Weight create by Parameters function
3、Callable layers create grad
4、Access weights by layer.weights
5、layer(input) - __call__(input) - forward(input)

[btsport]

Best Pytorch Course, very Intuitive.
Please complete the course as soon as possible.
Thank you.

[shaikshehanaz]

Aousome videos and content.
Any words are not sufficient to apprisiate you, deeplizard team.

[interested_in_everything]

Please do a complete tutorial on tensorflow 2.0! Love your videos!

[yasaminm]

Such a clear explanation. I appreciate how you step through each function so that the viewer gains a strong understanding of all the details. Thank you.

[pranavagarwal]

Along with the main context, I really like the ending of each video. Keep up the good stuff coming. Thanks

[JousefM]

Finally guys! :D Greetings from a Patreon :)

[Nissearne]

Very good tutorial. Easy to follow and you digg down in the details I was wounder about in PyTorch. Thanks ! It's perfect match for my education. I need both to learning python in general and PyTorch API (I have work with C/C++ before so when you describe python in general it's also relevant for me. I have knowledge about CNN structure before but not the PyTorch API so this hole series is perfect).

[FarooqComputerVision]

Thank you so much for sharing such knowledgeable videos. Waiting for the remaining stuff of this videos series. stay blessed!!!

[hlxhbhn]

Excellent video with clearly run through `forward()`! Thanks!

[nisargvp]

I love the deep dives into the PyTorch source code. Which IDE are you guys using? Can you guys develop a video on helping noobs with setting up and effectively using an IDE for data science?

[zhangkent]

Thanks for clear __call__() function explain. It is noticable that torch.nn.functional.linear is reusing torch._C._nn.linear, you will not see matmul directly now as in 10:23.

[engrnasirshah]

Sir. i need help in lstm using pytorch. Sir. the task is multi input <->multi-output LSTM using pytorch

[perecanals]

This series is absolutely fantastic, congratulations!

I am having trouble using the VS Code. I have never before used any this kind of softwares, and I am not familiar with how they work and what they are used for.

I am using a Mac and I created a folder in my Desktop for this series where I have the .py file that you are working on in the video, and I tried to follow your steps... but I couldn’t.

First, I can’t access the linear class source code by clicking as you do.

I also have a problem with the dtype, I get this in the terminal (I checked the code a million times):


t = torch.Tensor([1, 2, 3, 4], dtype=torch.float32)
TypeError: new() received an invalid combination of arguments - got (list, dtype=torch.dtype), but expected one of:
* (torch.device device)
* (torch.Storage storage)
* (Tensor other)
* (tuple of ints size, torch.device device)
didn't match because some of the keywords were incorrect: dtype
* (object data, torch.device device)
didn't match because some of the keywords were incorrect: dtype

But it works fine in Jupyter Notebook.

Perhaps these problems are very basic, but I’d like to get VSC working, so if anybody can help that’d be awesome.

Thanks folks!

[ellachen]

Thank you much for the amazing video!! Just wondering which IDE this is? Thanks! :D

[billtang]

Great video! I am really curious about how did you step into source code on VS code. My vs code just steps over codes that are not mine when i debug. How did you do this? Thanks

[akshaytiwari]

Why would you save the weight matrix as 3, 4 and then apply transpose rather than saving it as 4, 3 in the first place ?

[randaelanwar]

Hello, Great tutorial indeed. Lots of thanks.
I wonder if the NVIDIA CEO talk at the end of the video is available somewhere, I need to watch the entire talk if possible. Thank you very much again.