From Attention to Generative Language Models - One line of code at a time!

This in-depth tutorial is about writing Causal Generative Language models from scratch in Pytorch. I explain all the concepts you need to understand what goes under the hood in Transformers and Attention Models. I also tried to add some visualizations to help explain each concept, and walk you through every line of code to explain how all the math works. I aimed for the right balance of complexity of material and simplicity of explanation to make the best learning experience I can give. Follow-up topics such as Multi-Headed Attention, Multi-Query Attention, Grouped Query Attention will be covered in a future episode.

Join the channel on Patreon to receive updates about the channel, and get access to bonus content used in all my videos. Here is the link:

All the codes, slides, and notebooks used in this video have all been shared in the Patreon. Go have fun! :)

Videos you might like:

#pytorch #deeplearning #machinelearning

0:00 - Intro
1:00 - Semantic Similarity
3:00 - Matrix Multiplication
7:00 - Attention Scores
13:20 - Contextual Embeddings
21:18 - Attention with Pytorch
28:43 - Self Attention
30:37 - Causal Masked Attention
33:37 - Transformer Decoder Blocks
38:40 - Next Word Prediction

[alexitport]

Such a refreshing feeling while watching a young and brilliant mind presenting AI ... Thank you for your videos and work, much appreciated! And I will now subscribe on your patreon too!. If you celebrate Diwali/Deepavali tomorrow, I wish you happy celebration with your beloved ones!

[TheVishnu]

Thank you for explaining transformer based LM with code and concepts, in such a simple manner! hard to find such tutorials these days!

[actualBIAS]

Because I saw your upload yesterday I thought to myself, hmmm why not? Why don't I try to understand Transformers. So I began the journey by watching your first 4 videos, then I watched 2 lectures online, then I kinda watched 3b1b playlist on deep learning and transformers, then i red the paper and now I'm back here finally ready to understand what you are talking about. You were very helpful and a key motivator for this journey. Thank you very much. You deserve FAR MORE subscribers. Your videos are awesome and well produced.

[kunwar_divyanshu]

This became my one of the most fav channel on Neural Networks . The other favs are by karpathy and HeduAI

[josephmyalla]

You are really great Machine Learning teacher, from the math to language model, this is cool.

[harshitdubey]

This video is full of information, let me request one thing here I'm not a beginner but still have to struggle with lot of concepts to understand this could be because you have to explain whole course in a single video but If it is possible for you to teach this in more simple way that will be very helpful even for beginners. Thank you so much for this amazing video❤.

[alimaina]

You are good. Keep it up. Subscribed and Liked!

[naveenairani]

Bro can you make a video on image to 3d objects generation using neural radiance field

[wilfredomartel]

Great work! could you please describe a little more abput the book " neural attention" .

[sam-uwgf]

This is implementation of transformer from scratch right??

[vasanthravi]

I have one pdf of total 125pages and each page covering different topics. Example page from 10to20 covers about Health, than the page 21 to 30 covers the topic Education.

First I need to store this pdf 125 data into the vector database.

Then the actual requirement is if the user gives a topic as Health I need to retrieve the whole content from the page 10to20 and make it as simple blog.

How to make this possible? Suggest me a best approach to start this bro

[Elegant-Capybara]

Finally Some good phukin food. Let's go.

[nekytasnim]

Dude it's illegal for this to be free