filmov
tv
Coding the 124 million parameter GPT-2 model
Показать описание
In this lecture, we code the entire 124 million parameter GPT-2 Model class in Python.
This includes the following components:
Token + Positional embedding
Transformer block
Layer normalization
Output layer
We understand the theory, mathematical intuition and also do the coding for the entire implementation.
After this lecture, you will have a firm understanding of how the entire GPT-2 architecture works.
0:00 Birds eye view of GPT-2 architecture
7:45 Token, positional and input embeddings
17:29 Dropout layer
20:47 The 8 steps of the transformer block
32:37 Post transformer layer normalisation
33:36 Output layer
40:20 Coding the entire GPT-2 architecture in Python
51:42 Testing the GPT model class on a simple example
53:51 Parameter and memory calculations
57:56 Conclusion and summary
=================================================
=================================================
Vizuara philosophy:
As we learn AI/ML/DL the material, we will share thoughts on what is actually useful in industry and what has become irrelevant. We will also share a lot of information on which subject contains open areas of research. Interested students can also start their research journey there.
Students who are confused or stuck in their ML journey, maybe courses and offline videos are not inspiring enough. What might inspire you is if you see someone else learning and implementing machine learning from scratch.
No cost. No hidden charges. Pure old school teaching and learning.
=================================================
🌟 Meet Our Team: 🌟
🎓 Dr. Raj Dandekar (MIT PhD, IIT Madras department topper)
🎓 Dr. Rajat Dandekar (Purdue PhD, IIT Madras department gold medalist)
🎓 Dr. Sreedath Panat (MIT PhD, IIT Madras department gold medalist)
🎓 Sahil Pocker (Machine Learning Engineer at Vizuara)
🎓 Abhijeet Singh (Software Developer at Vizuara, GSOC 24, SOB 23)
🎓 Sourav Jana (Software Developer at Vizuara)
This includes the following components:
Token + Positional embedding
Transformer block
Layer normalization
Output layer
We understand the theory, mathematical intuition and also do the coding for the entire implementation.
After this lecture, you will have a firm understanding of how the entire GPT-2 architecture works.
0:00 Birds eye view of GPT-2 architecture
7:45 Token, positional and input embeddings
17:29 Dropout layer
20:47 The 8 steps of the transformer block
32:37 Post transformer layer normalisation
33:36 Output layer
40:20 Coding the entire GPT-2 architecture in Python
51:42 Testing the GPT model class on a simple example
53:51 Parameter and memory calculations
57:56 Conclusion and summary
=================================================
=================================================
Vizuara philosophy:
As we learn AI/ML/DL the material, we will share thoughts on what is actually useful in industry and what has become irrelevant. We will also share a lot of information on which subject contains open areas of research. Interested students can also start their research journey there.
Students who are confused or stuck in their ML journey, maybe courses and offline videos are not inspiring enough. What might inspire you is if you see someone else learning and implementing machine learning from scratch.
No cost. No hidden charges. Pure old school teaching and learning.
=================================================
🌟 Meet Our Team: 🌟
🎓 Dr. Raj Dandekar (MIT PhD, IIT Madras department topper)
🎓 Dr. Rajat Dandekar (Purdue PhD, IIT Madras department gold medalist)
🎓 Dr. Sreedath Panat (MIT PhD, IIT Madras department gold medalist)
🎓 Sahil Pocker (Machine Learning Engineer at Vizuara)
🎓 Abhijeet Singh (Software Developer at Vizuara, GSOC 24, SOB 23)
🎓 Sourav Jana (Software Developer at Vizuara)
1:01:33
Coding the 124 million parameter GPT-2 model
4:01:26
Let's reproduce GPT-2 (124M)
1:56:20
Let's build GPT: from scratch, in code, spelled out.
0:00:15
This guy bought #bitcoin at $1 😳 🥳 #btc #crypto #cryptocurrency #davincij15
1:48:55
The Thirty Million Line Problem
0:07:06
The Simple Question that Stumped Everyone Except Marilyn vos Savant
0:00:23
HOW CHINESE STUDENTS SO FAST IN SOLVING MATH OVER AMERICAN STUDENTS
0:28:18
Fine-tuning Large Language Models (LLMs) | w/ Example Code
0:15:42
LLMs Learn Tools at 775M Parameters!
0:10:19
The Most Controversial Problem in Philosophy
0:30:10
Introduction to building machine learning models in R with mikropml (CC124)
0:19:59
What Can Huge Neural Networks do?
0:28:23
TokenFormer: Rethinking Transformer Scaling with Tokenized Model Parameters (Paper Explained)
2:45:10
Building LLMs from the Ground Up: A 3-hour Coding Workshop
0:00:22
3BY3 CUBE CHALLENGE #viral #cube
0:13:59
Gopher Explained: 280 BILLION Parameter Model Beats GPT-3
1:25:52
WE WENT TO ARMY BOOT CAMP! -You Should Know Podcast- Episode 123
0:00:34
When siblings fight over the VR headset... #RecRoom
0:00:21
Ek jhatke mein ho jayega The End 💔
0:56:10
I programmed some creatures. They Evolved.
0:24:34
Tutorial 124 - Using pretrained models as encoders in U-Net
0:00:16
24 Carat Gold means 100 percent pure Gold and calculate gold purity on carat basis
0:50:43
12a: Neural Nets
0:40:55
PEFT LoRA Explained in Detail - Fine-Tune your LLM on your local GPU
Комментарии