What does GPT in ChatGPT do?

ABOUT ME

RESOURCES

PLAYLISTS FROM MY CHANNEL

MATH COURSES (7 day free trial)

OTHER RELATED COURSES (7 day free trial)

[victor-iyi]

I enjoy watching your videos. I watch them over and over again & learning sth new every time. I especially like how you do your “passes” throughout your videos even though you’ve made a video on those “pass” and not just referring people without a high level overview. Best explanation I’ve ever seen. Great job! 😊

[nerassurdo]

I enjoy your videos. You are actually very good. Keep it up! your channel will grow for sure.

[prashantlawhatre]

The videos in this playlist are by far the best I have seen on this topic. Specially the parts where Ajay discusses the Reward model and PPO.

[paull]

Great content. I especially like the ChatGPT series, very clear explained. Already excited about the upcoming video about GPT.

[josephpareti]

is the softmax layer only used for SFT - which I would expect ?

[vtrandal]

[Great channel. Love your channel.] Questions: Lacurt scale? What is that? Likert scale? Got it. And DALL-E? What is that? When I search videos on your YouTube channel it pulls up a few videos but I cannot find DALL-E in the video transcript.

[grownupgaming]

11:40 Cannot understand, why do we want to further fine-tune the original GPT parameters? Instead, freeze them and keep the learning to the newly added final linear layer, wouldnt that be enough?

[charlesje]

I play these videos in the background while learning python and it's starting to sink in.

[jeff__w]

13:24 “…and eventually it’s going to generate a vector which is going to be of the size [VOCAB SIZE + 1]”
If it’s the number of possible tokens in a given language, why it is “generating” anything at all? Isn’t that always the same for a given language? Why doesn’t it just “look up” that vector?

[pariotourpariotour]

I'm looking for how attention layers are trained.😊