High-Level Tutorial of OpenAI's GPT-3 | GPT-3 VS BERT Family of NLP Models

GPT-3 is the biggest language ever model built, and it has been attracting a lot of attention. Rather than argue about whether GPT-3 is overhyped or not, we wanted to dig in to the literature and understand what GPT-3 is (and is not) in light of it’s predecessors and alternative transformer models. In this video we share some of what we’ve learned. What is GPT-3 really good at? What are its constraints? How useful is it for business? Enjoy!

⏰ Time Stamps ⏰
00:40 - Comparison of latest Natural Language Processing Models
01:09 - What is a Transformer Model
01:50 - The Two Types of Transformer Models
02:15 - Difference between bi-directional encoders (BERT) and autoregressive decoders (GPT)
04:40 - GPT-3 is HUGE, does size matter?
05:24 - Presentation of size differences between GPT-3 relative to BERT, RoBERTa, GPT-2, and T5
07:40 - What does GPT do and how is it different than the BERT family?
18:05 - Is GPT-3 a Child Prodigy or a Parlor Trick?
18:44 - Back to the Issue of GPT-3's size
19:30 - Final thoughts on GPT-3 vs BERT

[basavarajukm]

The camera panning in and out is kinda distracting.. please keep it steady

[anuragsodhi]

surprised why this doesn't have more views. excellent video

[hafidhrendyanto]

This channel is really underrated. Please keep doing the good work! I known a fitness/self improvement channel that only have few hundred views for some years and now he have 100k subs.

[shifathrahman]

pretty awesome video to have an overview of models generated from transformers. I've come to know a lot from this video. thanks

[Olle_Green]

Great vid bud. Great to see more content like this.

Cheers.

[lingding]

great video. I agree about the auto-camera cropping issues, but honestly I got used to it after a while. Also, mostly I was just looking at the slides and listening, and both the explanations and the visuals were extremely well thought out and clear. good job!

[sunilsurendrasingh]

The time analogy is brilliant; it clarifies the comparison even to a non-technical person.

[malibulut]

Very good explanation, looking forward for more!

[tongluo]

great summary of the 2 family, and explaining the difference very clear. I subscribe you.

[free_guac]

Such a good video to understand BERT vs GPT

[JC-jzrx]

who is the speaker in this video? I like the way he presented the information

[HazemAzim]

Very informative Thank you ! wonderful 20 minutes capsule on the Topic . I wonder if there are updates concerning GPT 3.5, as to the Generic VS domain-specific edge that BERT models over GPT models . look forward for an update

[mdrnprimitve-wesupportsmal]

A great video for what was a tentative step into NLP - Tristan Harris mentioned some powerful demos i was looking for those. If you have an idea on that I would be very grateful thx

[alqods]

read an article about XLNet is better alternative combining both GPT-3 and BERT, is that true?

[riccardocoz]

Very interesting and useful video :)
As you state, it's not often really eseful having a so huge model which is pretty good at generating text, so why should we need it when we can fine-tune a Bert-like model? But a few weeks ago they developed DALL-E and for me it was astonishing seeing how it generate images. Would you like to do a video about it? It would be very cool.
Cheers?

[staticmind]

I read this argument a little while ago, can't remember where, that the training task for BERT Family is really just a more generalized form of the training task for GPT since you can condition BERTs such that the missing text is the last word

[VincentFulco]

Outstanding tutorial, please fix the auto framing...makes me nauseous

[srikika]

Wasn't BERT using a masked Language model too.

[abdulelahabuabat]

The video seems to be very useful. However, I could not watch more than 5 minutes because of the camera panning.

[gsutton]

please fix your camera so that when your face move the camera doesn't keep jumping all over that place. It's extremely annoying and makes you video unwatchable.