Attention is all you need; Attentional Neural Network Models | Łukasz Kaiser | Masterclass

If you are a brilliant post-graduate Machine Learning engineer and want to practise in real-world projects, apply for the Session 12 of the Pi School of AI starting on November 21 2022.

[autripat]

Starting @ 15:45, in well under 2 minutes, attention explained! Only a true master can do it. Love.

[lmaes]

The passion that he transmits is priceless

[MarcosPT]

This is the best explanation of attention I have seen so far! And I have been looking :)

[mosicr]

Great lecture. Best explanation of attention in just a few words.

[itshgirish]

Great presentation, he's having fun explaining the bits....great camera work- it was fun watching a moving cam than a boring still view.

[CharlesVanNoland]

I just wish he hadn't stood right in front of what he was trying to show people, but I love his passion for explaining what he's talking about.

[Cropinky]

very interesting of him to call deep learning a trade :)

[FranckDernoncourt]

Thanks for sharing! It'd be great if the video could pay more attention to the slides though.

[kadamparikh]

Great content in this video. Would love if you had the multi-headed devil covered! Though, great video to get the overall view..

[igorcherepanov]

"there is this guy, he never got his bachelor but he wrote most of these papers" - appreciation

[yacinebenaffane]

Nice explain about position and multihead ...

[vast]

They should invent a device that can always tell the time of day when the user wants.

[brandomiranda]

where is the library he talks about to get the details of training the DL "right"?

[rishabhshirke]

nothing beats GPT 2 TL;DR summarization trick

[nabinchaudhary]

does embedding gets trained or key or query or value gets trained i am confused. please help

[threeMetreJim]

"He didn't put a trophy into the suitcase because it was too small." is an ambiguous statement. "it" could refer to either the trophy or the suitcase. It seems like the answer is mainly decided on probability from past experience, rather than the intended (ambiguous) meaning, similar to a survey or experiment with too small a sample size. It is also possible that he didn't want to put a too small a trophy into the suitcase in case it ended up being jostled about too much, and became damaged; although that is a less likely, but still a possible explanation and would need a thought process to come to that conclusion, or some further context, to clarify the intended meaning. People on the Autistic spectrum (HFA / Asperger's) have that same problem when phrasing thoughts (ambiguous meaning), and are often misunderstood because of it. When a statement has two (or more) possible meanings, then it's probably unfair to judge the performance of a system in 'getting the answer right' as there isn't a definite correct answer to begin with, just a more likely one.
A word for word translation, with grammatical correction applied would probably achieve a better result in a case like this. Google translate seems to somewhat agree.
Original: He didn't put a trophy into the suitcase because it was too small
Google translate: Er hat keine Trophäe in den Koffer gesteckt, weil er zu klein war.
Back to english: He did not put a trophy in his suitcase because he was too small.
Word for word translation (incorrect, but probably still understandable if you speak German): er nicht stellen ein Trophäe in das koffer da es was auch klein.
Google translate of word to word to english (much better but still wrong - where did the 'also' come from?):he does not put a trophy in the suitcase as it is also small.

[KartoffelnSalatMitAlles]

What model is that at the beginning? Can I somehow get the machine produced texts which where shown at the beginning of the presentation?
"

[RobertElliotPahel-Short]

math majors/ graduate math students skip to 15:36

[rinkagamine]

Can I somehow get the machine produced texts which where shown at the beginning of the presentation?

[IExSet]

Strange thing, he mention "attention" term before explaining what it is. What was EXACT meaning of this Query Key Value magic ??? I suspect speakers just copy thoughts of another people mechanically, not understaning real meaning of operations !