Transformers for beginners | What are they and how do they work

This week we’re looking into transformers. Transformers were introduced a couple of years ago with the paper Attention is All You Need by Google Researchers. Since its introduction transformers has been widely adopted in the industry.

Get your Free Token for AssemblyAI Speech-To-Text API 👇

Models like BERT, GPT-3 made groundbreaking improvements in the world of NLP using transformers. Since then model libraries like hugging face made it possible for everyone to use transformer based models in their projects. But what are transformers and how do they work? How are they different from other deep learning models like RNNs, LSTMs? Why are they better?

In this video, we learn about it all!

Some of my favorite resources on Transformers:

Some images used in this video are from:

[pierluigiurru]

This is clearest explanation of transformers I’ve found so far, and I personally have seen many trying to wrap my head around them. No skimming over details. Very well done!

[ashermai]

This channel deserves more views and likes

[Yaddu]

I really want you talk about attention. Thank you, shinning in this video.

[testing]

I am a programmer, I have created many tools that were actually very useful. I even claim that I have 10+ years experience. But I feel very bad to realize that I am so dumb that I did not understand anything after the first 10 minutes of the video.

[dooseobkim]

You are my savior for being actually able to get ready to read all of those AI related papers which I’m completely unaware of. I was stuck at the part of my thesis which I have to provide theoretical background of ChatGPT. As a business student I’m super grateful to learn these knowledges in computer science through your short lecture👍👍

[stevemassicotte]

@16:14, , the binary table is wrong, there are two sevens.
The second column should start with 8 and not a second 7.
Attention is all you need ;)
Thanks for the video !

[andybrice]

Positional encodings are not that weird when you think of them as being similar to the hands on a clock: It's a way of representing arbitrarily long periods of time, within a confined space, with smooth continuous movement and no sudden jumps.

Picture the tips of clock hands. Their vertical position follows a sine wave, their horizontal position follows a cosine wave. And we add precision with more hands moving at different speeds.

[reshamgaire]

Finally found a perfect video that cleared all my confusions. Thank you so much ma'am, may god bless you 🙏

[moeal]

This is most clear and resourceful video I've seen. Thank you for your hard work and for sharing these resources

[mohamadhasanzeinali]

I saw numerous videos about Transformers architecture. In my opinion, your video is the best among them. Appreciate that.

[nikhilshrestha]

really love how you described the model. easier to understand 🙌

[PeterKoman]

Finally a transformer video that actually explains the theory in understandable way. Many thanks.

[anandanv]

The way you explained the concept was awesome. It is very easy to follow.👍

[sivad]

The best video on transformer architecture with great explanations and charming presentation.

[yourshanky]

Excellent explanation !! Sharp and clear. Thanks for sharing this.

[bdoriandasilva]

Great video with a clear explanation. thank you!

[nikhil]

Thank you so much!💓this has to be the best introduction video to Transformers. We are planning to use Transformers for our Video Processing project.

[imagnihton]

This made the concept sound incredibly simple compared to some other sources... Amazing!

[Zulu]

This video is the best technical explanation I have seen in years. Although Transformers are a breakthrough in the field in NLP, I am convinced that they do not describe completely and satisfactorily, the way humans process language.

For all civilizations, spoken language predates written language in communications. Those who do not read and write, still communicate clearly with others. This means humans do not represent natural language in their brains in terms of words, syntax and position of tokens but rather in terms of symbols, images and multimedia shows that make up stories we relate to.

Written language comes only later as an extra layer of communication to express transparently these internal representations that we carry within ourselves. If AI is able to access and decode these internal representations, then the written language, the extra layer, becomes a lot easier to understand, organize, and put on paper with simple techniques rather than using these intricate Transformers that I consider as temporary and unnatural ways of describing natural languages.

[moonlight-tded]

cleanest and most informative video ever.. covered whole attention is all you need paper in 19 mins.. damn.. thank you MISRA TURP and assembly ai