What are Transformer Neural Networks?

Показать описание

This short tutorial covers the basics of the Transformer, a neural network architecture designed for handling sequential data in machine learning.

Timestamps:
0:00 - Intro
1:18 - Motivation for developing the Transformer
2:44 - Input embeddings (start of encoder walk-through)
3:29 - Attention
6:29 - Multi-head attention
7:55 - Positional encodings
9:59 - Add & norm, feedforward, & stacking encoder layers
11:14 - Masked multi-head attention (start of decoder walk-through)
12:35 - Cross-attention
13:38 - Decoder output & prediction probabilities
14:46 - Complexity analysis
16:00 - Transformers as graph neural networks

Original Transformers paper:

Other papers mentioned:

Video style inspired by 3Blue1Brown

Music: Trinkets by Vincent Rubinetti

Links:

If you'd like to help support the channel (completely optional), you can donate a cup of coffee via the following:

Ari Seff

Рекомендации по теме

Комментарии

Love the 3blue1brown esque style. Keep em coming

CodeEmporium

I think I speak for everyone when I say.. please keep posting! This is excellent stuff!

Mutual_Information

Your videos exhibit exceptional quality. Thank you for this outstanding contribution.

jwine

I put 2 months into studying this. I’m a Cultural Science major and I can finally say that I understood every part of this video.
I’m really proud of myself and very thankful for your excellent didactic style.

PaperTigerLive

The best help in understanding transformers while reading „Attention is all you need“ I have found.

justusmzb

i thought i was smart - but then i started getting my head around all this amazing machine learning and then i was humbled. thanks for sharing

anmusic

This video is a mix of actual explanation about the nature of transformers and long tangents about implementation details that are mixed together so perfectly it ensures that it's impossible follow or even know if the author of this video understands how transformers work.

googleyoutubechannel

Thanks for the nice content. I am not a beginner, but unfortunately it was hard for me to follow this video. There were many concepts/terms mentioned without a brief explanation, and the pace was rather fast. If you could publish the same video, with additional examples and clarifications would be much appreciated. I understand that one would need to look up some topics and references while watching the video, but in this case it felt like I have to look up things very often. Thanks again for your effort!

konna

Thank YOU! this is the most precise and straight forward thing about Attention that I ever have. When you said the word compatibility, I finally understood why do I need to take dot product between Queries and Keys.

pradiptahafid

Thank you for the clear and concise explanation! I understood this more than any other video I've seen yet (although I'm still learning). Looking forward to more videos like this!

theJeet

Our community is the best!!! 💪💪 Thank you very much for the amazing review!!!

adrewkin

i am at 1/3rd of the video, stopped to come here express gratitude before getting back to it! 3B1B is a proven style, no harm in using it, later on adding to it.

samirelzein

These are great! I can see this channel easily becoming as popular as Yannic Kilcher’s. Thank you for all your work, your explanations give a lot of clarity without sacrificing depth!

rmac

Transformers are incredibly complicated but you tried to simplify it for us any way it does need another look
Thank you very much indeed

AIdevel

incredible, insightful, delightful, enlightening

bijan

Exceptionally explained! Please make more content! This stuff is worth paying for ;)

SaheelGodhane_TheTramp

Dude I was looking for a channel like yours for weeks !

nprm

I watched a bunch of videos on this and I feel like after this, I actually understand it. Thank you!

jcorey

Thanks for the vid. This is exactly level of the details and explanation style I needed. Many other explanations are either too vague and miss important details or too hardcore and hard to follow. This is ideal: most of the technical details are here, but still easy to follow.

alexandrsavochkin

Maybe the best transformer explanation out there

andreasgian

What are Transformer Neural Networks?

Transformers, explained: Understand the model behind GPT, BERT, and T5

Transformer Neural Networks, ChatGPT's foundation, Clearly Explained!!!

Illustrated Guide to Transformers Neural Network: A step by step explanation

What are Transformer Neural Networks?

Transformer Neural Networks - EXPLAINED! (Attention is all you need)

But what is a GPT? Visual intro to transformers | Chapter 5, Deep Learning

What are Transformers (Machine Learning Model)?

Transformer Neural Networks Derived from Scratch

What are transformers?

The complete guide to Transformer neural Networks!

Attention in transformers, visually explained | Chapter 6, Deep Learning

Transformers for beginners | What are they and how do they work

Attention is all you need (Transformer) - Model explanation (including math), Inference and Training

Transformer models and BERT model: Overview

MIT 6.S191 (2023): Recurrent Neural Networks, Transformers, and Attention

Visual Guide to Transformer Neural Networks - (Episode 2) Multi-Head & Self-Attention

The matrix math behind transformer neural networks, one step at a time!!!

Attention mechanism: Overview

What are Transformer Models and how do they work?

Attention for Neural Networks, Clearly Explained!!!

Positional Encoding in Transformer Neural Networks Explained

transformer neural network simply explained

Visual Guide to Transformer Neural Networks - (Episode 1) Position Embeddings

But what is a neural network? | Chapter 1, Deep learning