torch.nn.TransformerDecoderLayer - Part 2 - Embedding, First Multi-Head attention and Normalization

preview_player
Показать описание


  1. Machine Learning with Pytorch
  2. #nlp
  3. #transformers
  4. #attention
  5. #multihead attention
  6. #transformer decoder
Рекомендации по теме
torch.nn.TransformerDecoderLayer - Part 0:09:29

torch.nn.TransformerDecoderLayer - Part 2 - Embedding, First Multi-Head attention and Normalization

torch.nn.TransformerDecoderLayer - Part 0:09:32

torch.nn.TransformerDecoderLayer - Part 3 -Multi-Head attention and Normalization

torch.nn.TransformerDecoderLayer - Part 0:04:52

torch.nn.TransformerDecoderLayer - Part 4 - Multiple Linear Layers and Normalization

nn.TransformerDecoderLayer - Overview 0:07:32

nn.TransformerDecoderLayer - Overview

torch.nn.TransformerEncoderLayer - Part 0:05:12

torch.nn.TransformerEncoderLayer - Part 5 - Transformer Encoder Second Layer Normalization

torch.nn.TransformerEncoderLayer - Part 0:06:12

torch.nn.TransformerEncoderLayer - Part 0 - Module Overview

sparse tensors, nn.TransformerEncoder 0:04:00

sparse tensors, nn.TransformerEncoder for language modeling in PyTorch

Pytorch for Beginners 0:12:02

Pytorch for Beginners #42 | Transformer Model: Implement Decoder

PyTorch Fundamentals Exercises 0:04:50

PyTorch Fundamentals Exercises and Extra Curriculum | Exercises & Extra Curriculum for Deep Lear...

torch.nn.Embedding - How 0:17:28

torch.nn.Embedding - How embedding weights are updated in Backpropagation

Torch.nn.Linear Module explained 0:07:04

Torch.nn.Linear Module explained

14 Writing Code 0:08:47

14 Writing Code for a PyTorch Training Loop

Pytorch for Beginners 0:23:03

Pytorch for Beginners #41 | Transformer Model: Implement Encoder

RTDETR v2 - 0:46:41

RTDETR v2 - Real Time Object Detection: Updates, algorithm and code reading

Pytorch for Beginners 0:10:36

Pytorch for Beginners #37 | Transformer Model: Masked SelfAttention - Implementation

5 Creating a 0:10:36

5 Creating a Python Script to Create Our PyTorch DataLoaders

Dive into Deep 1:31:05

Dive into Deep Learning: Coding Session#5 Attention Mechanism II (Americas/EMEA)

A Practical Guide 1:09:46

A Practical Guide to PyTorch for Machine Learning Beginners

Slaying OOMs with 0:49:38

Slaying OOMs with PyTorch FSDP and torchao

Programming for AI 0:31:07

Programming for AI (AI504, Fall 2023), Project 2 Announcement

Data Science Project 1:59:38

Data Science Project | Part 1 | Speech to Text using TensorFlow

Transformer Positional Embeddings 0:06:21

Transformer Positional Embeddings With A Numerical Example.

Reading RT DETR 0:46:39

Reading RT DETR source code

Reading DAB Detr 0:57:29

Reading DAB Detr source code

Комментарии
Автор

First Multihead Attention is masked, and the second is normal multihead attention. I think you forgot to mention this is masked cause the decoder behaves as an autoencoder which sends the output back into the output. During training as we will need to mimic this behaviour of inference/testing in training, therefore attention is basically the attention with mask.

Great videos overall!

kushagrasharma