LLM Foundations (LLM Bootcamp)

preview_player
Показать описание
New course announcement ✨

We're teaching an in-person LLM bootcamp in the SF Bay Area on November 14, 2023. Come join us if you want to see the most up-to-date materials building LLM-powered products and learn in a hands-on environment.

Hope to see some of you there!

--------------------------------------------------------------------------------------------- In this video, Sergey covers the foundational ideas for large language models: core ML, the Transformer architecture, notable LLMs, and pretraining dataset composition.

00:00 Intro
00:47 Foundations of Machine Learning
12:11 The Transformer Architecture
12:57 Transformer Decoder Overview
14:27 Inputs
15:29 Input Embedding
16:51 Masked Multi-Head Attention
24:26 Positional Encoding
25:32 Skip Connections and Layer Norm
27:05 Feed-forward Layer
27:43 Transformer hyperparameters and Why they work so well
31:06 Notable LLM: BERT
32:28 Notable LLM: T5
34:29 Notable LLM: GPT
38:18 Notable LLM: Chinchilla and Scaling Laws
40:23 Notable LLM: LLaMA
41:18 Why include code in LLM training data?
42:07 Instruction Tuning
46:34 Notable LLM: RETRO
Рекомендации по теме
Комментарии
Автор

So far this has been the simplest brief explanation of transformers I have seen. Not complete though but goos starting point.

amirmohammadi
Автор

This is the best explanation so far for Language Model and its parameters. The speaker though didn't attributed Google for their Attention is all you need white paper.

USONOFAV
Автор

Amazing explanation of Transformers! And everything else... LOVED the talk! Thank you so much, Sergey!

oleksandrasaskia
Автор

As a data scientist prepping to retrain a transformer for a classification task at work, this is so helpful in refreshing what I learned in school!

andersw
Автор

obviously you need a background to understand these stuff but for an informed audience it is a nice refresher

muratcan__
Автор

This was amazing! As a "neophyte" to the field of AI and LLMs, highly recommend this for foundational knowledge and perspective.

robertcormia
Автор

really cool content. the part about training the models on code enhancing non-code capabilities is mind blowing. congrats!

allanrp
Автор

Sound. Very small
Subtitle. Will help

kocokan
Автор

Pls use AI to enhance this audio tnx. Either way, thanks for the vid!

ScientiaFilms