filmov
tv
Building LLMs from the Ground Up: A 3-hour Coding Workshop
Показать описание
REFERENCES:
DESCRIPTION:
This tutorial is aimed at coders interested in understanding the building blocks of large language models (LLMs), how LLMs work, and how to code them from the ground up in PyTorch. We will kick off this tutorial with an introduction to LLMs, recent milestones, and their use cases. Then, we will code a small GPT-like LLM, including its data input pipeline, core architecture components, and pretraining code ourselves. After understanding how everything fits together and how to pretrain an LLM, we will learn how to load pretrained weights and finetune LLMs using open-source libraries.
---
---
---
OUTLINE:
0:00 – Workshop overview
2:17 – Part 1: Intro to LLMs
9:14 – Workshop materials
10:48 – Part 2: Understanding LLM input data
23:25 – A simple tokenizer class
41:03 – Part 3: Coding an LLM architecture
45:01 – GPT-2 and Llama 2
1:07:11 – Part 4: Pretraining
1:29:37 – Part 5.1: Loading pretrained weights
1:45:12 – Part 5.2: Pretrained weights via LitGPT
1:53:09 – Part 6.1: Instruction finetuning
2:08:21 – Part 6.2: Instruction finetuning via LitGPT
02:26:45 – Part 6.3: Benchmark evaluation
02:36:55 – Part 6.4: Evaluating conversational performance
02:42:40 – Conclusion
DESCRIPTION:
This tutorial is aimed at coders interested in understanding the building blocks of large language models (LLMs), how LLMs work, and how to code them from the ground up in PyTorch. We will kick off this tutorial with an introduction to LLMs, recent milestones, and their use cases. Then, we will code a small GPT-like LLM, including its data input pipeline, core architecture components, and pretraining code ourselves. After understanding how everything fits together and how to pretrain an LLM, we will learn how to load pretrained weights and finetune LLMs using open-source libraries.
---
---
---
OUTLINE:
0:00 – Workshop overview
2:17 – Part 1: Intro to LLMs
9:14 – Workshop materials
10:48 – Part 2: Understanding LLM input data
23:25 – A simple tokenizer class
41:03 – Part 3: Coding an LLM architecture
45:01 – GPT-2 and Llama 2
1:07:11 – Part 4: Pretraining
1:29:37 – Part 5.1: Loading pretrained weights
1:45:12 – Part 5.2: Pretrained weights via LitGPT
1:53:09 – Part 6.1: Instruction finetuning
2:08:21 – Part 6.2: Instruction finetuning via LitGPT
02:26:45 – Part 6.3: Benchmark evaluation
02:36:55 – Part 6.4: Evaluating conversational performance
02:42:40 – Conclusion
2:45:10
Building LLMs from the Ground Up: A 3-hour Coding Workshop
0:35:45
How to Build an LLM from Scratch | An Overview
1:56:20
Let's build GPT: from scratch, in code, spelled out.
0:07:54
How ChatGPT Works Technically | ChatGPT Architecture
1:21:22
LLM Pretraining Course: Build LLMs From Ground Up
0:18:40
But what is a neural network? | Chapter 1, Deep learning
0:05:03
Ep 26. Build an Open Source LLM RAG for Your Code - Ground Crew
0:00:59
Training LLMs from Scratch #llmwithav #learnwithav #llm #datascience
0:01:13
Best Budget-Friendly University for LLM in the UK | CEO's Top Pick
1:30:16
Build Your Own LLM from Scratch: Step by Step
0:08:47
What are Generative AI models?
1:02:12
How to Build, Evaluate, and Iterate on LLM Agents
0:41:18
Zero to Hero - Develop your first app with Local LLMs on Windows | BRK142
0:35:21
Lessons From A Year Building With LLMs
0:03:03
Generative floor plan design using AI- Maket.ai #ai #architecture
0:08:33
What is Prompt Tuning?
0:07:12
Ep 8. Unexpected Skills Needed for LLM Development
0:00:59
Comparing Different Methods of Using an LLM #llmwithav #learnwithav #llm #datascience
1:22:15
Lets build an LLM from Scratch
0:28:18
Fine-tuning Large Language Models (LLMs) | w/ Example Code
0:59:35
Building with Instruction-Tuned LLMs: A Step-by-Step Guide
0:06:42
Knowledge Graph Construction Demo from raw text using an LLM
0:49:11
LLMOps (LLM Bootcamp)
0:08:49
5 Commandments of Building LLM Products
Комментарии