filmov
tv
Direct Preference Optimization: Forget RLHF (PPO)
Показать описание
DPO replaces RLHF: In this technical and informative video, we explore a groundbreaking methodology called direct preference optimization (DPO) by Stanford Univ that has the potential to replace reinforcement learning in the training of GPT systems.
Join us as we dive into the intricacies of direct preference optimization, dissecting its technical details and highlighting its advantages over the conventional reinforcement learning approach.
Discover how this innovative technique opens new possibilities in AI training, offering more precise control and improved performance.
Direct Preference Optimization - DPO can fine-tune Language Models (LLMs) to align with human preferences as well as or better than existing methods. Notably, fine-tuning with DPO exceeds RLHF’s ability (Reinforcement Learning from Human Feedback) to control sentiment of generations and improves response quality in summarization and single-turn dialogue while being substantially simpler to implement and train.
All rights with authors of:
"Direct Preference Optimization: Your Language Model is Secretly a Reward Model"
Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, Chelsea Finn (Stanford Univ)
Join us as we dive into the intricacies of direct preference optimization, dissecting its technical details and highlighting its advantages over the conventional reinforcement learning approach.
Discover how this innovative technique opens new possibilities in AI training, offering more precise control and improved performance.
Direct Preference Optimization - DPO can fine-tune Language Models (LLMs) to align with human preferences as well as or better than existing methods. Notably, fine-tuning with DPO exceeds RLHF’s ability (Reinforcement Learning from Human Feedback) to control sentiment of generations and improves response quality in summarization and single-turn dialogue while being substantially simpler to implement and train.
All rights with authors of:
"Direct Preference Optimization: Your Language Model is Secretly a Reward Model"
Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, Chelsea Finn (Stanford Univ)
0:09:10
Direct Preference Optimization: Forget RLHF (PPO)
0:08:55
Direct Preference Optimization: Your Language Model is Secretly a Reward Model | DPO paper explained
0:19:38
Reinforcement Learning from Human Feedback (RLHF) & Direct Preference Optimization (DPO) Explain...
0:48:46
Direct Preference Optimization (DPO) explained: Bradley-Terry model, log probabilities, math
0:21:15
Direct Preference Optimization (DPO) - How to fine-tune LLMs directly without reinforcement learning
0:36:25
Direct Preference Optimization (DPO): Your Language Model is Secretly a Reward Model Explained
1:01:56
Direct Preference Optimization (DPO)
1:03:55
Towards Reliable Use of Large Language Models: Better Detection, Consistency, and Instruction-Tuning
0:53:03
DPO - Part1 - Direct Preference Optimization Paper Explanation | DPO an alternative to RLHF??
0:14:14
Direct Preference Optimization
0:37:12
PR-453: Direct Preference Optimization
0:42:49
Direct Preference Optimization (DPO)
0:36:14
How to Code RLHF on LLama2 w/ LoRA, 4-bit, TRL, DPO
0:45:21
How DPO Works and Why It's Better Than RLHF
0:41:21
DPO - Part2 - Direct Preference Optimization Implementation using TRL | DPO an alternative to RLHF??
0:17:27
The DPO debate: Do we need RL for RLHF?
0:06:18
4 Ways to Align LLMs: RLHF, DPO, KTO, and ORPO
0:54:29
CS 285: Eric Mitchell: Reinforcement Learning from Human Feedback: Algorithms & Applications
0:00:04
If you work in ML — You get it. RLHF is just so dang hard to remember the acronym! XD
0:31:04
Direct Preference Optimization Your Language Model is Secretly a Reward Model Stanford 2023
0:55:54
791: Reinforcement Learning from Human Feedback (RLHF) — with Dr. Nathan Lambert
1:22:20
Direct Preference Optimization: Your Language Model is Secretly a Reward Model (Ko/En Subtitles)
1:27:21
RLHF, PPO and DPO for Large language models
0:17:50
Proximal Policy Optimization Explained
Комментарии