Reinforcement Learning through Human Feedback - EXPLAINED! | RLHF

Показать описание

We talk about reinforcement learning through human feedback. ChatGPT among other applications makes use of this.

ABOUT ME

PLAYLISTS FROM MY CHANNEL

MATH COURSES (7 day free trial)

OTHER RELATED COURSES (7 day free trial)

Рекомендации по теме

Комментарии

At 6:58, you have an error: PPO is not used to build the reward model.

theartofwar

Great video! I have a few questions:

1) Why do we need to manually train the reward model with human feedback if the point is to evaluate responses of another pretrained model? Can't we just cut out the reward model altogether, rate the responses directly using human feedback to generate a loss value for each response, then backpropagate on that? Does it require less human input to train the reward model than to train the GPT model directly?

2) When backpropagating the loss, do you need to do recurrent backpropagation for a number of steps that is the same as the length of the token output?

3) Does the loss value apply equally to every token that is output? Seems like this would overly punish some words e.g. if the question starts with "why" it's likely the response is going to start with "because" regardless of what comes after. Does RLHF only work with sentence embeddings rather than word embeddings?

neetpride

Brilliant Bro 👌. Excellent explanation. I never understand RLHF reading so many books and notes. Your examples are GREAT & simple to understand 👌
I am new to your channel and subscribed.

RameshKumar-ngnf

Sir, please make a video on function approximation in RL

sangeethashowrya

what about the generation of rewards, will there be another model to check the relativity of the answer and the precision of the answer, cause we have a lot of data

manigoyal

(1) supervised fine-tuning (SFT), (2) reward model (RM) training, and (3) reinforcement learning via proximal policy optimization (PPO) on this reward model explain me

thangarajr-qwwy

Acts as a randomizing factor depending on whom you are getting feedback from

manigoyal

haha quiz time again:

0) when the person knows me well
1)D
2)B if proper human feedback
3)C

xabaki

Aren't we users are the humans in feedback loop for openai

manigoyal

looking like indian but accent like britisher, where u from bro ?

harshsahu

The video is informative and good. but stop saying quiz time in an annoying way

aswinselva

Reinforcement Learning through Human Feedback - EXPLAINED! | RLHF

Reinforcement Learning from Human Feedback (RLHF) Explained

Reinforcement Learning through Human Feedback - EXPLAINED! | RLHF

Reinforcement Learning from Human Feedback Explained (and RLAIF)

What is Reinforcement Learning through Human Feedback (RLHF)?

Reinforcement Learning with Human Feedback - How to train and fine-tune Transformer Models

Reinforcement Learning from Human Feedback: From Zero to chatGPT

The Magic of Reinforcement Learning with Human Feedback RLHF

Reinforcement Learning from Human Feedback (RLHF) Explained

Reinforcement Learning from Human Feedback explained with math derivations and the PyTorch code.

RLHF+CHATGPT: What you must know

RLHF - Reinforcement Learning from Human Feedback

CS 285: Eric Mitchell: Reinforcement Learning from Human Feedback: Algorithms & Applications

15min History of Reinforcement Learning and Human Feedback

Reinforcement Learning: ChatGPT and RLHF

New course with Google Cloud: Reinforcement Learning from Human Feedback (RLHF)

Reinforcement Learning from Human Feedback (Natural Language Processing at UT Austin)

John Schulman - Reinforcement Learning from Human Feedback: Progress and Challenges

Reinforcement Learning Explained: Correcting models with feedback

What is RLHF (or reinforcement learning from human feedback)

Reinforcement Learning from Human Feedback (RLHF) - Beginners Guide | AI Foundation Learning

Learning Task Specifications for Reinforcement Learning from Human Feedback | David Lindner

RLHF: How to Learn from Human Feedback with Reinforcement Learning

Reinforcement Learning from Human Feedback (RLHF)

AI TeaTalk Singapore #1: Learn from Human Feedback with Reinforcement Learning - Natasha Jaques