Markov Decision Processes - Computerphile

Показать описание

Deterministic route finding isn't enough for the real world - Nick Hawes of the Oxford Robotics Institute takes us through some problems featuring probabilities.

This video was previously called "Robot Decision Making"

This video was filmed and edited by Sean Riley.

Рекомендации по теме

Комментарии

This guy was my lecturer about 10 years ago. He was very down to earth and explained the concepts in a really friendly way. Glad to see he's still doing it.

Deathhead

This channel makes me appreciate the human brain more. We do all that automatically with barely a moment's thought.

CalvinHikes

OMG as a Robotics student, I'm amazed how well explained that is. Love it <3

mateuszdziezok

Just took a RL course. Bellman equation and Markovian assumptions are so familiar. Btw, for those who are interested, the algorithm to solve discrete MDP (or model based RL problems in general) are Value Iterations and Policy Iterations, which are all based on Bellman equation.

tlxyxl

I made these decisions for my real commute. The train was fastest, but occasionally much longer. The car was fast, but the cost of parking equalled 2 hours of work, so was effectively slowest. The latest I could leave and be sure of being on time was walking.

gasdive

Where the formal definitions for concepts like MDP can get overwhelming, it really helps to have these easy to understand explanations

SachinVerma-lxbx

Nice one, I met Professor Nick at Pembroke College Oxford. It was an honour.

engineeringmadeasy

This was a fantastic simple explanation, very enlightening.

tobiaswegener

There is a 3% chance that, somewhere along the route, there's a half-duplex roadblock because they're fixing the overhead wires or something. There's a 0.1% chance that a power line or tree fell across the road, forcing you to take an extremely long detour, but half of the time this happens, you could get past it on a bike.

pierreabbat

I heared a lot about MDP and policy functions in the context of reinforcement learning. But this is the best explanation I ever heared.

Ceelvain

I'd like an autonomous taxi system that would decide it's all too hard to take me to the office, and would just take me back home, or, indeed, just refuse to take me to the office.
"Sorry, I"m working from home today because the car refused to drive itself."

cerealport

I rarely put a like on a video, but this one deserves it.
I definitely want to hear more about the algorithms to solve MDP problems.

Ceelvain

This is such a fascinating breakdown of Markov decision making. I love the mathematics that underpins Markov, but the creativity and imagination applied to the example and its host of solutions are delicious brain food.

tristanlouthrobins

MDP is the topic of my bachelorthesis and the example really helped understanding everything a lot better and I think I'll be using it throughout the thesis to understand the theory I have to write about. It's a lot easier to understand than some state a, b and c and action 1, 2, 3 :D

phil

the best explanation of this I've ever heard. many thanks.

elwood.downey

I literally had my final year project use a kalman filter to solve this problem. That's awesome!

Edit: spelling

asfandiyar

You can read passion in every word he is pronouncing. Very good explanation.

yvesamevoin

great video! Really well explained and interesting

BobWaist

Fascination look into decision-making.

lucrainville

So is there a way to compute the solutions? Like I assume some matrices show up. One for probabilities and one for the sum of times. Then you can multiply it and get different time distributions for every strategy?

Veptis

Markov Decision Processes - Computerphile

Markov Decision Processes - Computerphile

Markov Decision Process (MDP) - 5 Minutes with Cyrill

Section3 Markov Decision Processes MDPs II

Markov Decision Processes in Reinforcement Learning - Artificial Intelligence

First MDP Problem

Knowledge Graphs - Computerphile

10 1 Introduction to Markov Decision Process

How to solve problems with Reinforcement Learning | Markov Decision Process

MARKOV DECISION PROCESSES: POLICY ITERATION AND APPLICATIONS & EXTENSIONS OF MDPS

NPTEL: NPTEL: An Introduction to Artificial Intelligence -Markov Decision Process)

Example of calculation value function of Markov Decision Process

Ch(e)at GPT? - Computerphile

Chomsky Hierarchy - Computerphile

Introduction to the Markov Decision Process Software by SpiceLogic Inc.

Reinforcement Learning 2: Markov Decision Processes

Markov Decision Processes Two - Georgia Tech - Machine Learning

COMPSCI 188 - 2018-09-18 - Markov Decision Processes (MDPs) Part 1/2

Markov Decision Processes

Section 3 Worksheet Solutions: MDPs

010 Markov Decision Process

Markov decision problems

Model Based Reinforcement Learning: Policy Iteration, Value Iteration, and Dynamic Programming

Markov Decision Processes Three - Georgia Tech - Machine Learning

Unit- V Lecture 59 - Markov Decision Process