comma ai | Navigate on openpilot | Mitchell Goff | COMMA_CON talks | ML Engineer | HQ version

Date of the video 29 Jul 2023.

Mitchell Goff social media:
View original video on Youtube:
Connect subscription for navigation:

Chapters:
00:00:00 intro
00:00:30 navigate on openpilot
00:01:46 how it works
00:02:00 OpenStreetMap, overpass, valhalla, mapbox GL
00:02:20 how it works: the map
00:02:55 mapbox GL
00:03:15 maps to draw
00:04:12 how it works: the route
00:05:05 map matching
00:05:45 valhalla to snap to OSM graph
00:06:05 how it works: the model
00:06:15 need to squish down the maps
00:06:38 compressing the camera videos with neural network
00:07:40 how it works: the model autoencoder
00:07:55 encoder and decoder
00:08:55 reconstructed map
00:10:02 it works pretty well
00:11:05 deploying to comma three x
00:11:15 mapsd, navmodeld, modeld
00:12:20 taco bell video
00:12:44 the devil is in details
00:13:02 map scale isn't constant
00:13:38 earth is a sphere, mercator projection
00:14:45 dynamically adjust the zoom level of the map
00:15:07 routes are TOO accurate
00:15:30 model gets confused if it misses an exist
00:16:10 fix by training on small percentage of fake routes
00:17:43 tests, tests, and more tests
00:18:12 no testing in cars
00:19:15 offline tests
00:19:55 what should we test for
00:21:45 shipped in openpilot 0.9.4 in experimental model
00:22:18 q&a
00:22:40 speed limits as a model input
00:24:25 future input for real time traffic
00:24:35 mapbox traffic tracking system
00:25:14 text based navigation as input
00:26:02 vector maps usage
00:26:25 mapbox vector tiles
00:27:15 navigation crashes after one minute
00:27:45 showing map direction in the view
00:28:39 map encoder how it's trained
00:29:00 running map model on dsp (quantized)
00:30:05 video and map data what should the driving agent follow
00:31:10 fake routes, trusting the video

Official communication channels:

How to get a job:
How to collaborate:

Are you interested in openpilot?
Knowledge base:
Check out the code:
Is my car supported?
Frequently Asked Questions:
How to setup openpilot:
Comma Secure Shell:
API Documentation:
CAN analysis tool:
Review and annotate your driving data:
Leaderboard:
Comma Connect App:
Research:

Official George Hotz communication channels:

Follow for notifications:
Unofficial communities and resources:

Thank you for reading and using the SHOW MORE button.
We hope you enjoy watching George's videos as much as we do.
See you at the next video.

[geohotarchive]

Thank you to the comma team for this amazing event, sharing the livestream and high-quality videos. All credits to them. Follow their official youtube.com/commaai for more livestreams.
Connect subscription for navigation -> comma.ai/connect | from $1250 buy -> comma 3X comma.ai/shop/comma-3x | openpilot.comma.ai is the best ADAS system in the world | If you have skill apply on -> comma.ai/jobs | Join the community on github.com/commaai/openpilot & discord.comma.ai | Check if your car is supported -> comma.ai/vehicles | How to setup your car comma.ai/setup | Stay up to date by following twitter.com/comma_ai | Support comma.ai at comma.ai/shop
Chapters:
00:00:00 intro
00:00:30 navigate on openpilot
00:01:46 how it works
00:02:00 OpenStreetMap, overpass, valhalla, mapbox GL
00:02:20 how it works: the map
00:02:55 mapbox GL
00:03:15 maps to draw
00:04:12 how it works: the route
00:05:05 map matching
00:05:45 valhalla to snap to OSM graph
00:06:05 how it works: the model
00:06:15 need to squish down the maps
00:06:38 compressing the camera videos with neural network
00:07:40 how it works: the model autoencoder
00:07:55 encoder and decoder
00:08:55 reconstructed map
00:10:02 it works pretty well
00:11:05 deploying to comma three x
00:11:15 mapsd, navmodeld, modeld
00:12:20 taco bell video
00:12:44 the devil is in details
00:13:02 map scale isn't constant
00:13:38 earth is a sphere, mercator projection
00:14:45 dynamically adjust the zoom level of the map
00:15:07 routes are TOO accurate
00:15:30 model gets confused if it misses an exist
00:16:10 fix by training on small percentage of fake routes
00:17:43 tests, tests, and more tests
00:18:12 no testing in cars
00:19:15 offline tests
00:19:55 what should we test for
00:21:45 shipped in openpilot 0.9.4 in experimental model
00:22:18 q&a
00:22:40 speed limits as a model input
00:24:25 future input for real time traffic
00:24:35 mapbox traffic tracking system
00:25:14 text based navigation as input
00:26:02 vector maps usage
00:26:25 mapbox vector tiles
00:27:15 navigation crashes after one minute
00:27:45 showing map direction in the view
00:28:39 map encoder how it's trained
00:29:00 running map model on dsp (quantized)
00:30:05 video and map data what should the driving agent follow
00:31:10 fake routes, trusting the video

[crassflam]

This is epic! Thanks so much for sharing this!

[furkansevindi]

Charquell den geliyom cidden harika işler başarmışsın ve her şey için teşekkürler

[arif]

Vay be Charmquel olmasa ps cafelerde eğlenmemizi sağlayan adamı bilmezdik thank you geohot❤

[commonlogic]

Yo did you record the Twitter meeting George joined previously?

[sarath_sajan]

Reminds me of Sentdex's AI plays GTA 5 video from back in 2018

[sarath_sajan]

How will the model behave should the map rendered from the MapBox API change later on?

[Krishtafa]

Why not represent the original map as vector graphics? Isn’t that already the basis for the bitmap anyway, and already much more compressed?

[sashoroma]

hey george can you suggest few projects to make for someone studying cs

[oscarmoreno]

30+ minutes to tell us that you will now start pre-mapping.😉 Face it... Waymo was right all along. Tesla will eventually come around to pre-mapping as well once they find pure vision is just going to be a decade or more away.