Using FFMPEG to Encode/Decode Video for Offline Video-Based Machine Learning

FFMPEG is a tool that can be used to both decode and encode video between formats such as .MP4/.MOV to/from individual JPEG images. This technique allows machine learning programs to process the individual frames of a video and even generate entirely new videos.

Here are the three notebooks from this video:

[joliver]

This is awesome stuff. I always found ffmpeg difficult to use so thank you for this very clear walkthrough. You’re a great teacher and I always look forward to your videos.

[MawaMaverick]

You actually did a video on this. Excited to watch

[davidgeorge]

Hey! Waiting for the live video one, great video

[luisilabaca]

amazing work always, every video and lecture man.
pleasee more of this topic

[MawaMaverick]

And jeff gave this to us for free. Amazing

[MIGuy]

enjoyed ... looking forward to more ... Be safe!

[jokosusilo]

Hi. Jeff, thanks a lot for posting the AI tutorial. It helps a lot for a beginner like me to learn AI. I have a question, how do we crop images inside of the bounding box of YOLOV3 when the input is a video? So in the end, we have images of each class detected in the video. Would you please make a tutorial about it? Thanks

[efstathiasoufleri]

Hi, thank you for the video. I would like to ask you if there is some way to access the DCT coefficients from a compressed video?

[quisaldaoud]

Thank you for wonderful presentation, Is there a way to start from a user inserted image and begin the morphing from there?

[OkolieRaymond]

I have a setup where I'm using ffmpeg for encoding and transcoding for live streams but presently experiencing lots of latency despite deployment of CloudFront et all.

Is there a way i can set up ffmpeg to give better performance which in turn will help reduce latency?

[kenchang]

Hi Jeff, what about using ML to strip out a lead singer's voice from an MP3 file to make a karaoke track? As sound is time-series data, I thought it would be interesting but I have no idea if that is possible or if it's too much of a heavy lift for Youtube. I thought I read somewhere that that was possible.

[FMH]

i have a video of a liquid being heated, is there a way to make a neural network that if given a couple of seconds it will predict the next few seconds? will you be working on something similar in this series?

[frazierkok]

can we use ffmpeg for online realtime video processing also?

[bechti]

Hello Jeff and community,
I am currently using ffmpeg to decode/extract frames from video. It... just works. But its heck slow.
Im having a wrapper where I can use the function GerFrame() which basically pulls the next frame. It is a little faster than watching it in media player but at this rate, it will take 50 Minutes to process 1h of video into images. Is that normal? If so I have to look at other methods than ffmpeg. Apparently, I can watch a video at 4x speed within VLC mediaplayer. So it should not be an issue by hardware speed?
LG,
Julian

[ayandhar]

Can You please say how can I convert mp4 video file frame by frame to json format using ffmpeg or any source code is available?

[krantikumar]

It's amazing. Can you please build the model to translate speech from one language to another in video.

[kenchang]

Thanks for this video. I'm interested in CV but NLP is more practical for now, don't you think?!

[RedShipsofSpainAgain]

Hi Jeff,


How would we run this locally rather than on Google Colab? Specifically the command:


from google.colab import files
uploaded = files.upload()


Thank you Jeff!

[Lalancelot]

You talk about real time video processing with open CV and augmented reality. I would really like to see a video about AI following an objet on a (phone) screen and providing its position as fast as possible. It's for video games. Thanks.

[BradHelm]

"I like no work on my part." LOL! I've always told my students that we programmers are lazy - not in that we won't do what needs doing, but that we try to do it with a goal of maximizing functionality while minimizing effort. After ll, those cat videos won't watch themselves! :-)