Custom Hand Gesture Recognition with Hand Landmarks Using Google’s Mediapipe + OpenCV in Python

Hey what's up, y'all! In this video we'll take a look at a really cool GitHub repo that I found that allows us to easily train a Keras neural network to recognize our own custom hand gestures, and that runs flawlessly on a CPU. But there's more: I'll actually guide you through the entire process and explain the logic of how everything works behind the scenes too. We'll use Mediapipe for extracting hand landmarks (which runs a pipeline of TensorFlow neural nets under the hood), which will allow us to use relatively small training datasets for training on our new custom hand gestures.

Time stamps⏳
00:00 Intro
01:14 What's gonna be in the video
04:00 Top-level overview of the hand gesture recognition approach we'll use
14:56 Google's Mediapipe framework Python API
17:50 Hand gesture recognition GitHub repository we'll use
20:58 Modifying the repo's code to allow for multi-hand detection
25:23 Code walkthrough
32:46 Explaining hand landmarks preprocessing algorithm
48:05 Disabling point history classification
50:43 Training: hand gesture dataset
53:30 Training: adding a new hand gesture to the existing ones
01:02:43 Training: Retraining the model with all new hand gestures
01:09:51 Outro

▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
It would mean the world to me, if you decided to support me and the channel =)

►You may consider watching ads that show up on the videos

Making these videos takes a lot of time and effort, so If you decide to support me, please don't hesitate get in touch with me as I'd like to thank you personally!
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
Thanks for watching!

[IvanGoncharovAI]

Hey folks! This video was filmed weeks before the recent events in the region that I'm from, which almost gives a double-meaning to all the "peace sign" references in the video... I mean, I just really hope that things can normalize, and that if you or your loved ones are affected by it that you can stay safe and make it through. Lmk if I can help in any way, too.

[cesarams]

Hey! Just wanted to say thank yoy, because of your video we are making a project of universal hand gesture recognition applied in cobots, it´s really exciting and you really helped us out, greetings from Mexico! I hope you upload more videos soon

[CodeBrewCS]

One of the best and more enganging videos Ive watched on this topic ! Good stuff

[StefaanHimpe]

interesting tutorial... and with classification on the point history one should also be able to recognize movements like "swipe left", "swipe right", "clockwise turning" etc which makes the system even more useful

[jamesgalima]

I've developed image processing apps because of you, thank you for these tutorials

[ramanasai]

Kudos to you for explaining very clearly . I mean it, and kudos to the github repo owner as well. 👏👏👏

[itsmemattagain]

This video helped me greatly with a game I'm doing as i was planning to implement hand recognition to play it

[luisaurso]

Thanks for sharing this with us Ivan, very well explained and actually it will help me on a real use-case project I am working. Thanks a lot ! very well done.

[shireesh]

we are using numpad (0-9) to save each different gesture right.
can we only save 10 hand
how to save more than 10 gestures???

[ilonanietosvaara]

Hi Ivan! Such a great tutorial, I love that you really get into the details of the code and explain what is going on concretely. Perhaps a stupid question but is it possible to do transfer learning with the Mediapipe framework (for example by training it with images of hands with more keypoints added for more accuracy)?

[shawnloo]

Hi Ivan! I wanna ask why do I get an UnicodeDecodeError when I try to run the app.py? The error states that 'charmap' codec can't decode byte 0x81 in position 15718: character maps to <undefined>

[animehd]

Hi Ivan! Masterpiece video u have made here! Loved every part of it! Keep it going! Would love to see how you can build onto this and make it even better for real world application

[freenomon]

hey thanks for sharing. : ) Is there a way to get the wrist rotation data from this? I'm trying to retarget the finger motions (which I got it working somewhat), but I can't find a way to get rotation data of the hand itself.. (ex: hand facing palm up or down). any tips? thanks!

[ZINK]

Hi Ivan! This was a really helpful and amazing tutorial! The only question I have is how to change the model so that it can take more than 10 gestures. What/where/how should I change the code? Can you explain?

[bceg]

Thanks, I needed exactly that for my project, I'm working on a Libras translator, which is the Brazilian sign language.

[lolopupka]

Hi Ivan I was researching to make the same thing as your student! I want to make a sign language learning app as well, for fun. If your student completed the project, could you share a link? I would love to see what a finished project could look like.

[Swetha-lm]

SOME IMPORTANT KEYPOINTS:
- Custom hand gesture recognition using Google's Mediapipe and OpenCV in Python is explored in-depth.
- Two primary approaches, pixels-first and landmarks-first, are discussed for hand gesture recognition.
- The necessity of machine learning or deep learning for recognizing complex hand gestures is emphasized.
- Mediapipe simplifies hand detection and landmark extraction through its pre-trained models.
- Steps like pre-processing hand landmarks, converting to relative coordinates, and training models are detailed.
- Training data stored in dot cc files, normalization of values, and Jupyter notebooks for classification are highlighted.
- The video serves as a comprehensive guide for developing a reliable hand gesture recognition system.

[alankuo]

20:00 i tried running app.py in vscode yet it didn't work, giving me this error: ValueError: Could not open
any idea why?
Thank you so much!

[FaisalKhan-gxeg]

20:01 after running my camera window is not open in this code i tried it with basic code it is working there ... plz help

[zombiesoul]

thank you for your great video ... I have questions how I can use my existing dataset Image for labelling and training ?