Realtime Full-Body Tracking via Webcam! | Google MediaPipe Pose, Unity, Open Source

Using Google MediaPipe Pose together with Unity game engine to display the pose of my body in real time. Used pipes to connect a Python program which did the machine learning part to Unity. Greatly optimized in comparison to my previous Google MediaPipeHands experiment.

Latest Update:

Code & Download:

Install Tutorial:

[nicolasportu]

Outstanding! Now the holly grail is that working with an avatar 😊

[Flugnu]

Cool ! Next step is skeleton retargeting ! Mediapipe is not made to be use directly as a C++ / C# library, great initiative to use a Python pipeline.

[peileed]

This is amazing! I'm so sad to see down in the comments that this does not work on Android, getting some Android body tracking AR is a nightmare, yet IOS seems to have such an easy solution

[lairanderson]

Nice Work. I dowloaded it and tried it, pretty good. But the transform position values of the landmarks in Unity are always incrementing, even when I'm not moving. Do you have any advice?

[MaxLohMusic]

The displayed camera image is flipped, which is misleading to the user, so I tried to comment out the line "image = cv2.flip(image, 1)". But then, the detected points became wrong (right became left). Where in the code does it tell the pose detection to handle taking a flipped image as input?

[santiagopalomino]

How do I make a build and turn it into a single file?

[jejore]

hi, could u share the step-by-step process of this full body tracking.

[RmuS-hpnz]

Good! 🔥🔥Im doing a similar proyect and right now im losted :s i dont know how to render the model inside of the webcam or just in front of the cam, the thing is that i need it into WebGL. Im using Python for MediaPipe and Flask for the server, it works fine, the problem is when i try to render the 3D model.

[kimkaye]

Have you manage to map the Skeleton to your live feed webcam?

[thehustler]

How to setup mediapipe unity plugin for unity

[larsmaas]

is there a way to make this work for MacOS?