Using a Single RGB Frame for Real Time 3D Hand Pose Estimation in the Wild (IEEE WACV 2018)

We present a method for the real-time estimation of the full 3D pose of one or more human hands using a single commodity RGB camera. Recent work in the area has displayed impressive progress using RGBD input. However, since the introduction of RGBD sensors, there has been little progress for the case of monocular color input. We capitalize on the latest advancements of deep learning, combining
them with the power of generative hand pose estimation techniques to achieve real-time monocular 3D hand pose estimation in unrestricted scenarios. More specifically, given an RGB image and the relevant camera calibration information, we employ a state-of-the-art detector to localize hands. Given a crop of a hand in the image, we run the pretrained network of OpenPose for hands to estimate the 2D location of hand joints. Finally, non-linear least-squares minimization fits a 3D model of the hand to the estimated 2D joint positions, recovering the 3D hand pose. Extensive experimental results provide comparison to the state of the art as well as qualitative assessment of the method in the wild.

Contributors:
Paschalis Panteleris, Iason Oikonomidis, Antonis Argyros

[sandeeproy]

awesome it is analysizing the depth perfectly

It would be very interesting to see the other real-time hand pose estimated footage, such as "The Matrix", as shown in the paper.

[ataadevs]

1:00 are you not detecting hand when it's close to the head or it is just the detector?

[alexlin]

Any source code available for public? Thanks

[iDineshY]

Amazing, any helpful links for following this?

[sandeeproy]

does this is done by structured lights

[adityabastapure]

Hello Sir, I am undergraduate student and want to do pose estimation of hand joints for my project. I am new to the field of Machine Learning but I need to apply it. Is there any github link of your work?

[yatrikchauhan]

which camera or sensors have been used for this 3d hand pose estimation?