AI Motion Capture - Track Your Hands & Body WITHOUT Bodysuit [FrankMocap]

VTubers and VRChat ppl are gonna be happy with this FrankMocap in the future lol. Hopefully it'll be available for everyone to use it easily.

Today's Sponsor is Infinite Red
Infinite Red consulting handles your mobile, web, and AI needs

This video is supported by the kind Patron:
🙏Mazen Alotaibi, Wampipti, Sascha Henrichs

Support me on Patreon if you hope to see more:

FrankMocap GitHub:

FrankMocap Project Page:

Motion Capture Video Sources:

[Music] Steaminwaffles - Stay By Me

[chenkevin]

"2 more papers down the line, we will be stonks forever" instant classic

[GuillermoValleCosmos]

as someone who tries to make AIs run for a living i strongly sympathized with the pain of making it work (specially in windows)

[thebrax]

I needed this in my reccomended, this is high quality content for less than 10k subs

[TheChristmasCreeper]

Good video but I kind of wish you included more information on how much fps it got on what hardware. I decided to look at the paper and found that whole body + hands tracking seems to get about 9.5fps on a GeForce RTX
2080 GPU.

[EddieBurke]

Who is Frank and why does he like Mocap so much?

[Imzantheoriginal]

I need that technology so I could making animation with it.

[zloboslav_]

whoa! and I've been struggling here with 10 years old kinect 360... lmao 🤣 thank you so much for sharing!
edit: can it do it live? or only on recording?

[jafaralmahbub]

Man I want this! This should be used for Daz3D models. Clip studio pose scanner still bad and lag, I hope there is easier way to use for artists who uses WIndows and Mac

[visualdistortionvfx]

This would be so promising if it only allowed itself to fix the errors as it analyzes the movement on the reference video, that is, imagine that you could hit the analyze button and move forward until you reach a frame where the arm goes crazy, you pause, you accommodate the arm in the frames that are necessary and then you analyze again

[tiagotiagot]

The glitchiness could probably be significantly reduced if the system takes in consideration physics (momentum, collisions etc) and the strength of the various muscles involved in each motion.

[DanIel-flvc]

Motion capture from video in an app on your computer would save hundreds of hours of animating. Any updates on where this technology is today or is it still something you run from a console and have to spend weeks to get working?

[DeusInVitro]

i dont know why im here but I enjoyed it

[ethanwilliamson]

I think that it would still be worth to include some references for the AI, such as colored gloves that it could use as reference points and able to tell when something goes behind the back

[ktmochiii]

yo i love these AI vids so much. keep em coming. love it.

[eclecticwierdo]

You know those scenes in horror movies where someone looking in a mirror and the suddenly the mirror version of them starts to move on it’s own and looks unappealing? I’d like to create a interactive art instillation that does that using motion capture. I wonder if this is something I could use to create that.

[GamerArg]

Dude, make your computer dual boot with linux, it's really easy and will really help you with these research papers

[HexOverride]

Is there a way to export the data to FBX

[MightyZeusXo]

This is good, maybe adding more camera from several angles will definitely give it a steroid

[sifsif]

Aight, I legit wanna get into Computer science in Uni just so I can create an AI with realistic movements that can be used like a companion(I guess Siri basically, just with a virtual body)

[FedbeliteTrailers]

show how to run it, please on windows