IMMERSED IN: Markerless Tracking

In this IMMERSED seminar, MIT graduate students Jessica Rosendorf and Ritaank Tiwari present on markerless tracking.

ABSTRACT
Motion capture technology has a wide range of applications, including athletic performance, factory line efficiency, and injury rehabilitation. The gold standard technology is marker-based tracking, which requires extensive setup time, expensive equipment, and can only be used in a laboratory environment. On the other hand, markerless tracking is portable, inexpensive, and requires almost no setup time. This approach, which continues to be more widely used, combines camera calibration parameters and the 2D tracking of points to triangulate 3D coordinates from 2+ synchronized video cameras.

Another method for markerless tracking takes advantage of human body pose estimation. This is a rapidly developing field that uses deep learning to enable accurate and stable 3D shape reconstruction of human bodies simply from monocular video. Knowing the strengths and weaknesses of such pose estimation tools, and how to make best use of the extracted data, is key to unlocking their power.

ABOUT THE SPEAKERS
Jessica Rosendorf is a student in the MIT Mechanical Engineering SM/PhD program. She is an NSF Graduate Research Fellow working in the Device Realization Lab. Her research focuses on coordination and classifying human movements in the context of sports performance and injury. She graduated from Tufts University with a B.S. in Mechanical Engineering and Mathematics.

Ritaank Tiwari graduated from MIT with degrees in Computer Science (6-3) and Mathematics (18) in 2022. He is currently pursuing a Master of Engineering at MIT, concentrating in artificial intelligence. He is a current researcher of the MIT.nano Immersion Lab, where he works on generative modeling of human motion under Research Scientist Praneeth Namburi.

ABOUT IMMERSED
IMMERSED is a monthly seminar in which we explore the possibilities enabled by immersive technology and interactive experiences. Technologies such as motion capture, virtual and augmented reality, photogrammetry, and related computational advances bestow the power to gather, process, and interact with data from multiple modalities, providing unique insights and fostering interdisciplinary collaborations.

IMMERSED examines how immersive technology is shaping innovations across the sciences and the arts through a mix of lectures, demonstrations, and tutorials. IMMERSED is sponsored by the MIT.nano Immersion Lab, which provides space, tools, and a platform to connect scientists, engineers, artists, performers, and others through creative projects that bridge multiple disciplines.