Video for the ICRA 2021 paper: Visual Tracking of Deforming Objects Using Physics-based Models

"Visual Tracking of Deforming Objects Using Physics-based Models" - Agniva Sengupta, Alexandre Krupa, Eric Marchand
_________________________________________

In this paper, we propose a framework for tracking the deformation of soft objects using an RGB-D camera by utilizing the physically-based model of the considered object. A coarse, 3D template of the object being tracked is the only prior information required by the proposed method. The proposed approach does not rely on the accurate knowledge of the material properties of the object being tracked. In this paper, we integrate computer vision-based tracking methodology with a physical model-based deformation representation without requiring expensive numerical optimization for minimizing non-linear error terms. The proposed approach enables deformation tracking by joint minimization of a geometric error and a direct photometric intensity error while utilizing co-rotational FEM as the underlying deformation model. The proposed method has been validated both on synthetic data (with ground-truth) and real data.