Canonicalization: Experiments in setting some chairs in order - Prof' Srinath Sridhar

In this talk, I will introduce the notions of invariance, equivariance, and 'object canonicalization' (i.e., mapping object properties to canonical states). I will demonstrate how canonicalization enables us to better solve tasks in computer vision and robotics including 6DoF object pose estimation and 3D reconstruction. I will discuss our work on fully supervised and weakly supervised canonicalization methods, but focus on self-supervised methods. Finally, I will discuss future directions including opportunities for using canonicalization to understand articulated and non-rigid objects.

00:00 Intro
05:09 Canonicalization
07:52 Canonicalization in 2D Computer Vision
10:55 Canonicalization in 3D Computer Vision
11:40 3D Datasets: Explicit Canonicalization
16:20 Related Work
17:58 Supervised Canonicalization
19:37 NOCS: Normalized Object Coordinate System
21:52 6 DoF Object Pose Estimation
22:51 NOCS Maps
25:25 Predicting NOCS Maps
27:56 Qualitative Results: Real Data
31:11 X-NOCS: Network Architecture
35:02 Pix2Surf: Single-View Single-Chart
36:45 NOCS Applications
37:45 Temporal NOCS (T-NOCS)
39:17 Canonicalization Results
41:21 NOCS: Normalized Object Coordinate Space
42:03 Articulated NOCS
43:39 Limitations
46:28 Self-Supervised Canonicalization
47:53 (1) Permutation Equivariance
49:05 (2) Rotation Equivariance
50:27 ConDor: Self-Supervised Canonicalization of 3D Pose for Partial Shapes
51:09 Rotation Canonicalization
53:10 Translation Equivariance
56:15 Summary
01:00:25 Neural Fields
01:04:07 Discussion

[Chapters were auto-generated using our proprietary software - contact us if you are interested in access to the software]

References:
- Slides

​The talk is based on the speaker's papers:

​Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimation (CVPR 2019)

​Multiview Aggregation for Learning Category-Specific Shape Reconstruction (NeurIPS 2019)

​Pix2Surf: Learning Parametric 3D Surface Models of Objects from Images (ECCV 2020)

​CaSPR: Learning Canonical Spatiotemporal (NeurIPS 2020 Spotlight)

​DRACO: Weakly Supervised Dense Reconstruction And Canonicalization of Objects

​ConDor: Self-Supervised Canonicalization of 3D Pose for Partial Shapes

​

​Presenter Bio:

-------------------------
Find us at:

[ddai]

Would love to hear any comment or input you have. Did you find the content interesting? Is there a place for improvement? Please leave any feedback you have here in a comment :)