CES 2023 Canon Demonstrates Free ViewPoint for Movie Viewing

Free ViewPoint technology is similar to volumetric capture but has so far been applied to sports applications. The idea is to array a large number of cameras around an arena or stadium to be able to create virtual camera views from any postion in a dome around the action. The Brooklyn Nets and Clevland Caveliers basketball teams have such systems installed already in their arenas. The system is used for replay that allows the virtual camera to travel around a player to view the action from multiple perspectives.

At CES, Canon used the Free ViewPoint system to playback 4 different perspectives of a scene from filmmaker and director M. Night Shyamalan's new movie called "Knock at the Cabin". In the demo, users can view the sceen from the perspective of either charcter, from a cricket's perspective or a crow's perspective. It was pretty amazing that the camera arrays were able to create virtual images at the ground level of the cricket and top down view of the crow. The numner of cameras used and their position was not reveasled, but 100 arranged in a dome seems like a good guess.

Transmitting and processing 100 4K camera data is extremely challenging. To solve this, Canon places a pre-processor at each camera that analyzes the image to separte the foreground from the background. It does this purely using visual image processing - there is no depth camera to capture a point cloud. Only the foreground image is then transferred back to the server where additional image processing can be done to create a point cloud representation, virtual camera views, texture mapping and glide paths around the action.