Tomaso Poggio: 2011 Allen Institute for Brain Science Symposium

Tomaso Poggio, Massachusetts Institute of Technology "The computational magic of the ventral stream: A theory"

Proposing a new theory of the way in which the visual system of humans and monkeys recognizes objects, Dr. Poggio combined mathematical reasoning with computational experimentation and neuroscience data to demonstrate that the architecture of the ventral stream, a.k.a. the "what" stream, in the visual system, is strategically organized to factor out the viewpoint of an object. That is, the job of the ventral system is to recognize an object independent of where it is located in the visual field (positional invariance), whether it is close up or far away (scale invariance) and whether it is rotated in the image plane (rotational invariance). This task is computationally much more demanding than identifying one particular upright object at the center of the visual field from a database of other upright and similar sized objects. His theory predicts that if the organism experiences a single object undergoing various affine transformations (e.g. scaling, in-plane rotation, moving about on the image plane) in early development, the system will be viewpoint invariant for such transformations for any other object, even though the object may be novel. That is, upon maturity the brain automatically combines visual information in such a way that objects are seen as invariant to transformations. Furthermore, this library of transformation can be implemented in a particularly ingenious way in which the system first removes location dependencies, then rotation and then scale dependencies, leading to the multi-level hierarchy observed in the ventral stream of the macaque monley. Above all, Dr. Poggio urged, "We need theories, not only models" to understand the complexity of neuroscience.