Yoshua Bengio – Cognitively-inspired inductive biases for higher-level cognition

Yoshua Bengio – Cognitively-inspired inductive biases for higher-level cognition and systematic generalization

How can deep learning be extended to encompass the kind of high-level cognition and reasoning that humans enjoy and that seems to provide us with stronger out-of-distribution generalization than current state-of-the-art AI? Looking into neuroscience and cognitive science and translating these observations and theories into machine learning, we propose an initial set of inductive biases for representations, computations and probabilistic dependency structure. The Global Workspace Theory in particular suggests an important role for a communication bottleneck through a working memory, and this may impose a form of sparsity on the high-level dependencies. These inductive biases also strongly tie the notion of representation with that of actions, interventions and causality, possibly giving a key to stronger identifiability of latent causal structure and ensuing better sample complexity in and out of distribution, as well as meta-cognition abilities facilitating exploration that seeks to reduce epistemic uncertainty of the underlying causal understanding of the environment.

[theabrusch]

Such a great and inspirational talk! Thanks a lot for posting! Does anyone know whether the slides are available for download somewhere?