Qianxiao Li | A mean-field optimal control formulation of deep learning

Speaker: Qianxiao Li, National University of Singapore

Tile: A mean-field optimal control formulation of deep learning

Abstract: In this talk, we discuss formulating, through a continuous-time approximation, deep supervised learning as a mean field optimal control problem. This allows us to derive necessary conditions for optimality in deep learning in the form of a mean-field Pontryagin’s maximum principle, as well as global characterizations of optimality using Hamilton-Jacobi Bellman equations. This forms a connection between deep learning on the one hand, and partial differential equations and the calculus of variations on the other. We also discuss interesting numerical algorithms and generalization estimates that can be derived from this viewpoint, as well as some results on function approximation using flows of dynamical systems.