Naftali Tishby: The Synergy between Information and Control

Information, Control, and Learning: The Ingredients of Intelligent Behavior.

September 2016, Jerusalem

Abstract:
The link between communication and control was obvious to Shannon, as evident from the conclusion of his 1959 paper on the duality between source and channel coding. Shannon noticed then the analogy between source coding and inference from past observations, and between channel coding and predictions of future events. Regrettably, the two fields grew apart, partly due to their different mathematical languages, the asymptotic (long delays) nature of Shannon’s optimal coding versus the instantaneous nature of dynamical systems and optimal control. Yet, these two fundamental abstractions naturally merge when considering the joint-source-channel coding problem, with memory and feedback. The general distortion-cost tradeoff in communication can be compared, a-la Shannon, to the fundamental sensing-costs – actions-value tradeoff in optimal control and planning. The two theories begin to unify by considering optimal control under informational costs, as can be seen in several talks of this meeting. This unified view of information and control, when allowing adaptive channels and unspecified sources, connects directly with the fundamental problems of brain sciences and with the new computational methodologies of machine learning.
In this talk, I will sketch the intriguing analogies and synergy between communication and control and state some of the problems with their desired unification, as well as the main issues addressed in this meeting. I will then focus on the role of machine learning as a powerful framework for implementing this unification and applying it to the understanding of biological systems