Do we need a 'cognitive architecture' debate again? - Marc Toussaint

Abstract:
With recent successes around LLMs and foundation models, perhaps we can think big again also in robotics, e.g. aim at systems for general purpose physical reasoning or robotic manipulation. But this somehow raises a question of system architecture, even if the system can be fine-tuned end-to-end. Some would see LLMs or foundation models right in the middle -- fine, but what is exactly around them?

While in recent years we worked top-down, considering high-level Task-and-Motion Planning (TAMP) problems first and then developing methods to combine TAMP solvers with perception and control, in this talk I will go bottom-up: I'll first discuss our take on appropriate representations for reactive manipulation (e.g. using field-based object representations to learn manipulation constraints and multi-object dynamics, as well as Sequence-of-Constraints MPC for control), and then discuss how higher-level systems could build on this -- be it our TAMP-style solvers or LLMs.

Bio: Marc Toussaint
I am professor for Intelligent Systems at TU Berlin since 2020, spend some time at MIT in 2017/18, some time with Amazon Robotics in 2017, and was professor for Machine Learning and Robotics at U Stuttgart since 2012. My research interests are in the intersection of AI and robotics, in particular combining Machine Learning, optimization, and AI reasoning to tackle fundamental research questions in robotics. Integrating learning and reasoning is of particular interest to me. Concrete research topics we work on are models and algorithms for physical reasoning, task-and-motion planning (logic-geometric programming), learning representations, control, and learning to transfer model-based strategies to reactive and adaptive real-world behavior. Some of my earlier work was on the planning-as-inference paradigm, relational reinforcement learning, active learning, and way back also on evolutionary algorithms, and black holes (my diploma was on gravity theory).