#85 Dr. Petar Veličković (Deepmind) - Categories, Graphs, Reasoning [NEURIPS22 UNPLUGGED]

Dr. Petar Veličković is a Staff Research Scientist at DeepMind, he has firmly established himself as one of the most significant up and coming researchers in the deep learning space. He invented Graph Attention Networks in 2017 and has been a leading light in the field ever since pioneering research in Graph Neural Networks, Geometric Deep Learning and also Neural Algorithmic reasoning. If you haven’t already, you should check out our video on the Geometric Deep learning blueprint, featuring Petar. I caught up with him last week at NeurIPS. In this show, from NeurIPS 2022 we discussed his recent work on category theory and graph neural networks.

TOC:
Categories (Cats for AI) [00:00:00]
Reasoning [00:14:44]
Extrapolation [00:19:09]
Ishan Misra Skit [00:27:50]
Graphs (Expander Graph Propagation) [00:29:18]

Host: Dr. Tim Scarfe

References

MLST#60 Geometric Deep Learning Blueprint (Special Edition)

Categories for AI
Organised by:
Andrew Dudzik - DeepMind
Bruno Gavranović - University of Strathclyde
João Guilherme Araújo - Cohere / Universidade de São Paulo

Petar Veličković - DeepMind / University of Cambridge
Pim de Haan - University of Amsterdam / Qualcomm AI Research

[Petar Veličković] Graph Attention Networks

Learning to Configure Computer Networks with Neural Algorithmic Reasoning [NeurIPS 2022] [Luca Beurer-Kellner, Martin Vechev, Laurent Vanbever, Petar Veličković]

Graph Neural Networks are Dynamic Programmers [Andrew Joseph Dudzik, Petar Veličković]

Expander Graph Propagation [Andreea Deac, Marc Lackenby, Petar Veličković]

[Pim de Haan, Taco Cohen, Max Welling] Natural Graph Networks

[Uri Alon, Eran Yahav] On the Bottleneck of Graph Neural Networks and its Practical Implications (they discovered oversquashing)

[Topping,...,Bronstein] Understanding over-squashing and bottlenecks on graphs via curvature

[Andreea Deac, Petar Velickovic, Ognjen Milinkovic et al] XLVIN: eXecuted Latent Value Iteration Nets

[Petar Veličković et al] Reasoning-Modulated Representations (RMR)

Dual Algorithmic Reasoning [PetarV, under review ICLR]

[Petar Veličković, Charles Blundell] Neural Algorithmic Reasoning

[Andreea Deac, Petar Veličković, ...] Neural Algorithmic Reasoners are Implicit Planners (which got a NeurIPS spotlight in 2021!)

A Generalist Neural Algorithmic Learner

ETA Prediction with Graph Neural Networks in Google Maps

[Randall Balestriero] A Spline Theory of Deep Networks

[Ahmed Imtiaz Humayun ] Exact Visualization of Deep Neural Network Geometry and Decision Boundary

[Ahmed Imtiaz Humayun ] MaGNET: Uniform Sampling from Deep Generative Network Manifolds Without Retraining | ICLR 2022

[Randall Balestriero, Jerome Pesenti, Yann LeCun] Learning in High Dimension Always Amounts to Extrapolation

[Hattie Zhou] Teaching Algorithmic Reasoning via In-context Learning

[Ahmed Imtiaz] Exact Visualization of Deep Neural Network Geometry and Decision Boundary

[Beatrice Bevilacqua] Size-Invariant Graph Representations for Graph Classification Extrapolations

[Brendan Fong David I. Spivak] Seven Sketches in Compositionality: An Invitation to Applied Category Theory

Tim’s examples of applied Category theory cited:

[Lennox] Robert Rosen and Relational System Theory: An Overview

[Bob Coecke] Introducing categories to the practicing physicist

[Bob Coecke] Categorical Quantum Mechanics I: Causal Quantum Processes

[Bob Coecke] Quantum Natural Language Processing

[phamtheanhphu]

Really admire for your contributions on GNN❤

[DavenH]

This man does not need any time to think!

[ZachDoty]

This interview was fabulous! Thank you!

[kubricksghost]

This video is definitely fast forwarded

[bluewhale]

So much great ideas.I want new sorting algorithm.Amazing!

[asdf_]

Let me tell you, i love this Guy but he just scooped m’y master thesis topic i was working on yesterday 😢

[olegt]

Petar is amazing. How can ai reasoning help to solve complex social problems like: degrowth society, thesimplerway, lonelyness, poverty, self sufficient society, depressions, family breakup?

[johntanchongmin]

Interesting idea to use an expander graph to alleviate information bottleneck. Would the expander graph cause an exponential increase in the number of nodes required though?

[shiva_kondapalli]

What's your take on Algebraic geometry and categories for Deep learning, especially since the fact that categories were developed quite a bit for AG.

[vivekpetrolhead]

Ishan Mishra.... This was a crossover episode