Using Graph Neural Networks for Multi-Node Representation Learning | Pan Li

Abstract: In this paper, we provide a theory of using graph neural networks (GNNs) for multi-node representation learning (where we are interested in learning a representation for a set of more than one node). We know that GNN is designed to learn single-node representations. When we want to learn a node set representation involving multiple nodes, a common practice in previous works is to directly aggregate the multiple node representations learned by a GNN into a joint representation of the node set. In this paper, we show a fundamental constraint of such an approach, namely the inability to capture the dependence between nodes in the node set, and argue that directly aggregating individual node representations does not lead to an effective joint representation for multiple nodes. Then, we notice that a few previous successful works for multi-node representation learning, including SEAL, Distance Encoding, and ID-GNN, all used node labeling. These methods first label nodes in the graph according to their relationships with the target node set before applying a GNN. Then, the node representations obtained in the labeled graph are aggregated into a node set representation. By investigating their inner mechanisms, we unify these node labeling techniques into a single and most basic form, namely labeling trick. We prove that with labeling trick a sufficiently expressive GNN learns the most expressive node set representations, thus in principle can solve any joint learning tasks over node sets. Experiments on one important two-node representation learning task, link prediction, verified our theory. Our work establishes a theoretical foundation of using GNNs for joint prediction tasks over node sets.

Authors: Muhan Zhang, Pan Li, Yinglong Xia, Kai Wang, Long Jin

~
00:00 Start
00:06 Intro
01:48 Standard GNN’s Overview
03:46 Agenda
05:00 Limited Expressive Power of Standard GNN’s
09:53 Distance Features Over Graphs
23:27 Labelling Tricks - Overview
24:31 Node-Set Representation Learning Problems
28:28 Labelling Tricks
44:20 Distance Encoding: Mixing Two Concepts53:00 Scalable Distance Encoding
56:48 Summary and Conclusion
58:20 Q+A

[felipemello]

Very nice talk, thanks for making it public!