Retrieval at the Frontier: Beyond Text | Armen Aghajanyan | Multi-Modal AI (Chameleon), Meta

While retrieval has become a de-facto method of augmenting traditional, text-only language models, significantly less work has been done on multi-modal retrieval and retrieval with very long-context models. This talk will cover why retrieval is needed from first principles, state-of-the-art uses of retrieval apart from their traditional settings, specifically in multi-modal settings, and our long-term hypothesis of where retrieval as a technique will evolve too as context lengths continue to increase. Armen also delves into recent advancements, highlighting the development of Omni models at Meta, which integrate text and image data. The presentation covers techniques for reducing the computational cost of attention and anticipates future trends in retrieval models, particularly in handling multimodal and extensive data.

00:00 Introduction
01:20 Foundations of Retrieval
02:13 The Necessity and Function of Retrieval
03:24 Challenges and Solutions in Retrieval
06:27 Advancements in Attention Mechanisms
12:21 Future of Context Length and Fine-Tuning
15:47 Introduction to Omni Models
17:23 Training and Scaling Omni Models
22:18 Chameleon Project and Results
25:49 Token Economics and Future Predictions
27:44 Conclusion and Future of Retrieval