Opening AI's Black Box with Prof. David Bau, Koyena Pal, and Eric Todd of Northeastern University

In this episode, we dive deep into the inner workings of large language models with Professor David Bau and grad students Koyena Pal and Eric Todd from Northeastern University. Koyena shares insights from her Future Lens paper, which shows that even mid-sized language models think multiple tokens ahead. Eric discusses the fascinating concept of Function Vectors - complex patterns of activity spread across transformer layers that enable in-context learning. Professor Bau connects the dots between these projects and the lab's broader interpretability research agenda, identifying key abstractions that link low-level computations to higher-level model behaviors.

SPONSORS:

TIMESTAMPS:
(00:00) Intro
(04:03) Reverse Engineering AI
(10:10) Factual Knowledge Localization
(16:34) Sponsors: Oracle | Omneky | On Deck
(18:27) Future Lens Paper Intro
(23:54) Choosing GPT-J
(26:54) Vocabulary Prediction
(30:36) Sponsors: Brave | Plumb | Squad
(33:13) Fixed Prompt
(35:57) Soft Prompt
(41:32) Future Lens Results Analysis
(51:50) Tooling & Open Source Code
(56:07) Larger Models & Mamba Probes
(1:04:10) Function Vectors Paper
(1:09:39) Extracting & Patching Vectors
(1:13:30) Encoding Task Understanding
(1:15:36) Expert Models Implications
(1:18:09) Conclusion

Music licenses:
Y7AMC6WGEGV0C1NX
UVOQFS7SEPZF7JJQ
-
EXCBARTTNYZA0KPB
8TDCM1XEGMAGOJRV

[alfinal]

Cool episode. Maybe the host could let the guests talk a bit more and leave his opinions and insights to a post interview or another episode. Don’t get me wrong, it’s great he knows what he is talking about and his enthusiasm is contagious. But the flow gets disrupted.