Trends in AI — April 2022: DALL·E, Pathways, PaLM... + Generative Language Models Demo

So much happened in AI in late March and early April 2022: OpenAI's DALL·E 2, Google's Pathways and PaLM 540 Billion parameters Language Model, Bootstrapping Reasoning with Reasoning in Language models and much more. We walk you through all of it with Jakub Zavrel, Rodrigo Nogueira and Sergi Castella.

Timestamps
00:00 Introduction
10:20 BigScience training of the final 175 billion parameter multilingual model
12:50 Hardware: GraphCore IPU systems, NVidia's new H100 GPU.
16:10 174 Trillion parameters scale model from a collaboration of Chinese groups?
19:15 Zeta Alpha's new smart PDF reader with highlighting capabilities, taking notes, sharing them and even using paper snippets as search queries.
24:41 STaR: Self-Taught Reasoner. Bootstrapping Reasoning with Reasoning
30:40 Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priors
33:22 OpenAI's DALL·E 2: Hierarchical Text-Conditional Image Generation with CLIP Latents
37:32 Visual Prompt Tuning
38:30 Chinchilla model from Deepmind: Training Compute-Optimal Large Language Models
40:30 Pathways from Google: the system for heterogeneous distributed training on hardware accelerators and PaLM, the implementation and training of a 540 billion parameters model.
44:51 Large Generative Language Models Bakeoff: head-to-head comparison of GPT-J from EleutherAI, various sizes of GPT-3 from OpenAI, Cohere...
61:00 Will most text be artificially generated?
61:30 ECIR 2022, ICLR 2022