Scaling Laws, Emergent Behaviors, and AI Democratization with Irina Rish, PhD

Join Irina Rish, PhD, as she takes a dive into the recent advances in foundation models in her talk, "Recent Advances in Foundation Models: Scaling Laws, Emergent Behaviors, and AI Democratization." In this video, you’ll discover how large-scale, self-supervised pre-trained models like GPT-3, GPT-4, ChatGPT, and many others are revolutionizing the field of AI. Dr. Rish explores the rapid growth in neural scaling laws and their impact on predicting model performance and AI safety. Learn about the collaborative efforts of leading universities and non-profits, including U of Montreal/Mila and EleutherAI, to democratize AI and make cutting-edge technology accessible to all.

As you watch this talk, you’ll see how it highlights the challenges and solutions in obtaining large compute resources for academic and non-profit AI research, emphasizing the importance of open-source foundation models.

#MachineLearning #DeepLearning #ArtificialIntelligence #NLP #LanguageProcessing #DataScience #AI #DataEngineering #DataViz #DataVisualization #AIResearch #TechTalk #ODSC

Timecodes:
0:00 - Intro
1:48 - Building AGI as an Ultimate Goal of AI Field
4:30 - It’s All About (Further) Generalization
10:36 - Neural Scaling Revolution
16:39 - Successes of Large-Scale Models
23:07 - History of Neural Scaling Laws
30:05 - What’s Next?

You can also follow ODSC on: