Zeta Alpha Trends in AI - January 2024: 10 predictions for AI in 2024

In this month’s edition, we look into the crystal ball and discuss our 10 predictions for AI in 2024. Learn about Self-Play, RL and Synthetic Data. Mobile ALOHA and the race to Foundational Robotics models. Mistral-E5: A new SOTA for MTEB. LLMs on edge devices, and much much more.

Dissecting the current Trends in AI: News, R&D breakthroughs, trending papers and code, and the latest gossip. Live talk show from LAB42 with the Zeta Alpha crew, and online on Zoom.

0:00 - 2:29 Introduction by Jakub Zavrel, Dinos Papakostas, and Rodrigo Nogueira
2:30 - 9:03 [Prediction 1] OpenAI will soon respond to the competition. Is it going to be GPT-4.5 or GPT-5?
9:04 - 12:38 [Prediction 2] Legal issues won't stop large-scale GenAI pre-training.
12:39 - 16:27 [Prediction 3] NVIDIA will start facing serious competition in the hardware-accelerator space for AI training.
16:28 - 21:28 [Prediction 4] The new best models in 2024 will be trained on synthetic data, with techniques like self-play.
21:29 - 32:56 [Paper] Improving Text Embeddings with Large Language Models
32:57 - 37:35 [Paper] Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models
37:36 - 40:16 [Paper] Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Models
40:17 - 43:00 [Paper] ReST meets ReAct: Self-Improvement for Multi-Step Reasoning LLM Agent
43:01 - 44:56 [Prediction 5] Text-to-video and multimodal models will finally have their ChatGPT / DALL-E moment.
44:57 - 49:02 [Paper] Generative Multimodal Models are In-Context Learners
49:03 - 50:05 [Paper] VideoPoet: A Large Language Model for Zero-Shot Video Generation
50:06 - 50:24 [Paper] MagicVideo-V2: Multi-Stage High-Aesthetic Video Generation
50:25 - 53:52 [Prediction 6] LLMs will be available at everyone's fingertips ... and eyes?
53:53 - 56:32 [Paper] LLM in a flash: Efficient Large Language Model Inference with Limited Memory
56:33 - 57:53 [Paper] PowerInfer: Fast Large Language Model Serving with a Consumer-grade GPU
57:54 - 58:09 [Paper] MobileVLM : A Fast, Strong and Open Vision Language Assistant for Mobile Devices
58:10 - 1:00:33 [Prediction 7] We will have the first generation of foundational robotics models by the end of this year.
1:00:34 - 1:02:16 [Paper] Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperation
1:02:17 - 1:03:19 [Prediction 8] Everyone will have their own, personal LLM.
1:03:20 - 1:05:12 [Prediction 9] AI4Science: Drugs, materials, and designs "invented" by AI will make a real-world impact.
1:05:13 - 1:06:59 [Prediction 10] Generative AI will start to contribute to real revenue and profit in companies.
1:07:00 - 1:07:51 [Paper] Retrieval-Augmented Generation for Large Language Models: A Survey
1:07:52 - 1:09:06 Outro & What's next