LLaMA 3 Deep Dive! (Thomas Scialom - Meta)

Показать описание

Thomas joined us for the second time to talk about their latest work: LLaMA 3! We cover synthetic data for pre/post training, why didn't they go with MoE, privacy (was it trained on Facebook user data?), and much more.

▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬

⌚️ Timetable:
00:00 - 00:27 Intro
00:27 - 02:08 Hyperstack GPUs platform! (sponsored)
02:08 - 06:40 What is new in new Llama?
06:40 - 13:30 Synthetic data
13:30 - 15:35 Privacy - training on Facebook user data?
15:35 - 19:10 Scaling and distillation
19:10 - 25:35 MoE, new architectures?
25:35 - 37:15 Upper boundary for the quality of SX data?
37:15 - 45:10 Context length
45:10 - 46:40 What framework does Meta use for Llama
46:40 - 51:20 Playing with smaller Llamas
51:20 - 53:20 Multilingual capabilities

▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
💰 SPONSOR

Huge thank you to these AI Epiphany patreons:
Eli Mahler
Petar Veličković

▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬

▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬

#llama3 #llms #meta #opensource