New Mistral 7B – Is it that good?

In this video we dive into the Mistral 7b paper.
We look at Sliding Window Attention, Grouped-Query Attention, and the OpenOrca fine-tuning.

#llms #largelanguagemodels #mistral #mistral7b #opensource #llama

0:00 Intro
0:26 Base Info
01:09 Benchmarks
01:59 Sliding Window Attention
04:03 Grouped-Query Attention
05:00 MMLU benchmark
05:36 GSM8k
06:05 Equivalent model size
06:37 Instruct fine-tuning
06:52 OpenOrca fine-tuning
07:54 Outro