Mixtral 8x7B DESTROYS Other Models (MoE = AGI?)

MistralAI is at it again. They've released an MoE (mixture of experts) model that completely dominates the open-source world. Here's a breakdown of what they released, plus an installation guide and an LLM test.

* Sorry for the part where my face gets blurry

Enjoy :)

Join My Newsletter for Regular AI Updates 👇🏼

Need AI Consulting? ✅

Rent a GPU (MassedCompute) 🚀
USE CODE "MatthewBerman" for 50% discount

My Links 🔗

Media/Sponsorship Inquiries 📈

Links:

Chapters:
0:00 - About Mixtral 8x7B
9:00 - Installation Guide
13:06 - Mixtral Tests

#EdrawMind #EdrawMindAI #aipresentation #aimindmap

[kenchang]

Kudos to people who do open source, the talent and sharing is amazing :-)

[mehdihosseinimoghadam]

Could you please change your questions (a bit) from time to time, there is a slight chance of having your questions (they are available in your notion page) in the training dataset of newer models? So the fact that Mixtral 8X7B answers all of them might not necessarily reflect its strength (1 percent chance 😁). Can you change the questions a bit and then test the model and let us know what will happen (specially in the case of that marble and cup question). Many thanks🤗

[etunimenisukunimeni]

I always thought the answer to the killer question would be three, because one of the four killers is dead and therefore not really present anymore. Unless the question refers to both dead and alive killers 🙄

[nicosilva]

dashes '-' are markdown for bullet points. My suspicion is that this has nothing to do with the model, but the renderer recognizing the markdown bullet code (which is a dash), and displaying an actual bullet point. Great review. Thanks.

[MagnusMcManaman]

I think the problem here may be that the questions you ask and the correct answers to them have been publicly known for some time, so this particular model may have the correct answers to those questions implemented.

My suggestion is to create a second, implicit set of questions and use it when any models do suspiciously well with the first, and explicit set.

[stickmanland]

One of the best models in the world now! Mistral is a close second to OpenAI. They are proving that open source is very quickly catching up to propitiatory models.

[PermanentExile]

20 years ago, I worked at the Pentagon with military planners. The first level of military planning is the BOGSAT (Bunch of Guys Sitting Around Talking). They are a diversity of experts, each with their own take on a situation. Analysts would then try to pull relevant details from each “expert” opinion to form a composite opinion about the problem.

[shamimhussain]

1. It's not 8 separate models, but rather a single "sparse" model where only part of the model (weights) gets activated (in each FFN block, for each token). 2. The reason it takes 2xA100 is probably because of the fact that it's unquantized. It's quantized versions should be more lightweight.

[carstenmaul]

Only a couple of months and the 7B models evolved from toy status to ChatGPT 3.5 competitor. Impressive.

[jackflash]

The pace of innovation and progress in AI is wild.
The things we thought were state of the art just a month ago are now old hat.
What a time to be alive.

[Axenide]

At 18:14 it used dashes, but the WebUI uses Markdown so it turns into dots. :)

[RWS]

Hay Matt Going to try the with LM Studio 32.23 GB We will see how it goes!

[danielhenderson]

it failed the shirt question IMO.
Killers answer was impressively written.
The marble answer was also the most impressive I have seen. I will only nitpick about its obvious lack of physical understanding where it says that the marble "falls" out of the cup after it is picked up. Otherwise it is bang on.

Overall, very impressive and makes me excited for what is to come in 2024. Thanks for demonstrating it :)

[william]

Sounds like something you could replicate using ollama, so it can run on lower end hardware

[mirek]

Form huggingface - the bloke - you can downland already quantized version of that model like
which works on rtx 3090 ( 30 layers on gpu ) speed 30 tokens /s
Easily beats GPT-3.5 and is quite close to GPT-4

[jay]

You misinterpreted the AI's response here: 16:40 "How many words are in your response to this prompt" . The AI thinks you were asking about its *previous* response, and, frankly, your wording was ambiguous. It guessed 54, I count 52. That's pretty darn close!!

[modern_sapien]

You can run the 2x RTX A6000s for about half the price of th A1000 and meet minimum requirements.

Great video!

[ironknight]

Great video! I'd revise your drying question, that always seems to be unclear.

[GNKDS]

How do I run it on Colab if it's so hefty!

😭

[fernandoz]

Incredible, never thought a model would pass the killer room question so soon.