Mistral 7B 🖖 Beats LLaMA2 13b AND Can Run On Your Phone??

In this video, we thoroughly test Mistral 7b Open Orca GPTQ. It's an incredibly performant small model that illustrates the future of locally hosted edge models. How good is it? Does it beat LLaMA 13b? Let's find out!

Enjoy :)

Join My Newsletter for Regular AI Updates 👇🏼

Need AI Consulting? ✅

Rent a GPU (MassedCompute) 🚀
USE CODE "MatthewBerman" for 50% discount

My Links 🔗

Media/Sponsorship Inquiries 📈

Links:

[matthew_berman]

Want me to try to put Mistral 7B on a phone?

[mwissel]

Please try it on your phone, yes. And also I'd be very interested in a fine tuning video for this model. Seems like a great small model to try out fine tuning datasets with.

[amj]

This is seriously impressive. I used to laugh at Sci-Fi movies that had time travelers using devices that could store lots of information, I used to say that would never work because it would need a connection to a network, via some kind of system like the internet. Well turns out, nope, the movies spoke the truth lol.

[shadowdragon]

Excellent, these small models keep getting better and better. Now I'm just waiting for someone to bake one of these open source models directly into a Linux distro

[charetjc]

"It assumes serialized drying.. so that's still a correct answer..." If only my math teachers were this lenient.

[Eric_McBrearty]

Another great video Matt. There's no way I'd be up-to-date on the progress of AI if it wasn't for you churning out these videos like clockwork. Thanks.

[diadetediotedio]

Interestingly enought, even GPT-4 fails the cup test, I think it is pretty good for testing interpretation and causal inference and/or comprehension of these models

[alexjensen]

YES! GIVE IT A SHOT ON YOUR PHONE!! I would love to see that and have a heap of use cases for something like that...

[LukeSchoen]

Been waiting for this! I knew you would love Mistral! Thanks dude.

[pret]

I think for the cup and ball question, there could be a follow-up question or instruction like: 'Also consider gravity.' Or 'Take gravity into account.'

[ramp]

Thank you for the video. Curious how much did you end up paying for the GPU for creating and testing this out? Thank you

[IonSabatico]

Hey, cool comparison! Question: would it make sense to add text sentiment/tone detection? BERD models excelll, but LLMs are tailgsting them now

[Derick]

Probably a dumb question but since theres so many different ones is there like a recap or top list that gets updated as you go so we know which top models and platforms to try out? As i end up finding stuff in your old videos i like bit its outdated and not as efficient in some as the newer stuff but it was probably way ahead at the time. Maybe a list that gets updated all the time or weekly summary cap or something idk

[eccdakaj]

Hi! I watch almost all of your videos. I have a question: What are the hardware requirements (CPU and GPU) for running Mistral 7B locally?

[alexlindgren]

If I would want to deploy this as an API on runpod, how todo it?

[haneke]

I know it’s stupid question but just to be sure: when you run this like on the video, it is a closed system so you can share company data and keep it confidential?

[mvasa]

what is the foot print and how can we be sure (other than Matt testing this) it is compatible?

[stickmanland]

Been a big fan of this model. Absolutly incredible preformance! Thanks to Matt for the incredible video.

[SAVONASOTTERRANEASEGRETA]

Can the web ui text generation AI model work in the background? For example, telling him to do an accurate search without being there to give input? Or tell him: write me a message in a precise hour from now?

[RomboDawg]

Ive been trying to combime mistral 7b with codellama 7b (or wizardcoder 7b) to get a hopefully great coding and nom coding model. But its really hard to combine models with different architecture. Wish me luck. The final model would be 14b parameters in theory