MISTRAL 7B explained - Preview of LLama3 LLM

Simple inference code for your local PC with less than 8GB GPU: MISTRAL 7B Instruct. From Mistral AI. Mistral 7B beats LLama 2 7B and LLama 2 13B.
The next evolution of LLama models by Meta?

Grouped-query attention explained. CTransformer and GGUF and GPTQ code implementation. Mistral 7B on free COLAB NB. Live demo.

#ai
#aieducation
#coding

[parhatbazakov]

My favourite channel! Man, you are a hero! Impossible not to love!

[sebastiansosnowski]

this model is small enough that you probably could run it on a google pixel 7 pro. I really love the developments in making models that are small enough for on-device inference

[WIMGNU]

Great video, one more question that needs to be answered: How to fine tune it?

[VaclavKosar]

Hello! Great video! Do you share the inference Colab somewhere?

[nathanbanks]

Interesting overview of the company. Thanks for going through it. Overall I'm super impressed--Mistral does better than CodeLlama-13B despite even though it's fine-tuned for coding.

I skipped your instructions; I just ran it locally using text-generation-webui like all my other models. Mistral has better formatting with this interface than CodeLlama--I had trouble with the indentation using default settings with CodeLlama--but the quality of the programs is actually better too. TheBloke also already has Mistral in all sorts of formats on Hugging face. However I'm still using GPT-4 when I actually want to write code. I only have 16GB of video RAM, so I don't usually run anything higher than 13B even with quantization. I haven't figured out how to make LoRA's for any model yet, mostly because the quantization complicates things. Doing everything through the the web interface doesn't help. It's nice to have a model that's small enough I could fine-tune it using full FP-16 weights.

Thanks for the overview!

[Esteband]

I've been playing with Synthia-7B-v1.3 trained on Mistral 7b and I am very impressed

[TheManuforest]

Great Video ! ... I Wonder if this model can be applied with a RetrivalQA function when asking questions based on my own docs . Thanks !

[ajaychinni]

Compared to llama chat, how well is Mistral able to do chating compared to llama chat which has done RLHF to make chat great

[ryanlowe]

Anyone else notice it said released in Europa? Who knew they were training LLMs on the moons of Jupiter now?

[Esteband]

Would you be so kind to share the Colab Notebook?

[naevan]

Hi ! Since you're my go-to tutorial content creator on LLM's I wanted to ask your opinion on something. I'm thinking of doing a thesis on detecting depression from tweets using classical methods like sentiment analysis from HF, but, more importantly, Im thinking of fine tuning an LLM like MISTRAL or LLaMMA to detect specific phrases which would indicate depressive thoughts, absolute wording( always, never, etc.) or First person pronoun usage( I am a fool, I am lazy, I must work hard) things like that. Would you think that would be possible, that I can fine tune it to detect and extract those ? 😅 Think it would be a nice thing to explore..

[jeffwads]

Sadly it isn’t that good at reasoning.

[None_ya_B]

Anyone know why the models tend to be woke? They will somtimes rant about social justice. I am guessing it's because they are made by tech companies and they are known for being pretty one sided in their belief and hiring processes but don't know for sure.