I tested Mistral AI 7B vs ChatGPT (GPT 3.5 TURBO) on 20 Questions!!!

❤️ If you want to support the channel ❤️
Support here:

🧭 Follow me on 🧭

[proflicxx]

the one issue i noted is when you check llm please clear the chat history before new question, don't append new question, due to token issues that affect quality of response, chat gpt got high context window, but mistral doesnt

[Devjwal]

🎯 Key Takeaways for quick navigation:

00:00 🤖 The video introduces a comparison between Mistral AI's 7 billion parameter model and GPT 3.5 Turbo (ChatGPT) to evaluate their responses to various prompts.
01:18 🛡️ The presenter mentions concerns about data contamination and the difficulty of comparing Mistral AI's 7 billion parameter model with GPT-4 due to differences in model sizes and architecture.
03:37 💼 The comparison involves testing both models on a set of 20 questions covering categories like reflection, knowledge, code, and more, to observe and compare their responses in different contexts.
04:19 ⚖️ The comparison begins by evaluating responses to a reflection question regarding the use of Kubernetes, showcasing both Mistral AI and GPT 3.5 Turbo's generated arguments for and against Kubernetes.
05:17 🧮 The testing includes diverse questions, such as math problems and language puzzles, testing the models' ability to understand and provide accurate solutions, revealing variations in their performance.
06:51 🌍 The models are tested on questions related to general knowledge, including understanding family relations, countries' status, scientific concepts, and more, showcasing their comprehension and accuracy in different topics.
08:13 🗺️ The comparison includes political questions, testing the models' ability to provide neutral and accurate information regarding sensitive political topics like Taiwan's status.
09:46 🇫🇷 Both models are tasked with translating English text to French, showcasing variations in their translations and understanding of the given English phrases.
10:42 💻 The models are challenged with explaining code, finding bugs, and generating new code, revealing differences in their programming knowledge and capabilities.
12:47 📝 Mistral AI impresses with its detailed understanding and bug-spotting ability in code, showcasing its potential for code-related tasks.
13:39 🖥️ ChatGPT demonstrates accuracy in generating a Python function for finding leap years, highlighting its programming capabilities.
14:20 🎶 Both models attempt to generate a 12 Bar Blues chord progression in the key of E, showcasing their ability to generate musical content.
15:04 🌌 Mistral AI provides a JSON response for the five planets closest to the Sun, demonstrating its ability to understand and follow specific formatting instructions.
16:42 😃 ChatGPT impresses with an SVG code for a smiley face, showcasing its creativity and ability to generate specific design elements.
19:07 🛍️ Both models provide compelling product descriptions for a 100W wireless charger, demonstrating their capacity to generate marketing content.
20:05 🎤 ChatGPT excels in crafting a persuasive pitch to encourage YouTube viewers to subscribe, highlighting its ability to generate engaging and promotional content.
21:43 👏 The comparison concludes with an overall positive impression of Mistral AI's 7 billion parameter model, emphasizing its impressive performance throughout the test.

Made with HARPA AI

[vishalpanjeta]

The distance from earth is given in AU (astronomical unit)
It is a valid distance unit in astronomy when dealing with intra solar system distances as it's too large for kilometers but too small for light years.
1 AU = Avg distance of earth from the sun.

[yngeneer]

yep, the context issue is one thing, the overall chat settings like temp, top_K/P, rep penlty etc... it all can have possible influence on output [like not following the no-explain instruction].... still, thanks for your effort

[Serifinity]

Munch-House-En Tri-lemma (trilemma as in a triple dilemma) 👍 Great video, thanks for creating and sharing.

[mathematicalninja]

So early that even 720p is not available.

[IslandDave]

Look at the OpenOrca trained version of Mistral 7B next 🙂

[ViewpointsVortex]

I’ve been using Mistral 7B model for a while now and I feel it’s not as good as GPT3.5; now I prefer to use Orca2 13B model instead of Mistral 7B

[eric]

The svg had colour of white, it may be working specially on a colour background. Maybe check it by changing colour. BTW mistral app is really fast compared to Chatgpt. That’s a pro.

[cuentadeyoutube]

Sally can have either 0, 1, 2 or more sisters, if we consider she can have half sisters and so can her brothers

[leafdriving]

Re: Blue Progression: You don't have to know anything about music to see:
Mistral is just going back and forth between two chords (2 chords back and forth)
Chat-GPT3.5 - Gets all fancy and adds a B7 to the E7-A7 toggle. (All 1 chord, then toggle, then tri)
I don't even have to know what this sounds like to know GPT3.5 Won this round.

[anuvratshukla]

Can you implement Mistral AI on tabular dataset Q&A (likeusing langchain agents etc.)

[faaz]

I tested mistral on python function for leap years it did get it right even better than chatgpt

[alx]

Coding tasks are bit shallow. Try to add some more - like converting the code from language to language, modifying / refactoring code by verbal instructions, asking llm to optimize the code or finding edge cases, writing tests

[Techonsapevole]

Can you make a video on how to add knoeledge to Mistral 7B ?

[Nick_With_A_Stick]

I mean shoot that’s really good, if you train it with a gpt3.5 dataset it would preform exactly like it.

[gui-zxdi]

French translation was better in chatgpt, difference is so small but it is a tiny bit more poetic. But both are equivalent.

[Somebodythatoverthinks]

7:02 GPT 3.5 is actually right it says "including herself"

[HB-klik]

The audio feels muted 😅 thought it was my phone but I checked with other videos too

[NOFX]

Munchousen syndrome is where a captive empathizes with their captor. There was a bank robbery in that town i believe where the hostages began to help the person who took them hostage.