GPT-4o mini - Can it be used for Agentic RAG?

How good is GPT-4o mini for Agents?

LINKS:

💻 RAG Beyond Basics Course:

Let's Connect:

Signup for Newsletter, localgpt:

00:00 Introduction to GPT-4 Mini
00:59 Creating the RAG Pipeline with Llama Index
01:25 Installing Required Packages
01:53 Configuring LLM and Embedding Models
03:10 Data Loading and Processing
04:11 Embedding the Dataset
06:57 Creating the Vector Store
07:53 Building the Agent with Llama Index
09:51 Agent Performance Comparison

All Interesting Videos:

[DavidePasca]

One trick with less powerful models is to spend more time writing the instructions. If the model's output is limited, one can explicitly ask for something more elaborate. I noticed this in the past when I had GPT-4o as front end and an agent using 3.5 doing background research. It took some work to get 3.5 to a decent level, but it was worth it in terms of running costs.

[unclecode]

This is really fascinating topic, I always feel alive when I do such thing with model.Fascinating topic! I have a few questions: 1) How do we define "good enough"? What if the output from Gpt-4o-mini meets user needs while Claude's model exceeds them? This is rarely discussed. 2) We've seen how prompt tuning can significantly affect outcomes. We should explore the Lamaindex prompt template; perhaps some examples can help a model generate just what we need.

To me, when evaluating a model for task A, the priority is how effectively I can get it to produce the desired responses. This approach allows me to assess multiple models and maintain flexibility in decision-making. Sometimes, I start with a smaller model and gradually increase its complexity until I achieve the desired results. Other times, I adopt a reductive method, beginning with a larger model, refining my understanding of task specifics, and then switching to a smaller one until further reduction harms quality.

This is truly a captivating topic; I always feel energized when working with models in this way.

[ahmedgharbi]

In my experience, gpt-4 (or gpt-4o) function calling is not great for agentic RAG. However, i’ve seen a huge improvement when I used them with a ReAct agent. I found ReAct helps them in reasoning, elaborating the thinking steps and generating much more accurate and detailed queries. I’m curious if anyone had a similar experience. Would love to see a comparison video with ReAct, thank you for the video !

[jcksn]

Nice vid as always. I tested gpt-4o-mini for my own use-case (which is using an agentic workflow), and I agree with you on the notion that it's not really good for agents. In my tests, it sometimes performs even worse than gpt-3.5-turbo-0125. I can't replace gpt-3.5-turbo with this one, let alone gpt-4o or other superior models. Underwhelming is all I can say at this point.

[KS-tjfc]

As an agent, is GPT-4o mini fine if it is orchestrated by a superior model? I was wondering about executing an orchestrated prompt given to GPT-4o mini, Deepseek-V2 API and Gemini 1.5 Flash — each running through the steps and then allowing the superior model to review and decide which answer is best and/or pooling the best answers/points together and simply rephrasing that response. Start large/expensive — do many smaller tasks for each — recombine for large/expensive final output. Thoughts??

[masterapofis]

They should have warned us that GPT-4o mini only has 50 questions every 4 hours at this rate.
Does it force you to have multiple accounts or have to share it, ceasing to be free, and when does that happen?
We will go to another AI. We want GPT-3.5 back!

[shafai]

Can u covert your video in hindi/ urdu for easier understanding. Also take live stream session. And it would be helpful to tie this up with remote job offerings or something like that.

[KumR]

Can u extend this to having streamlit ui too please

[TheReferrer]

If AI's got good at agentic workflows would it not be in the interest of the big tech companies to keep it in house?

[bastabey]

gpt4o-mini is much better than Gemini flash

[stratos]

Can you run it locally? No? Not cost effective enough...