The REAL cost of LLM (And How to reduce 78%+ of Cost)

I want to give you step by step guide on how to reduce LLM cost by 70%, and unpack why it is costing so much now

🔗 Links

⏱️ Timestamps
0:00 How I burned $5000 on OpenAI
2:41 My experience with AI Girlfriend Project
6:01 HubSpot AI For Markter
7:23 How to Reduce LLM Cost
8:33 Method 1 - Finetune
9:51 Method 2 - Cascade
10:35 Method 3 - LLM Router
12:47 Method 4 - Multi-Agent
14:07 Method 5 - LLMLingua
16:31 Method 6 - Optimise tool input/output
17:16 Method 7 - Memory Optimisation
19:26 LLM Monitor & Analytics
20:09 Tutorial: Monitor & reduce 75% cost

👋🏻 About Me

#mixtral #gpt4turbo #gpt4 #ai #artificialintelligence #tutorial #stepbystep #openai #llm #chatgpt #largelanguagemodels #largelanguagemodel #bestaiagent #chatgpt #agentgpt #agent #autogen #autogpt

[jasonfinance]

I never got the point of setting up those LLM monitor before, but the step by step guide in the end showing how you use it & how it lead to real cost reduction is gold (70% is crazy!); Will try it out, thank you!

[agentDueDiligence]

Hi Jason!
another alternative to measure costs in your script is to simply use the chat completions information provided by the api of openai.
every time you call the API, it will return the total tokens in the response json in the "usage" dictionary. That way, you can monitor & control your usage as well.

[kguyrampage]

Bro that's crazyyy, I literally just wrote down notes on reducing costs in different approaches today. I was about to test them out and saw this video in my inbox. damn very on time.

[que-tangclan]

This is the best AI content I have seen all week. Thank you for this.

[Joe-bpmo]

Didn't realise the cost gap between GPT4 & Open source model like Mixtral is so big! 200x more expensive really change how I think of building LLM products;

Thanks for sharing! will definitely try to optimise my LLM apps!

[michaelwallace]

A step by step build of an agent architecture would be invaluable! Thank you for the video.

[betun]

Superb content Jason, I will highly recommend your videos to everyone getting their hands dirty with LLMs. I am gonna try some of these myself. It's a shame I didn't build it before because something like the AI router occurred to me but I do not have the patience to implement these.

[Ke_Mis]

Your content is just superb as always Jason!

[matten_zero]

This is the biggest flex ever! 💪I can only dream to be as cool of an AI Engineer as you. I thought building a digital agent with automatic voice that can do RAG was cool.

There are levels to this game an Jason is on a whole different world. Thanks for posting these videos. It's educational, funny and inspirational for me.

[quickcinemarecap]

00:05 Using autonomous sales agents led to unexpected high costs.
02:11 AI startup costs fluctuate with usage
06:12 Marketing teams are adopting AI for automation and hyper-personalized customer experiences.
08:19 Using smaller models can reduce cost by multiple magnitudes.
12:27 Customize router for cost reduction
14:23 Using small models can significantly reduce the token and word count for large language models
18:13 Reducing large language model costs
19:55 Analyze token consumption for cost optimization
23:18 Agent executor identifies cost breakdown and offers cost reduction strategies
25:00 Using GPT-3.5 turbo and staff documents for detailed and cost-effective summarization.

[ZacMagee]

Love your content man. You have helped me really expand my knowledge and push my boundaries

[chengchangyu]

A step by step build of an agent architecture would be very helpful. I am looking forward of it.

[leandroimail]

Tks very much for this video. I have been having problems with the cost of my agents. I will do this tips and clue that you gave. Thks again.

[gsolaich]

We were planning to build ai assistant kind apps but always pull back due to cost it incurs, this is a fabulous video that has given us a new direction to go ahead. Thanks a lot .... looking forward to see other videos

[serenditymuse]

Excellent. Most of his videos are but this one was especially useful to me.

[Beloved_Digital]

I am a newbie when it comes to build AI powered apps.
Although i don't fully understand all you say because i am still learning the basics all i can say is Thank you for sharing this valuable contents with us

[nicechannel]

A great dive into the cost of Al models as it is hard to find related content. Can you do a video about how much Openai is roughly spending on computaion cost and also how this constraint will hinder the adaptation of these models in the enterprise space. Great job man 👍

[clamhammer]

I had this idea for LLM routing a while back and wondered why nobody has done it. I figured there was some sort of information I didnt have that was stopping it.

[TimBnb]

Cette chaîne est la meilleure école existante à ce jour.
Merci Jason

[misterloafer]

Yes, please do a video on multi agent methods