How To Connect Local LLMs to CrewAI [Ollama, Llama2, Mistral]

🤖 Download the Source Code Here:

Don't forget to Like and Subscribe if you're a fan of free source code 😉

📆 Need help with CrewAI, join our free Skool Community:

This video is perfect for anyone eager to integrate powerful Local Large Language Models like Llama2 and Mistral with CrewAI using Ollama for smarter, faster project development.We'll show you the easiest way to link these LLMs with CrewAI, step by step. By the end of this tutorial, you'll know how to set up, customize, and use these AI models to automate tasks and boost your project's intelligence. Get ready to upgrade your tech skills and make your work with AI more productive and innovative. Start now and see how simple it is to bring the power of local LLMs into CrewAI.

📰 Stay updated with my latest projects and insights:

Resources:

Timestamps:
00:00 Introduction
00:47 Video Overview
01:13 Ollama Review
01:38 Llama 2 Review
02:38 Mistral Review
03:04 CrewAI Review
03:52 Download & Setup Ollama
06:24 Configure LLMs To Work With CrewAI
11:01 Connect CrewAI to LLama 2 (Simple Example)
18:08 Connect CrewAI to Mistral (Advanced Example)
24:33 Outro

[wadejohnson]

I asked and you delivered! I'm at a loss for words to describe you. Just know that you are one of the best in the biz. And from the other comments, you can see that your work is very much appreciated. Thank you.

[RetiredVet]

In this video Brandon mentions two other Crew AI videos he has created. I've taken the crash course video and it is about the best Crew AI video I have seen. There are other good ones on YouTube, but you should not miss Brandon's videos if you are interested in Crew AI.

[ewldcyd]

I just subscribed. Please focus on local models when you can, this is fascinating! Thank you!

[rickmurphy]

Thanks for the detailed explanation for setting up local LLMs and Agent crews, really informative for beginners like myself!

[tobiaslim]

Excellent! presented exactly as an educator would! I've been through many tutorials and all of them were too difficult to follow! You got it right by providing the workflow at the beginning as well as the programs needed! Great job!

[lalpremi]

Thank you for sharing, looking forward to testing Crewai on my local systems. Have a great day. :-)

[jdsharp]

You are a great teacher. Very easy to follow and cover the topics thoroughly. So glad I found your channel ! Thanks for all your hard work and dedication! 👍⭐⭐⭐⭐⭐👍

[LunaJLane]

Not sure where my previous comments went to but I was able to work out all the issues I ran into.

[johns.]

Thanks a ton for dropping this. I was literally working all afternoon on this very thing. That last part about which models support the different functionality was very applicable and time saver versus banging my desk.

I had also tried using both a local LLM with LMStudio (which mirrors the environmental setup) for my agents and GPT-4 for the manager_llm. Couldn’t get that to work.

I’ve been focused on making crews create a Python GUI for parsing videos as a PoC. I’d love to see you give it a whirl. Since we know that stand-alone chats are not great at full fledged more complex coding projects, I am trying to move past that hurdle by using a DevTeam Crew. This far, even with GPT-4, I’ve been unsuccessful and usually end up with just the example output I provide it.

[denijane]

Thank you for the detailed walk-trough. It took me the whole evening, two conda environments and gemini and finally chatgpt's help to set it up, but yay me. In the end, it worked, but damn, is it slow without cuda. I don't know which of my previous local llm experiments decided I don't need cuda in my life, so now I'm waiting for my major kde update to reinstall it. I think maybe the internet search won't work by default, maybe it will require some api for a search engine but as I said, it's slow.

[DannyBordwell]

Hey mate, first thank you for making this video! First video I have by you and I really like your teaching style. I went to your web site and filled out the form but didn't get an email. I'll keep an eye on it but wanted to give you a heads up that you might have a bug. Thanks again for your contributions to all of this, and keep up the great work :)

[AC-prsi]

Great video Brandon!!!Thanks for taking your time to make it.

[Taisen-oifp]

OMG, you are the Hero!! Nice video Brandon!!!

[petlivematters]

very nice content thanks for the resource and the continued email updates 🎉🎉

[CuriousCattery]

Absolutely brilliant! Any chance you could do a video on deploying this to a server so it could be run remotely?

[BobbyReedy-qywp]

That's a little bit of a lot is my favorite fray sound thank you

[johns.]

Got both examples working. One thing you didn't cover: While using a local LLM, can you define the Crew to use sequential or hierarchical processes?

[TopicTome]

any reason why I am getting this error : "It seems we encountered an unexpected error while trying to use the tool. This was the error: can only concatenate str (not "dict") to str ". It seems like it is not reading the MarkdownTools file correctly

[TheAmazonExplorer]

Thanks for the great video please make a video that crewai multi agent for computer vision

[musumo]

Awesome..would love to see this running with openrouter…