Function Calling Local LLMs!? LLaMa 3 Web Search Agent Breakdown (With Code!)

With the power of LangGraph, we can add the ability for tool use and function calling with local LLMs! In this video, we go over my entire process and breakdown all of the code needed to do this yourself using Llama 3 - 8b as a research agent with a web search tool. This idea can be extracted further to more advanced workflows, and additional tools outside of just web search. Enjoy!


FIX FOR ERROR 202 WITH DUCKDUCKGO_SEARCH PACKAGE:
pip install -U duckduckgo_search==5.3.0b4

Chapters:
00:00 - Intro
00:33 - Prerequisite Ollama
00:59 - Workflow Diagram Overview
02:11 - How This Simulates Function Calling
03:17 - Code Walkthrough Starts
04:10 - Defining LLMs
05:15 - Setting Up DuckDuckGo API
06:52 - Prompting Overview
07:21 - Llama 3 Special Token Explanation
09:07 - Generation Prompt
11:29 - Routing Prompt
13:23 - Transform Query Prompt
14:45 - Putting It Together With LangGraph
15:40 - Defining the State
17:40 - Router Node
18:48 - Transform Query Node
20:13 - Web Search Node
21:15 - Generation Node
22:17 - Adding Nodes & Edges
25:02 - Invoking the Agent!
27:15 - LangSmith Trace Overview
29:34 - Outro

[martintourneboeuf]

Awesome Class, thank you!
A very complicated and powerful topic, but you divided with great methodology the problem, your code is friendly to read and the presentation very smooth.
Congrats and thanks again as this for sure helps me.

[madhudson]

absolutely fantastic. I've been trying to do something similar today and experienced much of the 'going rogue' - with incorrect special tokens, until I followed your example.

[matthewturnerphd]

Thanks for this video! Several smaller details you emphasized were things I had missed in other tutorials, and really helped me.

[jimlynch]

I really enjoyed this video. I've seen lots of "how to" vids WRT programming using local LLMs but this is by far the best one I've viewed. I often get lost and have to re read some of the steps however you moved along at exactly the right pace for me and explained pretty much all of the questions I was dreaming up during the view. Thank you! Ollama does have function calling but this method seems to be more logical and easier to understand.

[Kidzwish]

You're awesome dude, neatly explained! Thanks !!

[redfield]

Excellent video and perfect hands-on introduction to langgraph. You make my day. Thank you.

[rlyzrgr]

Thank you for the wonderful lecture and example of source code. The example source code worked nicely in the local environment, and the test code inserted conveniently in the middle helped me understand the example a lot

[petermorvay]

Excelent tutorial. Thank you Adam, your content is really valuable. I was able to run the code in the local environment.

[_codegod]

Your presentation style is awesome bro.

[emporiumofthearcane]

Good video. Thanks for making it. Subbed.

[ofrylivney]

That is really high quality content! thanks!

[DakshSripada]

is there any other method of function calling using llama3 ollama and langgraph? Is there a way that would allow us to add tool nodes into the graph as we do with openai agents and then have llama3 use those tools? Or would this approach simply complicate things?

[JoshuaMillerDev]

I like seeing these. Something to consider is having the web search differentiate between sponsored and non sponsored results. I have not seen anyone tackle that yet. It seems to me that search results and LLM outputs would be more accurate when steering away from sponsored data.

[Watdooyoumeen]

Can you integrate your routing node to perplexica the clone of perplexity?

[TestMyHomeChannel]

You are an awesome teacher. I am already running almost the same setup, with agents created automatically by another PraisonAI but didn’t fully follow what was going on and many times not working and I didn’t know why. I loved the way you broke down and explained everything. Looking forward to see more videos from you. Best wishes

[amanmeghrajani]

loving your content, thank you for sharing this. learning a lot!
would you be interested in making a video to show how to deploy these models and have access to them from inputs like whatsapp chat, email? would be super helpful

[WladBlank]

Interesting concept. I try to force my llms to produce valid json and this would be easier.

[szpiegzkrainydeszczowcow]

Great job. Subscribing. Any chance you would make some video on long term memory in vector db? greetings

[JoshuaMillerDev]

I wonder if anyone has went through the thought exercise of how an AI model could benefit from having anyone "fold at home" in order to build it. In other words... instead of the owner (OpenAI, Llama, whatever..) dedicating servers to build a LLM, could it not be distributed such that video cards in PC's around the world contribute idle cycles towards the training? Seems like a good way to have an open sourced model to get off the ground, or build using more tokens. Could even have a reward system (minor) allowing X privileged API access per contribution node or whatever.

[_codegod]

I don't understand pipe operator. :(