Build your own RAG based LLM Application (Completely Offline!): AI for your documents

Retrieval Augmented Generation is one of most essential use cases with Large Language Models.
You can ground your large language model to answer questions based on the contents of your document.

For this tutorial, we are building a complete offline RAG-based LLM app which utilizes Ollama for inference, ChromaDB for vector store, streamlit for UI.

---
🔥 *Resources*

_Example Docs_

---
⚡️ *Follow me*

--
🎞️ Chapters

0:00 Intro
0:16 Application Demo
1:13 Prerequisites
1:50 Code: Env Setup
2:40 Code: App UI
3:40 Code: Splitting Document + Data Structures
8:06 Code: Embedding +Vector Database
15:58 Code: Adding LLM + Grounding
20:30 Code: Re-ranking with Cross-Encoders
24:29 Demo: Multi-document Relevance Scoring
25:47: Outro

[yankeem]

⚡ Watch Part II: Super Fast RAG with Semantic Cache:

[hurricanos]

This is by far the best and concise rag tutorial available online.

[Subaragam]

This is Amazing.. Heading straight to Part II

[starX]

bro this is the only tutorial which helped me so far of all the other youtube videos i have watched on rag based applications love you from india bro

[IamKonstantin]

what a time we live in. was having issues running it but copilot was able to walk me through to make changes to code and get it running. thanks for the cool app :-)

[adsirbu]

Very good tutorial, I will definitely follow for more tutorials. Works like a charm

[modest_supreme]

Great tutorial, thank you for going in depth and showing me these tools!

[KrishnaGupta-nvjj]

This video actually taught me, how to build rag application.
Thank you so much

[FabricioAlves]

You are genius! So I would like to know 3 things:
1. Let's say I want an easy way to know which documents are processed, and also by levels (for example: Finance, Goals, Personal Documents ID) So I need to create categories. How I do that?
2. With the LLM, can I add reasoning and other models like DeepSeek and/or others? (for example I processed all my electricity bills of last 24 months, then I ask, how can I lower by electricity bill for this year - and the reasoning model kicks How can I do that?
3. Report Creation Export ability (ex. If I processed all the recipes I have, in different formats {photos, handwriting}, different languages and etc, I and say: give me all the recipes with lemon as an ingredient and then I want to export or share....How Can I do that?

Thanks

[jenniferdsouza]

Awesome tutorial! Many thanks for this great resource

[TGIMonday]

Definitely subbed and liked and following for more! Thank you for sharing this wealth of info

[fancypetsulove]

Talk slowly to make it easy to follow. Thanks for this tutorial

[kerryjackson]

IndexError: list index out of range in upsert. I keep getting this error when I try to process a pdf. Please help...

[kristijantomic]

Very good and to the point. Thank you!

[robertonarvaez]

Hello Yankee. First of all thanks for the excellent video. Question for you. I implemented your RAG process and wanted to know how would you address issues of deleting files from RAG or updating files that contain newer information that were already in the database. I tested with one file by deleting it from the embeddings table but soon realized that the files references were in other tables and broke my query. The second question is, I noticed that when I enter a question in the prompt if goes only to the pdfs in the database. So in other words, I can no longer query general stuff like "what is the tallest mountain int he world". Any help will be greatly appreciated.

[programmingholic]

Thaks bro for sharing, it's helpful.

[mohammedabbas]

Amazing content. Definitely worthy of a like, share and sub. Will wait for similar high quality videos. Cheers!

[alwikah]

Best video about RAG!
Thanks a lot for sharing.
Which tools do you use to produce your youtube videos?

[JellosKanellos]

Thanks for the awesome video!
I am setting up a LLM/RAG project where I want the LLM to analyze log files. I noticed that the upsert actions into the chromadb take a long time, even with relatively small log files.
Inserting around 8000 chunks can easily take up 10 minutes. I believe the lack of concurrency in the underlying SQLite architecture is the problem.
Are there any alternatives to chromadb that I can use in order to solve this problem? I still want to run everything locally.
Thanks in advance.

[BanibrataDutta]

Loved the tutorial. Would it be possible to use LM-Studio instead of ollama, through the openAI compatible API interface for this ? On MacOS, LM-Studio is a bit more advanced around the support of the ANE (Neural Engline coprocessor), through the MLX framework, and I already use LM-Studio so this would be a good alternative if feasible.