Local LightRAG: A GraphRAG Alternative but Fully Local with Ollama

In this video, we explore how to set up and run LightRAG—a retrieval augmented generation (RAG) system that combines knowledge graphs with embedding-based retrieval—locally using OLLAMA. This video provides a step-by-step guide on cloning the repo, configuring local models like the Qwen2 LLM, adjusting context windows, and visualizing knowledge graphs generated from example data such as "A Christmas Carol" by Charles Dickens.

LINKS:

💻 RAG Beyond Basics Course:

Let's Connect:

Signup for Newsletter, localgpt:

TIMESTAMP
00:00 LightRAG with local models
01:38 Setup with Ollama
02:53 Serving Embeddings with Ollama
03:40 Changing the context window of the LLM
07:00 Configuring the Ingestion process
08:12 Advanced RAG Course
09:14 Indexing and Knowledge Graph Creation
10:45 Testing it out with local models

All Interesting Videos:

[optimistic_dipak]

Great Video!!! Kindly make more videos on LightRAG and all the latest cool technologies please. You are my one stop source to learn and know about new technologies. Thank you so much!!

[RodCoelho]

Is there an institution that ranks RAG systems? For example I would like to find out if this or multi-modal RAG from your recent video works better? Would you know?

[richardkuhne]

Yes we are interested, please add multi modal and pdf processing. Also use a cheap model with prompt caching for the chunking etc. and a smart model with large context window for retrieval. To get accurate results that are vetted out. I.e gpt4o-mini for ingesting, Claude 3.5 sonnet for retrieval or so

[sammcj]

I'd like to see a RAG system specifically built for working with large code bases. Most rag examples are optimised for document retrieval and citation, but I think there's a lot of room for advanced code modernisation / rewriting augmented with rag simply to enable working with large code bases (e.g. >100k tokens)

[SurajPrasad-bfqn]

Thank you, your videos are helping me a lot, please keep uploading such videos

[henkhbit]

Thanks for the update of lightrag with ollama. I am curious if you feed lightrag with a bunch of documents and how it impact the query/inference performance. In standard rag we store the embeddings in a vectorstore. Is this possible with lightrag? It would be nice to see an example with more complex documents and the embeddings stored in a vectorstore with an open source llm (for cost savings ;-) )

[BrunoEsteves-jl]

How can we make it return the "reference documents" that it uses for answering?

[TCC_thecuriouscat]

I was able to use both latest Phi and Llama models with Ollama and it works very smoothly with LightRAG. For large set of files, I was able to create Knowledge Graph based conditional Filter for LightRAG GraphML files which increased efficiency drastically otherwise hybrid query takes much longer.

[Othit]

wow, got it working, thank you so much. it took the better part of one day on my non-GPU laptop.
next step is to repeat this with some cloud-based GPU horsepower.

[angelfeliciano]

Thanks so much for your tutorials and demos. What if the data is related to products and I already process a txt with 200 products. Then next day the price is updated in 5 products. Do I need to process the whole list again? Does the old price will be remembered or it will be replaced from the rag?

[morristai]

is it possible to combine multiple article and build one big knowledge graph?

[brucewayne]

What about an existing knowledge graph in neo4j for example ? Can you enrich an existing graph ?

[mahmoudsamir]

Thanks for that. I am confused with the types of queries, what are naive vs local vs global vs hybrid ?

[SejalDatta-lu]

Everything seemed to have been taking extremely long. Not sure why on a RTX 4080 12GB VRAM.
When I typed in Ollama ps, during the launch, II saw Qwen working but not the embedding model.Perhaps this is the problem. Does anyone have an idea why the embedding model wouldn't be running? Any tips?

Thanks in advance

[greenhost]

I can't get "hybrid" mode to work with Ollama, it think like 10-15 minutes and print something unreadable as result... I try the example from repo without any modifications

[thangdoan]

How can I give a chunked csv or json file as input?

[MeinDeutschkurs]

What about options={num_ctx=32000} at the function? Is it not supported?

[SurajPrasad-bfqn]

Can we use Light RAG for documents that contain images/tables and charts?

[SuperJg]

I am getting this error when I run rag.insert

RuntimeError: This event loop is already running

Any idea on how to fix this?

[benjaminbirdsey]

Can you summarize what they are using for the edges in their graph?

Also, since the graph relations are in some generic text modes (json?) can you generate the graph in one LLM and run it in another LLM? Advantages? Disadvantages?