Is Tree-based RAG Struggling? Not with Knowledge Graphs!

Long-Context models such as Google Gemini Pro 1.5 or Large World Model are probably changing the way we think about RAG (retrieval-augmented generation). Some are starting to explore the potential application of “Long-Context RAG”. One example is RAPTOR (Recursive Abstractive Processing for Tree-Organized Retrieval), by clustering and summarizing documents, this method lets language models grasp both general concepts and granular details in the individual documents. Inspired by LangChain, we tested out constructing a tree-based long context RAG. Watch the video to find out whether this approach allows us to say goodbye to the lost-in-the-middle effect commonly seen in large language models. AND, most importantly, how knowledge graphs can come to the rescue to enhance the answer quality. 😉

0:00 Coming Up
0:30 Intro
1:41 What is RAPTOR?
2:35 Knowledge base for RAG
3:13 Constructing a Knowledge Graph with Diffbot API for RAG
3:59 Construct tree-based RAG with RAPTOR
4:22 Test questions on tree-based RAG!
6:45 Reference our knowledge graph for answer enhancement
7:11 Are answers enhanced by our knowledge graph?
8:25 What are your takeaways?

Want to turn your unstructured text data into knowledge graphs?

[ronifintech]

Love it! Finally Neo4j has good usage!

[watch]

1:23 reference to pineapple pen video.
it is nice to see somethign like that.

it is nice video . Keep going . I liked how you are telling subjects.

[KevenLiu-ut]

Great output! We are now need more exploration on KG with LLM. I am sure it's a must have. Please share more thoughts.

[Armenian-abrank]

Thank you for your explanation of integrating KG to RAG

[Taskade]

Super awesome, thanks for sharing this, useful for anyone getting strated w/ RAG :)

[madrush]

Your videos are fantastic! Entertaining, educational, and still highly technical.

[kayhal]

Do you think the same depth and accuracy could be achieved with a metadata-filtered vector search followed by a reranker? I worry about having to maintain two data representations that are prone to drift

[badashphilosophy]

before i thought the issue was a combo between input and output tokens, and figured llms just ommitted vital info because their output was constrained, now i know finding vital info is an issue also with rag. interpretation systems, thanks

[daniel_tenner]

Definitely learned a lot, and now keen to see some kind of tutorial on how to implement the knowledge graph backed RAG. I’m a Ruby dev so something language agnostic would be particularly helpful.

[wenliangzhang]

Thanks for the video and putting everything together! I also have a few questions
- Is you conclusion: Graph > RAPTOR > Regular RAG ?
- Not sure if there is any benefitial to save Raptor embedding into Neo4j graph since it has a hiarchy tree structure? I saw in the code it is saved into Chroma vector db
- How efficient is the clustering process? If it doesn't provide much benefit as you showed in the video comparing to Neo4j implementation. I wonder if it is worth the time.

[jaeboumkim]

Great explanation!
I have a questions: KG requires a strict structure, such as node, link and attribute, which depends on target domain and questions. How can we alleviate it?

[somjrgebn]

Do you know of any efforts on converting these entities and relationships further into formal logic representations?

Being able to pair these graph databases with formal logic representations would definitely help improve the quality of written text, organic exploration/discovery, and understanding over time.

[mordokai]

Recently, some compelling papers have discussed using LLMs to develop custom pseudolanguages that enhance RAG retrieval, logical reasoning, and planning through algorithmic thinking, such as described in 'Language Models As Compilers'. I've been working on a similar project for two years now, where I've developed a subpersona/agent-function for my custom GPTs and agent workflows called Mai-Ku (Macro-Assembly Integrated Compiler). Additionally, I recommend checking out videos and papers on employing the Wu method in training and prompting. The Wu method not only enhances models' capabilities in mathematics but also teaches them to apply algorithmic thinking universally. This approach allows models to decompose and tackle novel problems in zero-shot scenarios effectively. As a result, their performance improves across all domains, not just in mathematics. This broader application underscores the method’s potential to significantly enhance the adaptability and problem-solving abilities of models in various settings.

Your Video Transcript as Mai-Ku format:
"!!{Exploring Long Context Models and RAG with RAPTOR}

>: {Video Introduction}
!{Context}: "Introduction to the impact of Long Context models like Google Gemini Pro 1.5 and the Large World Model on retrieval-augmented generation (RAG). Discussion on their capabilities to handle extensive documents and their potential to eliminate the 'lost-in-the-middle' phenomenon.";

>:{Exploring RAPTOR's Approach}
!{Explanation}: "RAPTOR (Recursive Abstractive Processing for Tree-Organized Retrieval) enhances RAG by clustering and summarizing documents into a structured, tree-based format, allowing for efficient retrieval of both broad concepts and detailed information within large text corpora.";

>:{Integration of Knowledge Graphs}
!{Utility}: "Incorporation of knowledge graphs to aid and enhance the RAG process. Demonstrates how knowledge graphs can support RAG by providing structured and interconnected data that improves the accuracy and context of retrieved information.";

>:{Practical Application and Testing}
!{Implementation}: "Detailed steps of constructing a tree-based RAG using the RAPTOR methodology, complemented by the creation of a knowledge graph using Diffbot API, centered around the works of Hayao Miyazaki.";

>:{Enhancement of RAG with Knowledge Graphs}
!{Comparison}: "Comparison between traditional RAG outputs and those augmented by knowledge graphs, highlighting significant improvements in the depth and relevance of responses.";

>:{Dynamic Content Engagement}
!{Interactive Elements}: "Encourages viewer interaction by presenting direct comparisons of RAG performance with and without knowledge graph enhancements. Prompts viewers to consider the implications of integrating structured data sources into RAG systems.";

$: {Conclusion and Call to Action}
!{Summary}: "Final thoughts on the transformative potential of long context models and RAG when combined with knowledge graphs. Invitation to viewers to explore further and experiment with these technologies in their own projects.";

$: {Viewer Interaction}
!{Engagement}: "Solicits feedback and insights from viewers on their experiences and takeaways from the video, fostering a community discussion around the future of RAG and knowledge graphs.";

$: {Additional Resources}

[adennyh]

Thanks for the awesome summary about raptor

[ML-bfbz]

This is a fantastic video. Where the RAG fails due to potential lost in the middle, is there any trace or context to confirm it was provided to the LLM?

[alchemication]

It's a nice tutorial and I am super excited about trying KG's in education space.
It's probably not the purpose of this vid, but the RAPTOR clustering seems to be good to a static set of docs, like wiki, however would anyone know how can we handle constantly added new content? Do we then run the whole (expensive) thing over and over again?

[rembautimes]

Thanks for the video . Entertaining and knowledgeable 😂

[amarnamarpan]

The most valuable contents on youtube are the least liked.. Often..

[freeideas]

Another problem with large context windows is time and especially EXPENSE. If you have to pull a significant amount of your corpus into a prompt, you will probably have to pay per token and you will certainly have to wait per token. Some type of RAG will probably always be needed. Does a human brain review its entire contents before answering a question? Of course not. We ourselves are a type of RAG system.

[wryltxw]

due to the RDF structure, don't you lose information in constructing such knowledge base? text is richer in semantics than what can be encapsulated by triples?