You Need Better Knowledge Graphs for Your Graph RAG

RAG (Retrieval-Augmented Generation) has become the hype of Generative AI applications, so are knowledge graphs. You see lots of graph-based LLM apps out there and you're probably building one too. However, how you construct knowledge graphs determines the quality of your LLM-based application. Solely relying on GPT-4 for extracting entities and relationships without thorough evaluation will give you the garbage-in-garbage-out effect.

To get prepared for Data Day Texas 2024, I built a Graph RAG AI assistant using Diffbot API for both web scraping and knowledge graph construction. You'll see how I built it while monitoring the results throughout the video. Diffbot offers transparency in the information retrieval process and benchmarks for evaluating the accuracy of the information retrieved.

Diffbot's APIs are free to use, including the Natural Language API that was used in the video:

Note: This video is independently produced and is not sponsored by Diffbot, Neo4j, or Streamlit.

Here's the link to my Github repo for this project:

0:00 Intro
0:53 Step 1. Web Scraping with Diffbot API
1:37 Step 2. Construct knowledge graph with Diffbot Graph Transformer (Langchain)
3:31 Step 3. Customize Diffbot Graph Transformer
3:41 Step 4. Import Diffbot Knowledge Graph into Neo4j Database
5:03 Step 5. What Entity/Relationship Extraction Looks Like By GPT-4
5:41 Step 6. Meet My Graph RAG AI Assistant
7:08 Outro

#knowledgegraph #generativeai #llm #aichatbot

Music: Background Motivating Corporate by WinnieTheMoog

[JaredWoodruff]

Excellent video! Thanks Leann, I had no idea about Diffbot, i'll be checking that out for sure.
Best of luck on your GenAI Journey

[kenchang]

Thanks Leann. I'm going to have to give it a try for deeper dive with DiffBot.

[mohammedmahinuralam]

Great! Waiting for more of your videos!

[pouet]

It can be done by hand, but automatisation of this human feature is impressing. Good video!

[kwongster]

amazing video - hope to see more. this was very informational and inspirational to learn about Knowledge Graphs

[senthilkumarpalanisamy]

Excellent video, clear explanation, please do post more in the gen ai and knowledge graph space

[thomaskaminski]

KG are key for providing context to RAG. Still, I see the OWL/RDFs path outperforming LPG as it enables the user to explicitly define semantics and infer knowledge

[kingmouli]

Thank you for amazing short video, I am eagerly waiting for you to make a video on how to convert csv data into knowledge graphs and answers questions on the csv files

[AerialWaviator]

This was an interesting video. I was more focused on the process, and thinking behind using this process to organize and visualize data.

[BenjaminKing]

It's nice to see my old co-worker Michelle randomly popping up in a video. I hope you were able to meet her. She is great!

[jasonwong]

very cool! have been working on building a client-side profiling & Hyperthymesia second brain graph RAG kind of thing and really struggled with the bill with gpt graph construction! thanks!

[MrBekimpilo]

This is very insightful Leann.. cheers from South Africa

[mengni]

I feel for most of the RAG user cases, vectorDB is good enough to retrieve information for LLM. But I agree that KB is better when you need the LLM to answer complex questions with precise and explainable answers.

[Avman]

Interesting post, Lenann. Keep it up! It would be interesting to explore from a procedural perspective how graphs could supplement vector databases in RAG doc retrieval and relevancy evaluation.

[vbywrde]

Yes, to physically present yourself in multiple locations at the same time is quite challenging. My understanding is that it requires you to achieve presence on the fourth dimension. Once there, you can then enter multiple three-dimensional spaces at the same time. I wish I could do that, though I suspect it would be really disorienting at first! Best wishes!

Also, I learn something new with every one of your videos! Thank you! I really like your approach!

[janekaufman]

Very comprehensive. I am follow you 643

[mohammedshuaibiqbal]

Can you make a video on how to generate knowledge graphs for pdf books like DSM 5

[cemery]

I'm interested in creating a Little Logical Model based upon the command structure of an application and then using agents take voice to text and text to cmd. maybe with a coresponding graph view updated with current information avaiable in another window on another display screen.

[joshuacunningham]

Thank you so much for this incredible tutorial! I've discovered that "GenAI" is my newfound passion, and I hadn't even heard of the term until I watched your video. I look forward to your next video.

[rephechaun]

Hello, thank you very much for posting the video, I am very interested in the part where you also show the graoh with in the chatnot, what python packahe is that please?(, y apologies if the question is redundant, I couldn't find it in other comments)