LangChain Advanced RAG - Two-Stage Retrieval with Cross Encoder (BERT)

Показать описание

In this Video we discuss advanced retrieval techniques from Vector Databases with Query Expansion and Two-Stage Retrieval with a Cross Encoder. We use another Reranker on top to take the famous "lost in the middle" problem into consideration.

Timestamps:
0:00 Two-Stage Retrieval in theory
2:44 RAG and it´s issues
9:31 Query Expansion
11:50 Cross Encoder Reranking
13:50 LongContext Reorder

#openai #langchain

Рекомендации по теме

Комментарии

Best 15 mins of my day! You explained every single component in the code clear and crisp! Excited to check the other videos of yours. Thanks a bunch

sivi

Very informative.👍 Love the umap visualization 2 see the query and the embeddings.

henkhbit

Thanks for the video. Perfect timing…. Need this for tomorrow.

Canna_Science_and_Technology

UniqueList = list(set(ListWithDuplicates)) to replace those nested for loops. Love your content!

fuba

I don't know the umap library, its very interesting. Good explanation about RAG advanced techniques, sucess for you!

felipecordeiro

The most productive 14 minutes of my day watching and learning from this video :)

zehehwd

This video is terrific, I'll give it a try!

kenchang

Youre vids are insanely good. I doubt there is a better ai-prog-tuber

samyio

Awesome video. So I glad I found this channel. Long shot question:

After testing several chunk/overlaps, my experimentation indicates an optimal chunk_size=1000 and overlap=200. My RAG contains about 10 medical textbooks (~50, 000 pages). However, every video I see on RAG nobody uses chunks anywhere near that large. Does it seem improbable that my ideal chunk size is 1, 000, or is there likely another variable at play?

austinpatrick

Thank, always nice videos!
Do you have a favorite german cross-encoder?

maxlgemeinderat

Can u please make a video on retrieving data from SQL using SQL agents & Runnable using LCEL. If not possible here, if you can update the same in the udemy course. It helps alot

prnmdid

Thanks for the video..

But while genering queries using llm_chain.invoke(query), facing exception related to output parser.
OutputParserException: Invalid json output:

vinaychitturi

How much computation resource (specifically GPU) required in running this cross encoder model?

verybigwoods

What's the best way to evaluate this RAG?

erenbagc

LLMChain() is deprecated and the output_parser in the examples also cause json output error.
Would be nice, if you could update the github code. Thank you
If anyone having issue with json output, here is a fix:

from langchain_core.output_parsers import BaseOutputParser

class LineList(BaseModel):
lines: list[str] = Field(description="Lines of text")

class
def __init__(self) -> None:

def parse(self, text: str) -> list[str]:
lines = text.strip().split("\n")
return lines

sumangautam

LangChain Advanced RAG - Two-Stage Retrieval with Cross Encoder (BERT)

LangChain Advanced RAG - Two-Stage Retrieval with Cross Encoder (BERT)

LangChain - Advanced RAG Techniques for better Retrieval Performance

Advanced RAG 01 - Self Querying Retrieval

Learn RAG From Scratch – Python AI Tutorial from a LangChain Engineer

Vector Search RAG Tutorial – Combine Your Data with LLMs with Advanced Search

5-Langchain Series-Advanced RAG Q&A Chatbot With Chain And Retrievers Using Langchain

Advanced RAG 02 - Parent Document Retriever

RAG But Better: Rerankers with Cohere AI

End To End Advanced RAG App Using AWS Bedrock And Langchain

Building Production-Ready RAG Applications: Jerry Liu

Chatbots with RAG: LangChain Full Walkthrough

6-Building Advanced RAG Q&A Project With Multiple Data Sources With Langchain

Advanced RAG- Langchain and HuggingFace

ADVANCED Python AI Agent Tutorial - Using RAG

LangChain Explained in 13 Minutes | QuickStart Tutorial for Beginners

LangChain 'Advanced Retrieval' Webinar

RAG + Langchain Python Project: Easy AI/Chat For Your Docs

LangChain Agents - Joining Tools and Chains with Decisions

LangChain Multi-Query Retriever for RAG

Langchain vs Llama-Index - The Best RAG framework? (8 techniques)

Advanced RAG Concept: Improving RAG with Multi-stage Document Reranking

How to build Multimodal Retrieval-Augmented Generation (RAG) with Gemini

Building Corrective RAG from scratch with open-source, local LLMs

The 5 Levels Of Text Splitting For Retrieval