Chunking in RAG (Retrieval Augmented Generation) with hands-on in LangChain and LlamaIndex

Показать описание

Retrieval Augmented Generation or RAG is becoming the go-to approach to address the shortcomings of LLMs like hallucinations and model training cut-off. In the video series on RAG, this video is about chunking the input text to be ingested into the Vector DB used in the RAG pipeline.

Hope it's useful.

⌚️ ⌚️ ⌚️ TIMESTAMPS ⌚️ ⌚️ ⌚️
0:00 - Intro
0:13 - RAG refresher
1:04 - Ingestion in RAG
1:27 - What is Chunking?
2:05 - Why Chunking?
4:06 - Fixed-Size Chunking
7:15 - Recursive Chunking
10:18 - Document / Code Chunking
12:17 - Semantic Chunking
16:40 - Conclusion

RELATED LINKS

🛠 🛠 🛠 MY SOFTWARE TOOLS 🛠 🛠 🛠

MY KEY LINKS

WHO AM I?
I am a Machine Learning researcher/practitioner who has seen the grind of academia and start-ups. I started my career as a software engineer 15 years ago. Because of my love for Mathematics (coupled with a glimmer of luck), I graduated with a Master's in Computer Vision and Robotics in 2016 when the AI revolution started. Life has changed for the better ever since.

#machinelearning #deeplearning #aibites

Рекомендации по теме

Chunking in RAG (Retrieval Augmented Generation) with hands-on in LangChain and LlamaIndex

Chunking in RAG (Retrieval Augmented Generation) with hands-on in LangChain and LlamaIndex

What is Retrieval-Augmented Generation (RAG)?

Chunking Best Practices for RAG Applications

Advanced RAG: Chunking, Embeddings, and Vector Databases 🚀 | LLMOps

When Do You Use Fine-Tuning Vs. Retrieval Augmented Generation (RAG)? (Guest: Harpreet Sahota)

Retrieval Augmented Generation (RAG) | Embedding Model, Vector Database, LangChain, LLM

Understanding Embeddings in RAG and How to use them - Llama-Index

Choosing the right Chunk Size for RAG

Advanced RAG (Retrieval-Augmented Generation) Techniques

Building Production-Ready RAG Applications: Jerry Liu

Practical RAG - Choosing the Right Embedding Model, Chunking Strategy, and More

Semantic Chunking - 3 Methods for Better RAG

How to Set the Chunk Size in Document Splitter | RAG | LangChain

SubDocument RAG: If You Are NOT Using This, You're OUTDATED Already! (step-by-step LlamaIndex)

Semantic Chunking for RAG

Optimizing RAG With LLMS: Exploring Chunking Techniques and Reranking for Enhanced Results

LangChain Parent-Child Retriever for better RAG

Advanced RAG tutorial with Llamaindex & OpenAI GPT: Sentence Window Retrieval vs Basic Chunking

5 Chunking Strategies for better RAG | Better Retrieval | Text Splitting Methods | RAG #ai

Intro to RAG (Retrieval Augmented Generation) - Augment AI models with external memory

LangChain - Advanced RAG Techniques for better Retrieval Performance

What is Retrieval Augmented Generation (RAG) - Augmenting LLMs with a memory

Semantic Chunking Strategy | RAG Chunking | HuggingFaceEmbeddings | LLM | Gen AI | Better Chunking

Semantic Chunking for RAG