Chatbot Answering from Your Own Knowledge Base: Langchain, ChatGPT, Pinecone, and Streamlit: | Code

In this video, we take you through the process of creating a chatbot that leverages the power of Langchain, OpenAI's ChatGPT, Pinecone, and Streamlit Chat. This chatbot not only responds to user queries but also refines queries and pulls information from its own document index, making it incredibly efficient and capable of answering in-depth questions.

Building a Document-based Question Answering System with LangChain, Pinecone, and LLMs like GPT-4.

Revolutionizing Search: How to Combine Semantic Search with GPT-3 Q&A

Building a GPT-4 Chatbot using ChatGPT API and Streamlit Chat

🚀 Top Rated Plus Data Science Freelancer with 8+ years of experience, specializing in NLP and Back-End Development. Founder of FutureSmart AI, helping clients build custom AI NLP applications using cutting-edge models and techniques. Former Lead Data Scientist at Oracle, primarily working on NLP and MLOps.

💡 As a Freelancer on Upwork, I have earned over $60K with a 100% Job Success rate, creating custom NLP solutions using GPT-3, ChatGPT, GPT-4, and Hugging Face Transformers. Expert in building applications involving semantic search, sentence transformers, vector databases, and more.

[youssefaserrar]

Great video. First time I hear about the Refined Query 👍

[ngalenal]

You are my hero. I have a academic project in NLP and your tutorial is clearly explained. Thank you for your work. 🙏🙏

[znnjcqr]

Can you please mention versions to install as well, as i am getting error " installed packages version mismatch " while running the code

[keshavchander]

Can i use this to ask questions to the code base, for example : Find the repeated lines or clone lines in the repo .Will it answer

[robertovillegas]

have a use case with a haven't seen anywhere: I create a private GPT that has documents as contexts. This documents has criteria for a specific subject I give it system instructions so its function is to evaluate a user document that it's attached as a part of the prompt, to see if the document complies with the criteria in the context documents, and give detail response on the result of evaluation and the justification that includes, the content of the user document and the criteria in the context documents. I want to do that in LangChain but I don't know hot to add a user document as a part of the prompt for the RAG.. It would be great if you can you explain how to approach this implementation. Thanks you for the content!! Keep the good work.

[vashistnarayansingh]

So behind the scene what we are doing, is doing the similarity may be using cosine similarity of the input query and all the 30 doc embedding present into the ventor db and returning the one with highest similarity

[TheDigitalSight]

If I'm creating custom facing chatbot and I've to upload pdf of multiple customers how should I store it in pinecone? Can you please explain more on pinecone usage and how its used in wild?

Thanks for your easy to understand explanation 😇

[manikantasurapathi]

Hey, how to erase the user query after we generate a response to the user query. That’s one flaw I’m not able to sort.

[tzengyueng]

🎯 Key Takeaways for quick navigation:

00:00 🤖 The video discusses building a chatbot using Langchain, OpenAI's ChatGPT, Pinecone, and Streamlit for answering questions from your own documents or knowledge base.
01:09 📄 The chatbot answers questions from documents stored in Pinecone, demonstrating the ability to retrieve information from custom knowledge bases.
02:08 🔍 Refining queries is essential for accurate semantic search, helping the chatbot find relevant information based on user questions.
04:14 📊 The video covers indexing documents using Pinecone and the importance of choosing the right dimension for the index.
05:49 🧩 Langchain facilitates document processing and chunking for indexing in Pinecone, supporting various document formats.
11:57 📈 Demonstrates how to push document embeddings into a Pinecone index, enabling semantic search capabilities.
13:08 💬 The video utilizes Streamlit Chat for the user interface, displaying responses and user queries.
16:36 🧠 Explains the different types of memory (buffer, summary, etc.) used in Langchain to maintain chatbot conversation history and context.
18:14 📝 Discusses the use of prompt templates for instructing the chatbot's behavior and structuring conversation prompts.
20:01 🧩 Langchain is used to create chat prompt templates, message placeholders, and maintain conversation history.
20:30 🔗 A "chain" in Langchain combines memory, template, and language model components for chat interactions, such as question answering.
21:38 📝 Users input queries, which are refined for semantic search using Pinecone indexing.
22:49 🔄 Langchain's line chain handles queries and responses, utilizing ChatGPT for generating responses.
25:21 💬 The "conversation string" combines past conversations and the current query, allowing for refining queries to improve semantic search.
28:21 🪶 Refined queries are sent to Pinecone for context searching, enabling more accurate answers based on the conversation context.

Made with HARPA AI

[delphiDHS]

Great video.
But can you please help me resolve this error: ValueError: variable history should be a list of base messages, got
I have search everywhere nothing useful.
My code same as yours btw.

[cryptokingdom]

Brother thanks for the tutorial and i have a doubt

What is the maximum capacity of Sentence Transformer Embedding - all-MiniLLM-L6-v2 ?

Is possible to convert 50k words into an embedding format with sentence transformet embedding model or any other model available to achieve this?

[jocidcu]

I am getting error in "conversation.predict" method. Did anyone faced this issue?

[Priyanshukumar-zuei]

Hi sir, I'm getting n authentication eroor that says you didn't provide an api key whereas I have given api key the error is, AuthenticationError: You didn't provide an API key. You need to provide your API key in an Authorization header using Bearer auth (i.e. Authorization: Bearer YOUR_KEY), or as the password field (with blank username) if you're accessing the API from your browser and are prompted for a username and password.

[AaronAsherRandall]

What framework are you using to build out the actual UI in the browser that the user interacts with?

[hernandocastroarana]

Thanks for the video. Do you know what components I can make the Chatbot throw citations at the end? something similar to pihind or perplexity. I want it to extract the information from websites or documents, but I want it to tell me where it extracted the information (citations) from. I would really appreciate the help

[arora.ishant]

@Pradip- Thanks for video. However, I am getting SSL authentication error when running streamlit code on local machine. Did you do anything specifically to take care of that? How to resolve that?

[kunal]

Love your videos Pradip. Is it possible to define the query refiner function using an open source model like the latest MPT-30B-Chat, if yes how? I am finding it difficult to define the query refiner function

[tasneem]

Is it possible to change the icon image of the bot and user? How do I do that?

[justinechternach]

any recommendations for preparing your own code (in my case c#) to put into pinecone and provide as question context to chatgpt? Specifically the chunking part.

[kenzabaffoun]

hey, I really appreciate your work, and I really find myself learning with you, so I sincerely thank you for that.
I also wanted to ask you, I'm in the process of creating a recruitment bot for my company, I wanted to know if it was possible to reverse the bot's process, i.e. have it initiate the conversation and ask questions, not the other way around, but still keep its intelligent aspect? While indexing the company's own database in case the candidate had any questions to ask?