Build your own Q&A KnowledgeBot using GPT-Index & LangChain - Document to Chatbot

In this Applied NLP LLM Tutorial, We will build our Custom KnowledgeBot using GPT-Index and LangChain. LangChain for accessing OpenAI and GPT-Index for Vector Index Building and Index Querying - Information Retrieval.

Timestamps:

00:00 Introduction to Q&A KnowledgeBot
02:58 OpenAI Q&A Bot Demo
04:11 Full Code Explanation
05:27 OpenAI API Key Secret Token
07:21 Input Document Text File for the ChatBot
08:15 Constructing GPT Vector Index
14:00 Constructing Ask Bot Python unction
16:40 Running KnowledgeBot Q&A
18:21 Summary and Closure

❤️ If you want to support what we are doing ❤️
Support here:

[littlecoder]

Timestamps:

00:00 Introduction to Q&A KnowledgeBot
02:58 OpenAI Q&A Bot Demo
04:11 Full Code Explanation
05:27 OpenAI API Key Secret Token
07:21 Input Document Text File for the ChatBot
08:15 Constructing GPT Vector Index
14:00 Constructing Ask Bot Python unction
16:40 Running KnowledgeBot Q&A
18:21 Summary and Closure

[Davidkiania]

This is hands down the best tutorial on leveraging existing conversational AI I have seen. Thank you very much.

[jersainpasaran]

fine tuning will be the future of employment.Thank you! you always surprise us with new news and your brilliant contributions.

[sevarakenjaeva]

congratulations, this was the only video I have seen where the code actually worked. thank you for being so precise

[abhisheksarkar]

Once again! Towards the end of the weekend, I found it to be little worthy just because of this. Thanks again Sir!

[sea_tu]

I'm glad I found your YouTube channel when studying alone. Thanks to you, I am accumulating a lot of knowledge and experience.😄

[CristiVladZ]

Token cost optimization (especially when building a chatbot and having to account for all the prompts) is definitely the subject for a video! I'm struggling with this myself.

[rajeshkannanmj]

One of the best videos from you bro. I needed this. Actually, this is going to be future of Education system.

[drramasubramaniam]

Very helpful video. Thank you I was able to successfully run the code.

[halkkoi]

You always have great idea and content

[isaiahsgametube]

thats whats up! good way to start the morning

[KunjaBihariKrishna]

I had surprisingly good results. There are some incorrect answers so I wouldn't expose it to our users. But the most interesting issue I encounter is that the bot will seemingly reference things that weren't mentioned, as though its reply is an excerpt from a documentation.
Kinda makes sense, seeing as it's using our documentation. But it will say something like "as shown above" or "as outlined above".
That string of words isn't exactly in the txt files.

For other issues I was able to use the workspace search function in vscode to remove or replace certain strings in the txt and index.json

I need to learn more about how this works so I can improve it. Once I have a full grasp I could potentially create a massive index by using our tech support history

[reneeliu]

Thank you for your video! Curious to know where does LangChain come into play in the code? It imported OpenAI but the code seems to only use the modules from Gpt index. I might be missing things here!

[TRSTNGLRD]

This is awesome - are you able to grow this series and show us how we can prevent hallucinations, improve complex results, and ask complex questions? I've built this bot but it is rarely good at giving well-rounded responses

[maertscisum]

Can you explain more detail on how to do fine tune, especially with incremental index. It will also help to share how the tokens being consumed and how to optimise it.

[nitincsawant]

Those are some points like token optimization, saving the index to local are very helpful.
Other question I may have is what type of the inputs it can deal with? I have couple of HTML files with tables in it. Can I pass it on directly to this LL model as a knowledge base? or does it need any kind of conversion to txt?

[Red-fugb]

It is an excellent tutorial, thanks! But I ran into a problem. when I use my personal document specifying that SVA and DVA should be conducted every quarter(SVA and DVA is a kind of test), and ask the bot how often I should conduct NVA(another kind of test), the answer should be I don't know from the bot, but the bot answers every quarter. How to fine tune the model to answer I don't know when it can't find the answer in my personal document?

[jamespower]

Do many or even any websites contain a .txt file that could be crawled and would work for training as you described in the video?

[arturko]

great video! I've learned so much. Please follow-up with a UI video.

[krishnakompalli]

Great tutorial! I can see the response is more like a statement. How can we improve it to a conversational level? (I mean the response should be a question back to the user) Is it purely dependant on how the fine tuning is done?