Create ChatBot Based On The Data Feed By You - GPT-Index | OpenAI | Python

Check out this video to know how you can construct your own chatbot
by giving your own data. As we are feeding our own data and creating a knowledge base, all the responses will be based on our data.

** REFERRAL LINK *************
** REFERRAL LINK *************

How-To-Fix:

###### MORE PLAYLISTS ######

#chatbot #python #openai

[njorogekamau]

This tutorial truly stands out from the rest! After struggling with coding along other YouTube tutorials on GPT-index and langchain, I finally stumbled upon this gem, and behold, my code worked like a charm. Thanks for sharing your expertise and making it easy to follow. You're a lifesaver!

[glebp]

Thank you, Shweta. I was able to get something working based on your code. Please note that lots of libraries are now outdated (gpt_index is now llama_index). My complete code (works 100%!):

from langchain import OpenAI
import sys
import os
from llama_index import SimpleDirectoryReader, GPTListIndex, GPTVectorStoreIndex, LLMPredictor, PromptHelper, ServiceContext
from llama_index import StorageContext, load_index_from_storage

def create_index(path):
max_input = 4096
tokens = 200
chunk_size = 600 #for LLM, we need to define chunk size
max_chunk_overlap = 20


prompt_helper = PromptHelper(max_input, tokens, max_chunk_overlap, chunk_size_limit=chunk_size) #define prompt

llm_predictor = LLMPredictor(llm=OpenAI(temperature=0, model_name="text-ada-001", max_tokens=tokens)) #define LLM
docs = #load data


service_context = ServiceContext.from_defaults(llm_predictor=llm_predictor, prompt_helper=prompt_helper)

vectorIndex =
docs,
)



return vectorIndex

def answerMe():
storage_context =
index =
query_engine = index.as_query_engine()
q = "What is the best plan?"
print(q)

response = query_engine.query(q)
print(response)

#create_index('data')
answerMe()

[DandaFundas]

Thank you Swetha. It can’t get any better than your explanation with show and tell 👍

[Phrixos]

Thank you so much for this! You made something very intimidating to me super easy to understand. I am very grateful for the time and effort your put into this video. Thanks a lot!

[faisalsaddique]

This is really useful, Shweta! Thank you so much for making such an awesome content.

[usmanahmed]

Very good explanation. You speak very calm and make audience to follow all step in detail. Keep good work

[tchalaful]

awesome !! very inspiring, thanks a lot for your work !

[mangesh]

Your instructions are really good, thank you so much Shweta Ji

[congtycophanxaydungo.m.d]

thanks for your contribution, it's easy to understand for a beginner like me. You can continue to make the next video to guide the question and answer continuously like chatgpt.

[wspeunf]

Hi Shweta, Awesome video, great learning. Thanks. Just one concern - when we grant access to ChatGpt to our custom data stored in our machine, then is there a risk that the data can be copied/ used by users of ChatGPT or Open AI or otherwise?

[M-ABDULLAH-AZIZ]

having data in a file and real time embeddings vs embeddings in a db for chatbot for an application (provides information about an application)?

[sdrvpsy]

Ms. Shweta, your tutorials are superb!! Stands out first from the rest.

[jeffersonhope]

Outstanding preparation and presentation. Thanks so much!

[harshilpatel]

Hi Shweta, that was a great tutorial! However, I have a question. Just like you used the custom data from the local disk here, How can we use the data from aws/elasticsearch?. I have a huge database (About 20 million records) which our employees access via querying on elasticsearch, and If I wish to create a custom chatbot trained on that data, how would one achieve that?

[LearnWithSibin]

I noticed that in the implementation shown in the video, the GPT model is being called every time a user prompt is received, and this can be expensive in a real-world scenario where the application is serving multiple users concurrently. Each request to the GPT model requires a certain number of OpenAI tokens, and this can quickly add up and become expensive.

[oreos_overdose]

Hello and thanks for the video. Very useful. Is it possible to use a sphinx generated website (essentially a handful of html pages) as the resource documents instead of a plan txt file?

[deepaksartape]

Please cover UI part as well in one of the coming video. Thank you so much in advance.

[bivwwpg]

hi Shweta, i tried this code on my end but in { } is giving me an error so i tried { } this is not creating a vectorIndex.json file on my system its creating a folder name vectorIndex.json inside which i has 4 json file docsstore, graph_store, index_store and vector_store.. can you pls tell me where i am going wrong

[ddsharma]

Question about execution flow: Hi Sweta - Thanks for a very helpful tutorial. Trying to wrap my head around the execution flow. For example, createvectorIndex probably sends the document to openAI servers, the servers return embeddings that are then stored locally. During answerMe - is the prompt sent back to openAI for embeddings? Does answerMe again send document embeddings back to openAI servers? How much is done in the local process (on PC) vs how much is done by openAI servers? Thanks.

[mikhailtrufanov]

Hi, thank you for the video! I have tried this code to answer questions based on data about a very specific product. It answers well, however, it answers general questions as well (such as "What is Earth?") having no information at all about this in the files provided. How can I make this code answer based on the information that I provided only?