LangChain + OpenAI to chat w/ (query) own Database / CSV!

In part 2 of this series we use natural language (english) to query our database and CSV. LangChain integrates with GPT to convert natural language to the corresponding SQL statements, or pandas command, which is then executed to return a natural language response.

LangChain is a fantastic tool for developers looking to build AI systems using the variety of LLMs (large language models, like GPT-4, Alpaca, Llama etc), as it helps unify and standardize the developer experience in text embeddings, vector stores / databases (like Chroma), and chaining it for downstream applications through agents.

Mentioned in the video:

- CSV to Sqlite in 5 lines of code:

All the code for the LLM (large language models) series featuring GPT-3, ChatGPT, LangChain, LlamaIndex and more are on my github repository so go and ⭐ star or 🍴 fork it. Happy Coding!

[RCT]

Hello Samual, thanks for giving me a new idea.

[changkeunlee]

excellent tutorials ~💯 thank you for your efforts!!!

[harrydadson]

Thanks for this Tutorial, subscribed! :)

[uakki]

Thanks for the video. My question is: If we extend the idea of this implementation, to also generate relevant plots or dashboards, after we get the response from the SQL query. How can this Auto generation of dashboards be done, if we're querying from a cloud DB?

[anjuvikherssitio]

Hi Samuel, which prompt are charged? table structure only?

[eliwilner]

Great video. I was wondering how can the openai deal when there is a typo in one of the sentences you wrote that doesn't match the entires in the database

[RaushanKumar-utke]

Hi Samuel, do we have any method to connect multiple schema for database chatting?

[JoEl-jxdm]

Hey man would it be cool of i use firebase as my database instead of SQL?

[pnfjqkl]

Great video! Can you make one where you involve prompting when you query the sql server?

[rameshh]

Is it possible to plot charts on top of sql data ?

[aaronwilt]

I've been working with the csv loader agent and running into a lot of parser errors. have you run into that yet? Ex. OutputParserException: Could not parse LLM output -- from what i've read and seen in the langchain discord, sounds like we need to override the parser somehow to re-organize the output before processing it. That is a bit above my head so far, maybe a cool video to do if you have time

[kenchang]

Hi Sam, thanks for the video. Is it possible to do a similarity search on the contents of a column, for example a description column, in a csv file or do I need to import the contents of the description column into a vector Db and then do a similarity search against the vector Db. Thanks in advance for your advice.

[germanlujan]

Thank you very much for this video! is it possible to get as result a dataframe or another kind of data table wich with you can work to?

[ryan_nuvo]

Are you able to load multiple CSV files?

[proxWeddall]

is there a condition in the prompt or something that can be added in order for the agent to only seek the required data columns as i have a csv with 111 columns and i keep hitting the maximum number of tokens

[moreshk]

From time to time the agent goes into a wrong direction of querying and then eventually stops executing and the chain is finished. Not sure if this is a restriction on how long the process can go. Works for simpler stuff.

[evanshlom]

the csv version makes it perfect for a kedro pipeline to manage the data engineering pipeline with the chat agent pipeline

[WondrousHello]

Great video!

I’m confused about whether this is storing the csv in some vectorised form. Surely for a larger csv, the request would be far too much for the api to handle? Also, is there anyway to track how many tokens you’re using per .run() query? I really wanna try this but am scared to burn through all my tokens 😅

[RandomShart]

Thanks for the tutorial, subscribed!
Few questions:
1 -- Will you do an additional video to show a db with some more complexity, e.g mutliple tables that require joins?
2 -- What would be the options to ensure security with such a model, so that access to data can be properly controlled? Would it require different views of the data for different users?
3 -- How would you (personally) validate the results and accuracy of the output you received? In a large organisation there will typically be an analyst responsible for creating the SQL, checking accuracy and quality of output before releasing to an end user for use in decision making etc. Is there a smarter way to validate accuracy through training or metrics etc?

Thanks!

[shahn]

I need help to connect and chat with Oracle database,