Talk to your CSV & Excel with LangChain

In this video, we look at how to use LangChain Agents to query CSV and Excel files. This allows you to have all the searching power of a tool like Pandas but done through natural language using an LLM to help.

My Links:

Github:

#LangChain #BuildingAppswithLLMs

[bseddonmusic]

You are producing great content that's showing me how to exploit GPT. Thanks.

[Player-oznk]

🎯 Key Takeaways for quick navigation:

00:00 🗂️ *Introduction to LangChain for querying CSV and Excel files*
- Overview of using LangChain with OpenAI models to extract data from CSV and Excel files.
01:25 🔒 *Security considerations for CSV agent*
- The CSV agent runs a Python agent under the hood, caution advised for prompt injection attacks.
02:22 🛠️ *Setting up the CSV agent with OpenAI language model*
- How to create a CSV agent and configure it to minimize hallucination by setting the temperature to zero.
03:48 📊 *Understanding the CSV agent's prompt and scratch pad*
- Explanation of the CSV agent's prompt structure and the use of a scratch pad for iterative language model calls.
05:14 🤔 *Asking the CSV agent simple and complex questions*
- Demonstrating the CSV agent's ability to answer simple queries like row counts and more complex ones involving data filtering.
07:32 🔄 *Using LangChain with Excel files and custom agents*
- Converting Excel files to CSV for use with LangChain and the possibility of creating custom agents for specific tasks.
09:22 🎓 *Conclusion and practical applications of LangChain*
- Summarizing the capabilities of LangChain for non-technical users to query data and the invitation for feedback and subscription.

Made with HARPA AI

[rickeras]

Might be a good idea for a new video is Lang Flow. A GUI based tool for Lang Chain

[joelwilson]

Great stuff Sam. Quick question - How do we improve the model if it answers a question incorrectly? Is there a "training" mechanism or reward function to let them know it was incorrect?

[abbuu_]

hey Sam, great video and content in general, just a quick question, how would you go about adding short term memory to a chain with Dataframe/CSV? The dataframe or csv agents have no parameter for MemoryBuffer. There are ways to read the csv or dataframe using a separate loader, but how do you incorporate it into a chain with an llm, prompt and most importantly, a memory buffer? I am trying to make it remember the questions I asked before (memory in the same chat instance, not historically - e.g. when you correct a question the llm does not understand, "I meant X")
Thanks much

[rickmoni]

Possible to use Matplotlib or Seaborn to display Data Visualization as the additional output after we query the data? So you think this would work?

[mahenderp]

Good article with a workable example. Great work.

[ibrahim-sfod]

Hey hi sam, I have one main question.
Is there any open-source model where I can do the same thing ?
or is there any open-source even close to doing what you have done here ? maybe I can fine tune and use that.

[surajkhan]

can please tell me how can we use pinecone into this to store memory

[RudolphMatongo]

hello there, thank you for this interesting video. I am trying to replicate this notebook but I am getting errors when I try to view the agent prompt template using this line
It looks like the library has changed considerably in the time since this video was posted
Any help would be appreciated to be able to do this step

[MinuteFlipDoc]

i was not aware of this -- cool!
Welcome to LangChain
LangChain is a framework for developing applications powered by language models. We believe that the most powerful and differentiated applications will not only call out to a language model via an API, but will also:

[BerwinSingh]

Hey sam,

Great video! Can i achieve the same using Mistral or Llama 2?

[ygfrjyd]

Hey I ran into an issue which I found quite weird. create_csv_agent worked for me as in the video, but then suddenly I started getting an error while running the same code as before on the same file. The error was a token limit error. its only a 157 row csv file and again, it worked before on the same file, but suddenly even upon restarting kernel and reloading everything, it will not query because of this error. Anyone ran into this weird issue?

[RedCloudServices]

Sam can you make a video showing how to get a reply as a Plotly chart? or a PyVis with networkx graph?

[Miko]

In a database of cars would LangChain be able to compare cars with everything about them (brand, series, model, HP, option list, etc) to another to give me a good comparison car for example a Mercedes A-Class to an Audi A3 or something like that.
Series and model would an input from myself for which car could compare to what and some it should solve itself by comparing body types etc, but option list is not normalised for different car producers. Would vector embedding be needed for that?

Or is a different model a better solution? For example BERT?
Would be grateful about a response, thank you.

[clray]

Would it be capable of doing (complex) joins between SQL tables to answer arbitrary predicate logic questions using a database?

[TienPham-rxgk]

Hi Sam, thank you for this great tutorial. If possible, can you also show us how to use HuggingFace models for the csv agent? Also, do you have any recommendation which LLMs from Huggingface is great for this kind of task? Look forward to hearing from you soon.

[adriangabriel]

could you make a video on how to correctly use a csv_agent in langchain with alpaca? I have tried the approach you showed with Alpaca and it doesn't seem to produce good results at all, so I would be curious to see how you go about it

[ambresh]

The videos are great. Very helpful. I've a question. After loading the csv file using CSVLoader, which custom chain/agent I can use? Can you share some insights on that? Share any reference/notebook if possible.

[kennethleung]

Great stuff Sam. Looks like those legacy Excel spreadsheets with macros and multiple indexes still require plenty of cleaning and preprocessing before we can use any agent on them