LocalGPT: OFFLINE CHAT FOR YOUR FILES [Installation & Code Walkthrough]

In this video, I will walk you through my own project that I am calling localGPT. This uses Instructor-Embeddings along with Vicuna-7B to enable you to chat with your files offline without the internet. It's under development and I will be adding more features on a regular basis.

If you like the repo, don't forget to give it a ⭐

▬▬▬▬▬▬▬▬▬▬▬▬▬▬ CONNECT ▬▬▬▬▬▬▬▬▬▬▬

▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
LINKS:

▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
All Interesting Videos:

[engineerprompt]

What are you going to create with this tool?
|🔴 Join Patreon:

[bahramboutorabi]

As usual, well structured and easy to digest. Great presentation, thank you.

[ScottWinterringer]

Perfect! you stated exactly what I wanted to know in the first 60 seconds :) Thank you for this

[devthinkai]

Another GREAT!! tutorial. You present these concepts in a very understandable manner, natural born teacher!!

[yousuze]

Nice work. I was about to create similar project by forking PrivateGPT but your video come right on time. I will create a PR.

[leojax]

Thank you for your work. Very clear and informative.

[michaelbellamy]

Great project. It would be super educational if you could discuss the prerequisites starting with a fresh Windows install.

[xavicastello]

Great work and thank you for explaining in simple terms how this works

[uwemisrael]

Hands down one of the coolest things I've seen this year

[Sancarn]

Instead of dumping your files into a folder, it'd be nice if you could list a set of directories / grep paths to find files in automatically.

[serikazero]

I do have a simple question as I'm interested in this topic but:
1) can the llm support more than english? lets say I have a document in french, could it still parse it and answer questions properly?
2) can we train/improve the current model? as all I've seen here is organizing the document in vector arrays to create a knowledge db, and then simply trying to query the db using an existing model. I'm curios if we can tweak with the model itself?

[StijnSmits]

Great video & repo. How do you finetune the Embeddings model InstructorEmbeddings to improve performance in this case for Causal Question input & Answer output? Do you have I link where I can find how to finetune or download the finetuned model for QA-Retrieval? And lastly, if there's already a finetuned model avalaible for QA-Retrieval, why didn't you implement this finetuned model from the get-go to achieve better results? Thanks in advance, and again great work.

[IndianCapitalMarkets]

Great video, Thanks. I have a quick question : Do you need to pre-train the LLM before starting to use it ?

[kingpatty]

I can't get to run the localgpt because chroma-hnswlib wheel failed to build. I had the VS build tools 2022 installed and everything but still failed. Error said 'iostream' cannot be found on directory.

[VastIllumination]

Quick Question: I got this running on an i7-10700KF 3.8GHZ, Win10, RTX 3080 & 32GB ram with Cuda. It works and generates correct responses, but it takes 2-4 minutes to complete each query. In this video it looks like you get near instant query results. I see my ram usage is at 28GB, would going to 64GB ram make the queries nearly instant like in your video or am I missing something else that achieves this speeds. Much thanks in advance!

[defaultcharacter]

Hi. Awesome video and everything is excellently explained. One question that i had is that, can i use the model in this instead of a model based on LLaMA?

[jpznzmf]

The video is informative and useful, but running it on the CPU is slow, taking nearly 10 minutes to respond to a single question. I attempted using an EC2 instance, but it still doesn't work. Could you recommend the necessary EC2 instance specifications for successful execution?

[nomsatsotetsi]

I understand the query takes some to produce some answers bases obviously how large the ingested documents are? but for a small document it is still relatively takes a while before it comes up with an answer. How do I enhance the perfomance

[derekisgames]

I'm looking to make a program that can act as a Dungeon Master. If I had the program ingest character sheets and world information would it be able to use that information to facilitate a game?

[WagesOfDestruction]

Could someone tell me please how at 3:37, how in visual studio do you open a local virtual environment with the screen with problems output debug console and terminal appearing.