Your NVIDIA GPU can power a free ChatGPT (Chat with RTX)

Chat with RTX is a ChatGPT-style application that uses Retrieval Augmented Generation (RAG) to allow you to chat with your own data on your own local NVIDIA GPU/PC.

Links mentioned:

PDF links:

Other links:

Connect elsewhere:

Timestamps:
0:00 - Intro
0:17 - Chat with RTX requirements
054 - What is Chat with RTX?
1:51 - Downloading Chat with RTX
2:06 - Installing Chat with RTX
2:09 - Make sure to upgrade your NVIDIA Graphics Driver
2:41 - Chat with RTX setup steps
3:04 - What are INT4 models?
3:25 - Opening Chat with RTX
3:48 - Chat with RTX running in the browser
4:33 - Allowing Chat with RTX to work with your own custom data
5:11 - Turning your own custom data into embeddings
6:43 - Question 1: What are embeddings?
7:18 - Question 2: What is the attention mechanism?
7:51 - Question 3: What is matrix multiplication?
8:31 - Question 4: What is Charlie's secret project? (asking a novel)
9:06 - Question 5: What are the macronutrients? And their functions?
10:07 - Outro/NVIDIA GTC

#AI #NVIDIA #GTC24

[astrocoastalprocessor]

🎉 Thank you! This is just the kind of video I've been hoping to find

[ibriarmiroi]

Thank you very much as always. Though out of the context of tis video, i was wondering what would be the replacement for Tensorflow examination Certificate since it would be discontinues on 31st of may this year ? and also, would be possible to make video about the available Certificates online ? since your expertise opinion on each Certificate would be Gold ! thank you

[amitkumarchejara]

This is awesome, but I don't own a GPU yet, still learning!!

Do you know whether is it possible to have something similar to DALL-E with this tool?

[MK-ceim]

this used to work fine but after changing my folder path for the docs, I can no longer open it. I wish I can change it somehow without opening it

[fabiosoave]

Thank you for the very comprehensive tutorial! I'm interested in creating an interactive chat BOT to respond to Facebook messages related to medical topics. What I'm asking is, given the issues some have reported regarding the management of very large datasets of scientific articles, how can we obtain a system that is both comprehensive and effective? How can we organize the material questions correctly to make the most of the program's capabilities? Thank you.

[MED_Laaguidi]

can i use it to translate a pdf from english to other languages by keeping the same style and designs of the pdf?

[dexterroy]

My GPU is an older RTX 2070. After downloading this nvidia package, I realised it's not going to run on my GPU.

I created a similar chat bot that can do Q&A using a pdf file of your choice. I'm using llama-2-7B there. I had to use Google Colab to get 16GB GPU. But man, it's so slowww..

@Daniel Can you please do a video on pdf chat using langchain and one of those free LLM models?

[justusbatekerd]

Did you have any errors when you installed the software? I have had numerous errors with the sentence-transformers not being installed. Can anyone help with this?

[FernandoLXIX]

Don't load a huge dataset. I uploaded every college digital file I had and the thing went crazy. After a few hours of it trying to "digest" the data I killed the program and had to reinstall because there doesn't seem to be a way of telling it to stop reading your data. I'll try again with a way samaller dataset as you did in the video.

[bigdhav]

I have a 4070, but i cant get it to work due to some strange dependency.

[sidrajhans]

This beautiful RTX 4090 is love. I wish I had one. If you ever feel like donating, sign me up.

[AnshumanKumar]

Will this on my RTX 3060 ? I have a laptop.

Wish I bought the RTX 3090 back then, I could have afforded it.

Still can but now I am hearing of layoffs in my org, so I am gonna have to save up in case I get axed.

EDIT: It doesn't seem it can. I wish my 3060 had like 16 gbs of VRAM