This Isn't Just A Chatbot (OpenAI Should Be Scared...)

I heard nvidia was doing some chat bot stuff, but Chat With RTX ended up being much more interesting than I expected. Retrieval-augmented generation (RAG) is a fascinating new technique and I'm curious how we see it adopted over time. Compared to ChatGPT and ollama, this is very different.

"insert statement about Tensorflow for SEO reasons here"

Sources:

S/O Ph4se0n3 for the awesome edit 🙏

[yowwn]

That start with linus is makes my heart drop.

[hugazo]

Finally one use for my 4080 that doesn't involve crying trying to play cities skylines 2

[MrSofazocker]

ONLY addition, it HAS to support markdown.
Imagine, just setting this to your obsidian vaults folder path and boom, you can chat with your second brain 🤯

[MrLenell]

It's not training the model just doing RAG. Retrieval is basically querying for relevant docs based on semantic similarity basically doing a sql query which a vectors in the where clause

[YomiTosh]

Hey Theo, just wanted to point out a few inconsistencies. RAG doesn’t train a model, it indexes the text files in a vector database and uses word similarity to look up relevant text. So the model, such as llama2 or mistral is unchanged but it is able to add context and make the retrieved text more conversational.

There are loads of great AI/RAG projects other than Ollama out in the git seas too. Many not quite as simple or easy to use though.

Thanks for all the great videos. Already subscribed ;)

[rusyaidimusa]

I would consider giving a shoutout to the llamacpp project that serves as the backend engine to many of the open source programs like Ollama, and the many many talented engineers who brought support to so many different systems configurations.

The open source scene has been on fire since Llama dropped and running models locally has never been easier.

[tuckerbeauchamp]

Oh man, was not ready for that intro. I love LTT and your channel, that was a great little combination

[medalikhaled]

they are not directly using svelte, they are using a project OSS project called Gradio for the UI which uses svelte

[Sindoku]

I’ll definitely be checking this out this weekend when I don’t have to work. This looks bad ass!

[ofadiman]

Finally 🎉 I couldn't wait any longer for ray tracing support in my chat bot GUI

[SenorRobinHood]

Would this work on the codebase for a library? For example inputting a freshly downloaded wordpress directory and then also digesting the wordpresss developer docs to make it your private Q&A tutor for platform you're trying to learn?

[Petyr]

Feedback: Superb video, more AI stuff from you would be great. Specially with open source stuff with our own data.

[adam_k]

Good stuff! Could you make a video on how well it performs as a coding assistant?

[aloufin]

Yes, your explanation of RAG was very nice and easy to understand

[DaniDipp]

This is huge for my wiki. I can just give it a directory of markdown files. 🤯

[niteshbaskaran]

If all of the python docs were fed to an LLM model, would you use query that LLM model or still refer to the original docs?

[pcvnymr]

I have to admit, that is the MOST creative L&S I've ever seen on here. And I normally swear at the screen in response.

Maybe.

[arianj]

Could you make a small RAG project :-)?
Or do you have a channel who is like the theo of open source LLMs?

[arnaudlelong]

You and Prime need to get with this soon

[jzeltman]

Would love to see more AI content. Great look into this new release from NVIDIA