Run ANY Open-Source Model LOCALLY (LM Studio Tutorial)

Use the #UPDF to make your study and work more efficient! The best #adobealternative tool for everyone!

In this video, we look at LMStudio and I give you an in-depth tutorial of the easiest-to-use LLM software. You can run any open-source AI model easily, even if you know nothing about how to run them.

Join My Newsletter for Regular AI Updates 👇🏼

Need AI Consulting? ✅

Rent a GPU (MassedCompute) 🚀
USE CODE "MatthewBerman" for 50% discount

My Links 🔗

Media/Sponsorship Inquiries 📈

Links:

[kanishak]

I m blown by the possibilities it brings to the users who are not comfortable with the earlier present methods.

[jcl]

In your demo you seem to only use 1 GPU layer. For my "old" Nvidia 2060 with 6GB I can easly do 40 layers on the GPU and it is very fast with for example the mistral dolphin 2.2.1 Q5 models. The API feature is brilliant, I use it for developing my own agent using a system message to give it some interesting features in its output (calling functions).

[jsmythib]

I just tried the local server of LM Studio...using it, and its examples I had a c# console app setup and talking to it in about 15 minutes. Easiest API to use, maybe ever. So good I came here to mention it! :)

[Boneless]

Do you have a running list of the best models for each category? I can't always remember which one you tested last for either coding or uncensored ect. Thanks for any comments.

[godned]

LM studio is awesome . running the server and operating open source models from an IDE I was able to get it to perform pretty much on par with gpt 3 j. just a bit slower . running the server is the way to give your llm the most tokens possible for inference while you formulate your questions around json's and SPR sparse primary representation prompts in the IDE. At one point I had dolphin 2.2 telling a story for over an hour strait with out stopping and not even repeating itself until I shut it off . Massive unexplored potential there.

[kai_s]

Do they have a document upload feature, so that we can chat about our document like the custom GPTs?

[TheZanzz]

I like how with nearly no context, "Mario" just pumped out a romance novel

[ManiSaintVictor]

Just in time! Thank you. How is the MemGPT setup process? I’m gonna try this out after work. Thanks.

[kalvinarts]

I know this is very easy to use but there are plenty open source solutions to do the same. It would be good to inform about the data collection these companies are doi g on the users who use their software.

[lukasareskog]

Is it possible doing document retrieval within LMStudio? For example, a chatbot that can chat about .pdfs / .csvs / .txts, given to it?. If not possible, would privategpt be a better alternative? It seems very intuitive there.

Couldn't find anything on google.

[spencerfunk]

please do a tutorial with this for memgpt. ive been using lm studio for a couple weeks now. ive seen people get memgpt to work with the server but some people have issue, me included

[markelshnops]

Would be a little more useful if the system would allow you to upload documents so you could perform actions like summarization

[svcupc]

This looks much easier than TextGen WebUI. I haven't looked into it but I hope LM Studio will not record my usage for anything. Another interesting thing would be if we can use AutoGen or MemGPT to extend its capabilities. And if we can "chat with our own doc" using LM Studio.

[rakly]

Those 'should work' etc is not based on your system, it's about compatibility with the LM Studio app. (GGUF models)
I have 128GB system and 40GB VRAM, and it also shows the 30GB+ required warning.

[bobbytables]

LMStudio lacks local documents, what a bummer I will continue to use GPT4All

[theresalwaysanotherway]

looks nice, but I wouldn't rely on it for testing in your videos until you can specify prompt formats (there's a good chance the model might be handicapped by the wrong format, currently it only lets you edit context, not the full prompt format). Also it only uses llama.cpp, which means anyone with an nvidia GPU could double their speed by switching to ExLlamaV2 and EXL2 quants.

[issiewizzie]

got difusionbee for local picture generation
so its about time we had an easy way to use LLM on our local Maschine

Is there a way to add your own text files, datafiles etc.? So when using the chat, it also knows the specific info about a subject from the files I provided?

[pipoviola]

That was amazing. You are helping us so much introducing to all this tools. Thank you very much.

[fossil]

10:04
😂 Indeed. I think we know what its finetuned on hahaha.