Run Your Own LLM Locally: LLaMa, Mistral & More

Показать описание

Today I show you how you can run your own LLM instance locally at home with a tool called Ollama.

◾◾◾◾◾◾◾◾◾◾◾◾◾◾◾◾◾
📚 Programming Books & Merch 📚

💼 Services 💼

🌐 Social Media & Contact 🌐

Рекомендации по теме

Комментарии

I'd really like to see a video how you can train those models on your own documents.

matthias_schumm

I love how I used Lm studio 1 day before you posted this video 😂

Deltaforce_

Hi, thanks for the video, it is really great but since you do tutorial for python as well and I’m really interested about it, can you please provide us with a tutorial on how to run the model in python? It would be pretty useful for making own applications or voice interacting.

devox

would be nice if you could give a demo on how to run custom/fine-tuned open source models on cloud services (or even non fine-tuned models)

mmzzzmeemee

Awsome! How could i run this in VSC as to extract the response generated?

diogomarques

Do you have any videos about post-processing LLM outputs and achieving the best accuracy (e.g. we have math tasks with an exact and only one answer)?

mak_kry

Looks cool. Now all I need is a mega-spec machine :-(

daveys

I've just switched to nvidia and the model runs so much faster. I definitely can't recommend using this if you don't have a nvidia gpu.

seasong

Thanks for video, can you please also let us know, how to uninstall any module, it is eating up huge space,

malikrajat

one month later and there is still no support for windows and also no support for AMD gpu

seasong

The Flood gates just got breached. 😂 kidding, but this is awesome
i wish you mentioned ollama-webui as well. it might be more helpful for people with home lab/lab setup

HomelessDeamon

Thanks for the video! Even on my 16 Gb ram it runs but answers slowly but it s ok anyway

ekszmui

You can run most of the models without a GPU; however, it isn't as fast.

maka

is there a way to hire a VPS with things like T4 GPU by the query so we can use the high-end hardware on a query-based pay-as-you-go plan?

oliverli

Do I need to have GPU to run these models?

erodotosdemetriou

Are you using any gpu? If so what is the configuration?

mdbayazid

I am a java developer . I domt have idea about these AI stuff. How to train these systems or models when deployed locally . With documents??. From whom.and where to get . Whats the aurhenticity of those docs 🤔🤔

subramanianchenniappan

Got the Windows version, how to know that it is the 70B version of LLama2?

luminaire

If you run in Windows via WSL, can you use GPU? Since virtualization doesn't use native devices

dominiks

How to interact with these models programatically? Like in a Java server application???

WikiPeoples

Run Your Own LLM Locally: LLaMa, Mistral & More

Run Your Own LLM Locally: LLaMa, Mistral & More

All You Need To Know About Running LLMs Locally

Run your own AI (but private)

This new AI is powerful and uncensored… Let’s run it

host ALL your AI locally

Run ANY Open-Source LLM Locally (No-Code LMStudio Tutorial)

Run Your Own ChatGPT-like LLM on Your Windows PC!

I Analyzed My Finance With Local LLMs

Lecture 27: gpu.cpp - Portable GPU compute using WebGPU

'I want Llama3 to perform 10x with my private knowledge' - Local Agentic RAG w/ llama3

Running a Hugging Face LLM on your laptop

FREE Local LLMs on Apple Silicon | FAST!

Run LLMs locally - 5 Must-Know Frameworks!

How-to Run Your Own AI LLM For Free & Locally

Run ANY Open-Source Model LOCALLY (LM Studio Tutorial)

Ollama-Run large language models Locally-Run Llama 2, Code Llama, and other models

Replace Github Copilot with a Local LLM

Unleash the power of Local LLM's with Ollama x AnythingLLM

How To Install Any LLM Locally! Open WebUI (Ollama) - SUPER EASY!

Run Your Own Local ChatGPT: Ollama WebUI

Using Ollama To Build a FULLY LOCAL 'ChatGPT Clone'

Run Any Chatbot FREE Locally on Your Computer

Training Your Own AI Model Is Not As Hard As You (Probably) Think

This Llama 3 is powerful and uncensored, let’s run it