How to Self-Host an LLM | Fly GPUs + Ollama

preview_player
Показать описание


Video written and produced by Annie Sexton.
  1. Fly․io
  2. fly.io
  3. fly io
  4. self hosting an AI model
  5. self hosting an llm
  6. self host an llm
Рекомендации по теме
How to Self-Host 0:05:26

How to Self-Host an LLM | Fly GPUs + Ollama

Self-Hosted AI That's 0:12:19

Self-Hosted AI That's Actually Useful

How to self-host 0:06:44

How to self-host and hyperscale AI with Nvidia NIM

host ALL your 0:24:20

host ALL your AI locally

Run Your Own 0:06:55

Run Your Own LLM Locally: LLaMa, Mistral & More

All You Need 0:10:30

All You Need To Know About Running LLMs Locally

Set up a 0:13:22

Set up a Local AI like ChatGPT on your own machine!

Run ALL Your 0:20:19

Run ALL Your AI Locally in Minutes (LLMs, RAG, and more)

Exploring AI's Future: 2:10:35

Exploring AI's Future: How LLMs Are Reshaping Software Development & Beyond

Self-Hosted LLM Chatbot 0:07:50

Self-Hosted LLM Chatbot with Ollama and Open WebUI (No GPU Required)

Use Your Self-Hosted 0:10:03

Use Your Self-Hosted LLM Anywhere with Ollama Web UI

Uncensored self-hosted LLM 0:05:08

Uncensored self-hosted LLM | PowerEdge R630 with Nvidia Tesla P4

Self-Host an LLM 0:00:59

Self-Host an LLM (using ONE file) | Fly GPUs + Ollama #llm #aienthusiast #ollama #aimodel

'I want Llama3 0:24:02

'I want Llama3 to perform 10x with my private knowledge' - Local Agentic RAG w/ llama3

Run your own 0:22:13

Run your own AI (but private)

Raspberry Pi versus 0:08:39

Raspberry Pi versus AWS // How to host your website on the RPi4

Self-Hosted LLM | 0:18:49

Self-Hosted LLM | CodiLime

This new AI 0:04:37

This new AI is powerful and uncensored… Let’s run it

API For Open-Source 0:08:17

API For Open-Source Models 🔥 Easily Build With ANY Open-Source LLM

Integrate web search 0:12:13

Integrate web search with your self hosted LLM

Get Started with 0:11:49

Get Started with Langfuse - Open-Source LLM Monitoring

EASIEST Way to 0:05:18

EASIEST Way to Fine-Tune a LLM and Use It With Ollama

Self-Hosted LLM Agent 0:35:21

Self-Hosted LLM Agent on Your Own Laptop or Edge Device - Michael Yuan, Second State

Wake up babe, 0:04:45

Wake up babe, a dangerous new open-source AI model is here

Комментарии
Автор

These videos are really cool. I'm not a beginner, far from it, but it is soooo nice to get this information in such a distilled manner, and from a person that clearly knows what they are talking about. So natural!

MrManafon
Автор

I see alot of explainer videos and yours are the best!, just grate content delivery and tone, prefection at all!

יהוידעשטינמץ
Автор

I don't understand, how is this self hosting, isn't this cloud hosting?

hassenalhandy
Автор

Can we use any llama based model ? In the destination xan we use the llm we have downloaded ? Imean the custom llm based on llama ?

sidiocity
Автор

Can you provide wireguard instructions you mentioned? Btw perfect tutorial :)

miro
Автор

That looks like a nice way to run an LLM for my personal use, but I’d like to also try out one of the bigger LLM models.
Is that doable at all?
Or will I need to stick to models that fit within the 40gb GPU memory of the a100 for instance?

thedavymac
Автор

Hey, I tried setting this up but I have this error:

2024-08-24T00:27:36.386 runner[***] ord [info] Machine started in 3.517s
2024-08-24T00:27:37.133 app[***] ord [info] INFO Main child exited normally with code: 1
2024-08-24T00:27:37.152 app[***] ord [info] INFO Starting clean up.
2024-08-24T00:27:37.266 app[***] ord [info] INFO Umounting /dev/vdc from /root/.ollama
2024-08-24T00:27:37.268 app[***] ord [info] WARN could not unmount /rootfs: EINVAL: Invalid argument
2024-08-24T00:27:37.269 app[***] ord [info] [ 3.718685] reboot: Power down

any ideas on what would cause this?>

TheloniousBird