Getting Started with Ollama and Web UI

Показать описание

In this tutorial you will lean how to install Ollama and run a Large Language Model like Meta AI's Llama 3.1. When you install Ollama you have access to a command line interface to talk to the LLM. You will also learn how to install a project called Web UI that will give you access to a Chat GPT like interface for interacting with the model.

🔗Resources & Links mentioned in this video:

👋🏻Connect with me:

Рекомендации по теме

Комментарии

Interesting tutorial with Web UI and Ollama, Thanks!!!

zkzmsmu

Great one Dan! Keep ups updated on the AI stuff!

AleksandarT

Ollama should integrate a feature like artifact that allow you to test your html css code in a mini webview

bause

Love your terminal, which tools do you use to customize it?

je

How can I connect my local ollama3 with webUi, My webUI couldn't find the locally running ollama3

ymtbxbv

Dan, what the specs for your local machine?

chameleon_bp

Hey, could you make a video on how to edit the login page? I want to make the login page to my liking.

trapez_yt

Note for 405B:
We are releasing multiple versions of the 405B model to accommodate its large size and facilitate multiple deployment options:
MP16 (Model Parallel 16) is the full version of BF16 weights. These weights can only be served on multiple nodes using pipelined parallel inference. At minimum it would need 2 nodes of 8 GPUs to serve.
MP8 (Model Parallel 8) is also the full version of BF16 weights, but can be served on a single node with 8 GPUs by using dynamic FP8 (Floating Point 8) quantization. We are providing reference code for it. You can download these weights and experiment with different quantization techniques outside of what we are providing.
FP8 (Floating Point 8) is a quantized version of the weights. These weights can be served on a single node with 8 GPUs by using the static FP quantization. We have provided reference code for it as well.
405B model requires significant storage and computational resources, occupying approximately 750GB of disk storage space and necessitating two nodes on MP16 for inferencing.

borntobomb

in Ollama Is there an admin dashboard for tuning the model, sir?

mochammadrevaldi

Would make a video on how to integrate llama 3 to wordpress website, making chatbot or co pilot

zolef

finally setup open webui thanks to you. i'd approached it, seen "docker" and left it on my todo list for weeks/months. I'm running gemma2 2b on my gtx 1060 6gb vram. any suggestions on good models for my size?

expire

How we can tune a model with custom data?

vikas-jztv

hello. After installing OpenWebUI, I am unable to find OLLAM under 'Select a Model'. Is this due to a specific configuration? For information, my system is running Ubuntu
24.04.

elhadjibrahimabalde