Private AI Revolution: Setting Up Ollama with WebUI on Raspberry Pi 5!

Join the private AI Revolution. Learn how to set up Ollama with WebUI on Raspberry Pi 5!

🎖To join the membership at 🥉bronze, 🥈silver or 🥇gold levels, head over to

#Ollama​ #RaspberryPi​ #AI

Build a Private Chat GPT Server with Raspberry Pi 5 | Step by Step Tutorial

In this episode, Kevin offers a comprehensive guide to building a private chat GPT server that runs locally using a Raspberry Pi 5 or Raspberry Pi 4. This setup ensures data privacy and security as there is no data sent to the cloud. Kevin explores the application of a private offline AI known as Olama, detailing how it's superior to ChatGPT clone. He also shares valuable insights on how to set up Olama and WebUI to resemble the ChatGPT interface. Viewers are taken through demos on how to interact with Olama, set up the server, select and install models, and even code a simple Python program using LangChain to interact with Olama. The session concludes with an outline of the benefits of Olama, details of various downloadable models, and an interactive QnA segment.

00:00 Introduction to Building a Private Chat GPT Server
00:27 Overview of Olama: The Offline AI
00:41 Setting Up Olama and WebUI on Raspberry Pi
01:51 Exploring the Features of Olama
03:06 Benefits of Using Olama
04:49 Demonstration of Olama in Action
08:25 Exploring Different Models in Olama
14:59 Considerations for Using Uncensored Models
17:24 Choosing the Right Model for Your Needs
19:22 Setting Up a LLAMA on Your Computer
19:59 Understanding Docker Compose Files
22:03 Running the LLAMA Web UI
23:03 Creating an Account on LLAMA Web UI
23:32 Downloading and Using Models
27:55 Exploring LangChain for Python
31:15 Running a Python Program with LangChain
35:40 Exploring the LLAMA Website
37:25 Supporting the Channel and Community

[whitneydesignlabs]

Thanks, Kevin! I have been running Ollama on a Pi5 for a few weeks now. But I look forward to following your docker example and trying out the GUI. My use case is robotics and still trying to solve how to run a Pi5 at full power on a DC battery. (harder than it seems)

[jajabinx]

Is there a way to give it a voice, and speech recognition?

[NicolasSilvaVasault]

pretty good stuff, i'm planning to install ollama on a nas xd, hope your channel grows up

[jonathanmellette]

The first time you send a prompt it the WebUI loads the model into memory, so the first response always takes the longest.

[chrisumali]

Thanks for the demo and info, have a great day

[bx]

it would be awesome if you could tell it to ssh to other devices via ssh and run commands

[galdakaMusic]

Is posible remake this video with the new Hat AI Hailo 8L?

[cartermclaughlin]

Is there anything like this that can do image recognition? I do wound care and have always wondered when my computer could help with wound assessment - provide measurements, describe wound bed, maybe even temperature with the right camera.

I'm only at 15:30, but i get the impression you are not using TPU? I am just getting into raspberry pi and I'm looking for an excuse to get one. There only seem to be a couple of TPU options and I'm leaning toward the USB 3 version but I have no idea how to judge them.

[babbagebrassworks]

Got another reason to get a few Pi5's.

[hkiswatch]

Fantastic video. Thank you very much. I have a Coral TPU. Would it be possible to use it with Ollama on the Pi 5?

[peterhickman]

This looked interesting so I went to the github page and followed the instructions to run the docker version and the first thing I saw was a sign in screen with the need to create an account! Not really sure this counts as private at this point

[Bakobiibizo]

do you have a seed on your prompt? because if not you will generate the same prompt each time, with out some for of randomization a transformer model is more or less deterministic with most modern schedulers. If you do have a see you might have it cached.

[BillYovino]

Thanks for this. I'm having trouble getting started. I have a fresh install of bookworm on a RPi5. I installed Docker and created the compose.yaml file as shown in your blog. I'm getting an error when I try to run "docker-compose up -d". "yaml line 1:did not find expected key". Do you know what I'm missing?

[marioa]

Is there anyway to reliably increase the performance of a raspberry pi 5 to support the larger models in ollama?

[haydenc]

Can it run in a cluster on multiple RPi's?
ooh looks like it can on a kubernetes cluster!!!

[saokids]

Nice tutorial, possible deploy ollama as stack in docker swarm as cluster improve performance?

[royotech]

Ollama in Orange Pi 5 plus in NPU chip, it's possible?

[sidneyking]

@kevinmcaleer, How would you run this on a proxmox server? would you use a lxc or full vm running ubuntu? Thank you

[zerotsu]

Can I run ollama on a powerful system and serve the webui on another system?

[GeistschroffReikoku]

is there a way to update the ollama it builds? i follow the video and is working fine but gemma just come out and it works only with the latest ollama 1.26... and when i followed the video it installed 1.23... what do i have to erase or what do i do?