Low Power Cluster - Small, Efficient, BUT Powerful!

I built a lower power, efficient, and near silent server cluster! Although this cluster is small and efficient, it's still powerful enough to run a high availability Kubernetes cluster with many services running in High Availability mode! There are so many options with running a small cluster like this, the possibilities are endless!

A HUGE thanks to Datree for sponsoring this video!
Combat misconfigurations. Empower engineers.

Disclosures:

- I purchased everything with my own money

See the kit here:

Want to set these up with Proxmox? No problem!

Looking to se up a cluster yourself? Check out the video and the docs!

Thanks (Red Shirt) Jeff Geerling for the clip!

(Affiliate links may be included in this description. I may receive a small commission at no cost to you.)

00:00 - How to Build a Low Power Cluster
00:18 - Hardware Specs
01:20 - Rack Mounting Intel NUCs
02:17 - Remote Control & Power Control
02:58 - Provisioning the Cluster
03:28 - Why Kubernetes?
03:39 - Automated Install of Kubernetes
04:20 - Kubernetes Policy Management (Today's Sponsor, Datree)
05:46 - Installing a High Availability Kubernetes Cluster
06:51 - Testing High Availability with NGINX
08:57 - Introducing Chaos
10:04 - What Can We Do with a Cluster?
10:35 - How Much Power Does This Cluster Used?
11:20 - What Do I think of a Low Power but Powerful Cluster?

Thank you for watching!

[JeffGeerling]

I like the clean rack mount most of all. I just finished a project with two NUCs and agree on their handy utility! (And I'm glad Red Shirt Jeff has been pestering you lately... finally I can get some work done!)

[twon]

Crazy you did this! Exactly what i did about 8 months ago. I had a dell R720, readynas, 2 switchs (with POE) AND BOY WAS MY POWER METER SPINNING. Got to the point where I didnt see the need to have such a power hungry server in the house and moved it to a data center where the space and bandwith was far cheaper than my costs to have it at home. But having this 3 node cluster was perfect for my needs and i could still do some low end development on the home lab if need be. Great video man looking forward to the next one!

[RyanJones]

My cluster is 3 NUCs for workers and 3 hp mini PCs for the control plane. Been using this setup for a while. Very stable

[TerraMagnus]

Literally how Chick-Fil-A stores run their edge clusters in-store. K8s on NUCs.

[deckardstp]

Thanks for the video. I run a DN2820FYK Celeron now for more than 9 years. It was my entrypoint to Linux on a x64 platform. Evaluated numerous stuff on it. Learned about Docker and so on. NUCs are super reliable from my point of view and worth every penny over time.

[TweakMDS]

Love this video, it's exactly into my hobby sphere.
A NUC is great because of a small footprint. If you have a rack (and it's somewhere where a bit more noise doesn't matter), you'll get a lot more back for your buck with an actual server or 3. Typically, the dell/lenovo/hp machines are quite affordable, especially if you buy used/refurbished. I'm running 2 lenovo's with a 6-core I5, 64GB of RAM and 2 500GB SSD's each.

For a homelab, anything "tiny mini micro" is great, and suits a K8s cluster quite well, but for me the real strength of a homelab comes from building experience in setup + teardown.
From that perspective, getting 3 machines to form a Proxmox cluster would make a lot more sense. On this setup with virtualization, you could easily fit 4 or 5 clusters with a controlplane and 3 nodes per k8s cluster, and have a bit more room to play with.

In any case - for anyone wanting to build more hosting / devops experience, I highly recommend finding a small machine, cram as much RAM into it as you can, and start setting stuff up.
Right now I'm working on an "end to end terraform solution" where I setup 4 VM's on proxmox with cloud init on an ubuntu template, initiate k0s as a distro with 1 controlplane and 3 nodes, and then also automate all of the final details, like storage using nfs to my nas, metallb and ingress.

Challenges always remain, but it's fun to perfect (and then toss out and start over with something else).

[pgm]

Relating to the remote KVM part: there are NUCs with full vPro functionality with remote KVM over VNC. They also got a web interface to power cycle etc. - and there are also command line tools to start them remotely

[lgic]

I'm so glad to see you getting into low(er) power boxes. Excited to watch this!

[rankenfile]

The Mk1 hardware looks pretty nice. Glad to see you having lots of fun in the lab!

[testdasi]

Tim's K3s Ansible playbook really simplifies setting up an HA Kubernetes cluster. On Ubuntu, it might be faster to set up microk8s but after trying both, I'm like k3s slightly more because of lower idle CPU usage.
And even with stateful apps (which is... uhm... pretty much everything?), you can still achieve almost-HA with a single replica in a HA cluster + replicated storage (e.g. OpenEBS Jiva or GlusterFS). Just need to set tolerations to something like 10 seconds. So if the node (that carries the pod) is down, 10 seconds later, a new pod will automatically spins up on another online node. That's HA enough for me.

[fatalsg]

This is really good! I was just thinking of deploying a full functioning project locally because I don't have money to maintain cloud services fees while still in development! I think you just saved me my friend.

[RaidOwl]

Drag race? Your NUC k3s cluster vs mine 💪🏼

[henrysowell]

Tim! Great video. I’m going to have to explore this. I’ve been thinking of selling my 8 x Raspi 4B (8gb) cluster and doing something like this. By the way, for your last Cloud Init video, I put together a ansible playbook based on the video to automate the process. It was awesome, used it to make templates with a single click across five proxmox nodes. Thank you for all you’re doing! 👏

[SneakyJoeRu]

I'm not sure if I should be proud or afraid that I start to understand what you're talking about and envy your setup after joining devops 2 years ago...

[dosdaysareover]

Cool to see I’m not the only one clustering mini PC’s. 👍 those small form factor devices are quite powerful lately.

[TonyGonzales]

I either ran into you or your doppelganger just now in downtown Houston. Couldn't put my finger on who I was reminded of before the convo ended but wanted to say thanks after my memory was sufficiently jogged. Have appreciated your work for some time now. ✌️

[supernerd]

The unexpected Geerling cameo had me laughing out loud!

[CohanRobinson]

I'm super glad I found your channel. This is the pure kind of nerdery I need in my life. Love it.

[seethruhead]

I am thinking to run 3 R86S but as a proxmox cluster
the 2 10gig interfaces are really interesting, you can use couple of ways
1x for client, 1x for ceph network
2x for ceph ring network

and on top of proxmox cluster you can of course run k8s nodes as well so it seems best of both worlds

[DavidAshwell]

Those Thunderbolt ports will directly do 10gig+ Ethernet host to host (iPerf around 14). 3 hosts, 2 ports each so you can daisy chain. Takes some configuration, but it's a cheap way to get fast networking.