Building a Low Energy Virtualization Server for Your Office/Homelab with Proxmox

Following up on my low-energy FreeNAS build, here's another server build - this time for virtualization. In this video, you'll see the process of building a low-energy Proxmox server, and just like the last build - this one uses only around 55 watts!

🙌 Support me on Patreon and get early access to new content!

🛒 Affiliate store for Linux compatible hardware/accessories (commission earned):

💻 Check out the Tiny Pilot KVM for your Homelab (commission earned):

🐦 Follow me on Twitter!

📖 Check out jay's latest book, Mastering Ubuntu Server 3rd Edition. Available now!

Purchase the parts used for this build:

Parts list:

SuperMicro Motherboard product page:

Old vs New CPU Comparison:

Wiki article for this video:

👨 More about me:

💽 How to create a bootable flash drive for installing Linux:

🐧 Which distro do I use?

🔐 How to better secure OpenSSH:

[smashogre]

I'm new to this channel and was wondering... anyone else find this kinda, well, soothing? The steady, calm delivery works! Great video, really enjoyed the content.

[bytswap]

I built a virtualization server using Proxmox a few years back using the same motherboard and a similar 2u case a few years back. Added an old quadro w/ PCIE passthrough for video transcoding and the thing works like a dream. It's been awesome a first "real" home server, running absolutely everything I need (with exception of a separate storage server) and has had pretty much zero downtime since building.

The one thing I will say for anyone looking to use this board or similar (if you can find a deal on a several year old board), that passive heatsink really doesn't cut it if you aren't in a 1u case with loud high static pressure fans. I ended up designing a 3d printed fan mount that just pressure fits on the top of the heatsink, with a 20x40mm noctua fan blowing down on the heatsink. I haven't seen the CPU exceed 50c since installing (used to hit high 90s and throttle), and everything is still nice and quiet.

[bobkoss]

All the stuff that you did off camera - is exactly where I'm stuck.

[davidg]

Hey Jay, the proxmox team added a way to apply network changes without needing a reboot. Click apply configuration button. However, I should note that it will ask you to install a package, which will need a reboot (but only one time after installing it). Now you can make future network changes without ever needing a reboot.

[ianrobertson]

Thanks for the video. This is bit expensive. I have used Lenovo Tiny 93P got from ebay for US$125 and changed to SSD256BM and 16GB Ram. This set up cost me less than US200. It is core i5 Gen 2 cores 4 threads and work well for over a year running a) Debian-> Docker-> Home assistant to control 200+ devices running heaps of automations b) VM to run Blue Iris on Win7 with 10 cameras + Unifi controller for my unif network. I do have a HP Gen8 server with SAS drives (4)+ 8 Code (2 process) +24GB system configred proxmox but not running it as it uses over 250w and my Tiny PC uses less than 40w (my network rack use only 75w with Tiny 93P+ Synology 213 NAS+ Unif USG+ 2 PoE AC-Pros+ Modem+ Netgear Switch+ 3 Hubs for energy monitoring and home automatons). I am sure your system use more than 50w. So there is cheaper ways to get thing going and that does not need lot of power.

[stephanweinberger]

@9:15 the IO-shield is not so much about airflow (notice that in your case you have a large vent right above it) but for EMI shielding.

Also, if power consumption is of concern I'd rather use a PSU with Gold or Platinum rating. Especially under partial load those are much more efficient. Alternatively: dont oversize the PSU that much. If the system is only going to draw ~50W you're better off with a PSU in the 150-200W range, as it will run closer to its most efficient operating point (usually around 50-60% load).

@30:45 the updates you get without key are those from the base Debian system

[lsatenstein]

Hi NewYorker. How are you doing? I do hope the Corona virus has skipped over your household and that of your close family.
I must say that I follow your presentations as a matter of course. I am a retired old IT guy, with limited need. And that need is to enjoy watching talented people like yourself.
Regards from Montreal, Canada

[MichaRutkowskiEngineering]

I also did 55W 4 years ago, but i went for cheaper option. I am using ASROCK j1900 motherboard with 16GB of ram, the TDP of the CPU is 10W. To that i added 1tb HDD for proxmox and 3x 2tb for ZFS NAS, currently on openmediavault with sata passthrough. I have added a second NIC to access two networks because i did not wanted to go VLAN, and recently a 5th HDD, it's a 10tb drive for cctv and computer backups. I am waiting for mini pcie SATA card to attach it, currently it is on USB adapter, without the 10TB i measured it to be also 55W with 4 HDD drives. I am planing to replace it as soon as i will purchase a solar power installation with my HP z620 workstation with Intel Xeon 12 thread CPU and 16gb of ram.
The cpu is 10 times weaker than yours but it was fine even for plex transcoding from 1080p. And i also run Proxmox ;).
For network i went Mikrotik, with HEX S as a main router (because of it's 450mbit/s ipsec performance) and two RB951 series for WiFi and i am waiting for hAP ac2 for my main wifi since those two are meant to serve all the smart things in my house.

[Shadepariah]

I would add a few case fans with that board so the CPU doesn't overheat. Those fanless boards are designed for 1U cases with push pwm airflow

[ridingforpresident-jelzins]

Did nearly the same setup like you with AMD Ryzen 2000 CPUs, not much power consumption and samsung NVMe's. You should always buy the pro versions when there is a lot read/write activities on it. It is so powerful that you are able to run multiple Windows Servers (+ Domaincontrollers) without any issues. Great for smart comercials.

[barbarella]

Wow. I never knew that you could mount ISO images in the Supermicro IPMI. Thank you!!

[cls]

I've just built something similar some months ago: 5 x 512GB M.2 SATA SSD (PCIe-Board), 4 x 4TB WD Red, Supermicro
X11SCL-IF with IPMI, Core i3 9100F (4 Cores), 2 x 16GB ECC and be quiet! Pure Power 11 400W PSU within a Fractal Node 304 case. My system idles at *22 Watts* and is pretty quiet. Services running as LXCs on Proxmox: haproxy, mqtt, influxdb, node red, grafana, nextcloud and gitea.
The systems will get two additional backup drives attached to a M.2 PCIe 2 x SATA controller soon (waiting for the M.2 controller board to arrive). The backup drives are only powered up when a backup is due to keep the power consumption low. The power consumption is even so low, that it doesn't draw power at all when the CPU throttles down thanks to the caps within the power supply. This seems to confuse my CyberPower UPS :) (I have the suspicion that their PowerPanel software has a division by zero bug). Currently I could bridge power outages of about 4+ hours.
I built this system because my previous HP ML330 G6 Xeon with 5 spinning drives drew 105 Watts (not that bad actually for such an old and huge system) and was awfully loud. It's great that it's finally possible to build such real low power systems - even with nice headroom (the above i3 has some punch when needed - not as much as your system though). The long UPS times are a nice side effect to the lower energy consumption. The nice thing about the motherboard is its built in IPMI/BMC as yours seems to have as well.

[williamp]

Nice server.

You can do this on a budget if you go with used server parts from ebay. Or to do it cheaper and really low power, do it on a Raspberry Pi 4. The model with 4GB of RAM can host six or more containers. Under $100 and 5 watts. The Self-hosted podcast is a great source of info.

And Rapberry Pi are the only way to build a cluster without going broke.

[dariopetrusic]

Please note that usually when a server part (CPU, HBA, NIC...) comes with a passive heatsink it doesn't mean that this is enough but requires some amount of airflow to be properly cooled.
I strongly suggest you to check the CPU temperature because if in idle it should be fine, I can almost guarantee that under load at 55W and without any airflow it's thermally throttling like crazy!

[MrNoBSgiven]

Can we expect a follow up on how this server has been doing since installation?

[JoePlomo]

Whoever may need this build guide; the part they need is exactly what happened @ 16:03 off-camera. The people that know how to do that part don’t need the rest of this video, so I’m not entirely sure who this video is for

[MatthewYakel]

Wow this is fantastic I’m in the planning phase of upgrading a clients existing HP DL360 G8 servers. Now I’m thinking I may build them some. Also planning on moving them from ESXi to Proxmox. Thanks for sharing everything.

[Xzb]

If im not wrong, the reason you don't have any output when joining the cluster is because the http certificate is changed on the server who join the cluster (it is replaced by existing certificate of cluster). Anyway good video, i order this motherboard today

[kbhasi]

I forgot low power AMD EPYC SoCs existed!
Also, I can't remember when in the video you mentioned that ZFS isn't for everyone or something along those lines, but that reminds me of why I used BTRFS. It's hard for me to believe that I initially decided on hardware RAID when building my first custom NAS, but used BTRFS RAID instead, and when the secondary SATA controller on its mainboard died, I was still able to recover my data on another mainboard thanks to myself choosing to go with software RAID instead of doing it in hardware. That does mean I replaced the mainboard in my NAS, but because the rest of the previous mainboard still works, I was able to repurpose it in a different case with different components as a secondary always-on PC, that just like my NAS, runs regular Debian, but unlike the NAS, has a limited set of components in order to free up more system resources since that second system would just run one specific service, whereas the NAS runs multiple services and even a full Trinity session over a VNC connection.

[doman]

For arround 200-300$ you can buy Fujitsu PRIMERGY TX120 S3P. Small Form Factor, Xeon cpu (like ex. E3-1265L), ECC Ram, 4x2, 5'disks bay, 2xGbps nic, remote access. I think its even better deal than HP Microservers.