LXCs vs VMs - What Was My Rationale?

So in my last video, we took a look at what I've got running in my home lab back here, and we briefly discussed the hardware.

Then we talked about the two Proxmox servers I've got for production and the one I've got for testing and development. I don't develop, but it's for testing and tutorials, that sort of thing. I guess that's the better way to word that.

In the comment section of the last video I released about all of this stuff, a couple of people asked what was my rationale in deploying LXCs or Linux containers for each of my individual services rather than throwing up a couple of VMs, installing Docker, and then putting all of my services in those couple of VMs.

And I think that's a great question.

We discuss my rationale of my setup in this video.

Blog post/transcript:

/=========================================/

Additional info on LXCs and VMs:

✅ LXC Pros:
- Lightweight: LXCs share the host system’s kernel, making them much faster to start and use fewer resources than VMs.

✅ LXC Cons:
- Limited Isolation: They provide some isolation, but since they share the kernel, a security issue in one container could potentially impact others.
-Linux-only: LXCs can only run Linux distributions that are compatible with the host’s kernel.

✅ Use for VMs:
- Running applications that require a specific operating system not compatible with the host.
- Situations where a high degree of security and isolation is critical.
- Running software that requires direct access to hardware resources.

/=========================================/

Get early, ad-free access to new content by becoming a channel member, or a Patron!

All My Social Links:

Join Discord!

[KEBSM]

Good points for using LXC’s over VM’s! I’ve seen multiple ways to passthrough hardware to LXC’s. I’d love to see a definitive video on hardware passthrough to LXC’s! Especially for unprivileged LXC’s

[donglobal]

I like the new yellow backgrounds you are using in your last two videos, it really does make them stand out from the crowd.

[bps]

I am also a total fan of LXC. Fast, small and uses little hard disk space.
The reason why you install Docker in a VM (should not have to) is the security of the Horst server.
We visited the Proxmox developers.
When I asked why not Docker in LXC?
We were shown how. With one LXC (docker ) you can bring down the entire Proxmox Horst.
Nothing happened in a VM.
VM containers use more resources, but prevent a total failure of the server.


The backup strategy for VMs is snapshots + a weekly full backup (stop mode) in individual operation.
Proxmox Backup Server & Cluster Nodes is a separate topic.
It was a very impressive seminar with the developer.
I apologize for my English, writing in a foreign language on a cell phone is a disaster.
Greetings from Germany

[Rockwolf]

Whilst I completely agree with your rationale Dave and do things the same way (and get flamed on Reddit for it), you could restore a single container on a VM. If you go into the interface of PBS and click on the backup of that machine you want to use, you can actually access to the folder structure of that VM and can download folders directly to your desktop, where you could just scp it back in to your VM.

I don’t myself, as I prefer the simplicity of LXC containers and splitting all my services out.

[julianmoorhouse]

Great video and I completely agree with you. I have had one of the containers within a proxmox VM get corrupted by an update, and had to revert to an earlier backup of the whole VM to recover, but that meant I was restoring to an older version of everything else in that VM which was annoying as well. So now I strategically choose what can be in the same VM and what I split across multiple VMs.

[PoeLemic]

How you explained it, is very helpful. I am trying to build my own home lab, so that's why I love your channel. You help make it more understandable and show us that we can do it too.

[cease]

I had this exact question while watching the video. I like, and would almost say that I prefer, LXC containers to a VM running multiple Docker containers but think there's a balance to be struck between the two. I think I saw/read from another YouTuber I follow (can't remember who it was at the moment) that hosted LXC's for internal-only applications and did a VM with Docker for anything that was exposed to the internet, which probably makes sense from a threat surface standpoint. Great to hear your rationale!

[donaldmull]

So very happy that you are back to posting videos. Missed ya man!

[evanbaker]

Makes complete sense. I may be wrong, but an LXC is essentially a container just running in proxmox natively instead of something like portainer on a VM. Least that's how I understand them

[PotatoStudioTV]

I had the same dilemma with my home server.

The reason I use VMs on some mission critical services is due to HA and Backups.

Since I use my NAS's SSD pool as storage via NFS, if 1 node shuts down, the HA manager can migrate without any issues. If I use local-lvm and my node powers off for any reason, HA cannot migrate since the storage is on the offline PVE node. If I use LXC with NFS as storage, backups will fail.

So It's mix and match for me until I upgrade my hardware and move to CEPH, this is my setup.

[DanielBrown]

Loved the rational, especially the snapshot restore. I guess my two lazy points are what would keep from doing it.

1. Manually updating by logging in and pulling images, etc.
2. Having to think about resources for each lxc.

My favorite thing about docker in a VM is I don't have to care if one container uses more resources than another. Only have to monitor the overall VM system usage.

Either way though, going to move pi hole to an lxc for sure!

[ronm]

Makes sense to me. Thanks for sharing.

[mrpopsko]

hi, the way you suggested doing things in terms of easy backup also exist for a monolithic docker host too. most of the ways people do it (like me) are to make use of BTRFS snapshotting and also to make use of docker compose bindmounts. so you'd create a directory like /appdata/utility_name and then you'd have timeshift or snapper or btrfs-progs or whatever similar utility then snapshot that /appdata/ at regular intervals (either on another separate disk or as part of a mirror) and should something bad happen to any specific docker container then its as simple as just going into that specific subvolume (snapshot) for the files and yoinking them out

[lessonswithricardo]

Good thing about LXC you can run docker inside of it as well. Outside of running desktops, security onion, and a few rdp managers, VM not really needed.

[ManelRodero]

When I started with Proxmox, I used LXCs based on TurnKey Core because they were supposed to be very lightweight.

If you use Proxmox Helper Scripts, many of the LXCs are Debian 12.

I understand that the Proxmox kernel supports both of these options, right?

What do you use? Do you use templates and install applications afterwards? Do you use Proxmox Helper Scripts?

Thanks.

[basdfgwe]

If you can containerise it then it should be containerised, the real question I keep going back and forth between docker and lxc. Half my services are on LXC and the other half are on docker.

[miclgael]

good to see you back :)

also just realised i've been spruiking your vids and i wasn't even subbed 🤦‍♀

[kevinhughes]

Great point never thought of it like that. Can you do a video on how you created your lxc template and how you map external storage to it please?

[PoeLemic]

Also, probably, dumb question. But could you (maybe?) make a video explaining what a LXC container is, more contrast it against a Docker container. I believe that I understand Portman (sorry, edit: Portainer) and how you run Docker containers on it. But, I didn't know that you could run a LXC container (I guess that's what it is) straight on ProxMox. So, I am assuming that LXC is just another container model, probably similar to Docker, by probably a different vendor. Or, that'd be my guess. But, be nice to see how to put a LXC container together and what software you use. Or, maybe, I guess that people put LXC's together -- just like people put Docker containers together. Okay, that's just a topic that I am searching now and trying to understand, and that's why I came back to your channel to see if you expanded upon it in this video. [Thanks, @DB].

[tfraiseo]

I go for an LXC out of the gate because my lab doesn't have a lot of horsepower and I like how lightweight they are. I tried getting AWX running in an LXC and couldn't make it work. It's a hobby for me and getting frustrated makes me stop before pulling my hair out so i don't completely give up on it from burnout. I'd be curious to see if anyone has won that battle and what their process was.