Bare Metal vs Virtual Machines for Linux?

In this episode of the CyberGizmo we explore Bare metal vs Virtual Machine for Linux, which is best? I wanted to do this video based on comments I have gotten from my viewers about using Virtual Machines to demonstrate the capabilities of a particular Linux distribution (distro). I had not preconceptions about which method Bare Metal (which is a direct install on hardware of linux) vs installing the Linux distro on a virtual machine. I used a number of benchmarks to try and find out if there were any significant differences in performance on either platform, and I think the results may surprise you.

Follow me:
Twitter @djware55

Werq by Kevin MacLeod

Industrial Cinematic by Kevin MacLeod

Music Used in this video
Licensed under Creative Commons: By Attribution 3.0 License

[agevlakh]

Hi! Great performance comparison with charts and work too. Thanks DJ!

[Remigrator]

I had no idea what Bare Metal meant. I thought it was music or so 😅

[michaelhill]

This is a great video sir. The 8 core sweet spot was illuminating.

[manit]

if you're a developer or network engineer then having virtual environment is a must for learning.

[prashanthb]

10:30 I did a cluster computing benchmark and this chart shows what was happening.
I wrote a finding prime number script using Python multiprocessing pool map function. The results surprised me.
Cluster of 3 nodes of Ryzen 7 1700 @3GHz with hyperthreading finished the task in 791 seconds.
Cluster of 5 nodes of Athlon3000g @3.5GHz with hyperthreading finished the same task in 715 seconds.
The important point here is the first cluster took 24 cores to achieve this while the second cluster took only 10 cores. Far more "computational efficiency" with reduced resource contention !!!

[KingZero]

great video... very thoughtful and informative... thanks

[RichardSwift]

Thanks! Very nice review. I would like to hear more about the issues you ran into with Virtualbox, just curious.

[lenwhatever]

VMs are really good for testing. They make very good use of a testers time and resources. They give the ability to continue talking to other parts of their work group via irc (or whatever), file bug reports even when the the vm is not in a state where they can be files directly and the ability in many cases to do screen shots when the vm is frozen or in a mode where it can't do so (running grub?).
I see the word "faster" and "speed" a lot. When performance is mentioned, throughput speed seems always what is mentioned. The one place vms seems to be less useful is with real time or low latency loads. This goes along with hyperthreading as well. When testing with a vm audio latency tends to have to be set around 20ms or more. This is fine for most recording and playback but is a fail for use as a softsynth or effects rack where 5ms latency is expected. More modern kernels seem to do this better (maybe hardware?). However, my older 4 core 4 thread i5 seems to handle low latency better than "faster" machines. It is true that audio DSP is not really a high CPU process so much as a rigid schedule process.
Over all a good presentation.

[AdmVrln]

Thanks DJ - another great video.

I just want to add a couple of thoughts.
In many cases if you have a piece of hardware dependence - gaming and a GPU, while you can go through work to setup, if you are unconcerned by scale (ie you just want one box) then bare metal is the way to go. In other cases like a capture device, and with USB devices - where there is a hardware element - the direct bare metal is the simpler solution.

In terms of the CPU and SMT, if we take a look at the new release stuff, they are messing with cache, schedulers, CCXs (in the case of AMD), and development follows. A new 'cache' coherency issue is if on AMD data is crossing the CCXs. But this is only a late era issue that existed before in a different form. Previously there were things like NUMA nodes that would be a bottle neck.

Where I think things really shine for VMs and production as DJ says - is in better usage of hardware and a return of investment, as is in the ability to move a server or workload. The ability to backup and snapshot. The ability - if you buy into or arrange the backup infrastructure - to backup, recover, restore in a way that means bare metal tends to be a harder road. This would really be the place I would add in on as a real value add for shifting from bare metal - - assuming the workload is the fit - to VMs. There is a downside - which I will go to next..

OS support and vendor support. In many cases you can do the above, but vendors from MS to Nvidia to VMware to backup vendors - don't tend to let you walk the road for free. There are horrible gotchas in areas like licensing and in support where it can crank the costs up. A classic is starting down the path to Virt cloud gaming and then hit Nvidia licensing. In addition, as you scale (this does apply to bare metal, but I'll go with VM structure for this point) - the upgrade of things at scale, catering for the gotchas of the Host hardware, the Hosting VM software, and all the guests can create very difficult upgrade paths - not for the faint hearted. Sometimes, IT is best in simplified form, over a super complex system that in theory offers more, but has a high price tag in other areas.

I had one more point I was going to say but its slipped my mind. Bugger. Ok.. I'll end with this, for most people who have many workloads (IT people, devs, sec folks, tinkerers) - a VM structure is nicer to 'play with' in terms of being able to spin up a machine/wipe test machines, and to create things in what I will loosely name IT minecraft. Its fun to do, so if you never did it, jump in - try it out.

:)

[BrucesWorldofStuff]

Yes that was my first thought in the home front. "It all depends on what"...
I see VM's as a great tool to test and look at things. Doing Metal is just not going to work
as every one has so many different hardwares and not all the same hardware works the same either so most but not all the same.
I don't game. Altho I do like playing with Distros and a VM will tell you 99% of what you want to know. With that said, if you know your hardware as you should! You will know certainly how it will work on your system... Testing is the way to know. If you test your system enough both ways you will know how 99% of the software will act... :-)
Thanks for the video DJ!
LLAP

[DrZingo_]

There is also Amdahl's Law that still apply. If some portion of a program has to be single threaded, there will be a roof of how much performance boost you can get with more cores (on that particular program). I guess virtualization will be more important the more cores we get to utilize it better. Maybe you mentioned it already, (I was doing other stuff while listening, I can only handle one thread ;) ).

[AngryPacman]

I'm using VM instead of physical hardware at work for years. Virtual GPU and storage speeds are bottleneck in apps where it matters. Everything else works just fine. CPU feels the same as on physical hardware, benchmark scores are similar. I like to work in VM with all these snapshots, it's very easy to fix it without reinstall. It's also very portable, can be easily moved to new hardware by copying 60GB file and takes 20-30 minutes instead of wasting a day for reinstall OS and customize it. And I don't care about which is host OS.

[Remigrator]

On the subject: I never buy virtual machines because I have no idea what's behind them if they are oversold and also they are very expensive. I can get a Celoron 2 GHz with 8 GB of RAM and 1 TB disk for as low as €5 with a gigabit connection, I mean come on, right? I actually follow the one application, one server rule, because I can. :D

[aceflamez]

Very cool video, I think your shutter speed is a bit too slow on the face cam (if that's intended my bad lol)

[oscarpradilla]

Great video !! Is virtualization type-1 hyper visor run same or better in Linux vs Windows (Hyper-V) ?

[keratishvili]

is there big difference between esxi and kvm?