Modern Servers Are Guzzling Power - Here is Why

Anyone that has looked at recent servers has seen power consumption go way up (along with performance.) We go into why power consumption is increasing, and some common pitfalls of sizing power in data centers. We also go into some of the data center gear that goes behind the servers all the way to generating electricity through solar arrays and hydrogen fuel cells.

Thanks to Supermicro and Vertiv for sponsoring this video. Supermicro has let us review its servers for years. Vertiv gave us access to two of its facilities in Ohio. Thanks to the STH YouTube Members for helping with the travel to Ohio.

----------------------------------------------------------------------
Become a STH YT Member and Support Us
----------------------------------------------------------------------

----------------------------------------------------------------------
Where to Find The Unit We Purchased
Note we may earn a small commission if you use these links to purchase a product through them.
----------------------------------------------------------------------

----------------------------------------------------------------------
Where to Find STH
----------------------------------------------------------------------

----------------------------------------------------------------------
Other STH Content Mentioned in this Video
----------------------------------------------------------------------

----------------------------------------------------------------------
Timestamps
----------------------------------------------------------------------
00:00 Introduction
02:13 Server Power
09:57 Data Center Power

[BrunoTorrente]

When I worked in a data center, a redundant 25kW rack was excessive, reserved for clients who requested a high-density rack as a project requirement.
Today 50kW is "normal", 120kW is the new limit, and there are people talking about 200kW ultra density.
Working in a data center is increasingly dangerous.

[userbosco]

I've been building data centers for nearly 20 years - it blows my mind to see what the enterprise and hyperscale sites are consuming. I also live in VA, so the vast majoruty of HSDC's are in my back yard. The energy footprint of these are beyond most peoples comprehension.

[james.telfer]

All the CEOs happy to blow shareholder funds on shiny shiny because FOMO and they have no clue what 'AI' really means.
Might make for some real bargain GPUs when they go bust.

[rgalik]

I'm sorry for being "that guy" but I am a bit appalled at the rhetoric around hydrogen as fuel. Hydrogen does not grow on trees. It's not a "power source". It is a method of storing and transporting energy. It's only as clean as the technology used to generate and transport it. By itself, it's like a battery.

[BLASTIC]

well... so much for all those net-zero targets. Microsoft is using 30% more power than when they said they would be net 0 by 2023 lmao. from what I gather, their explanation is: "... yea.... but AI!"

[carlschumacher]

Emmett Brown : "Marty, I'm sorry, but the only power source capable of generating 1.21 gigawatts of electricity is a bolt of lightning."

[lemagreengreen]

Iceland might as well just turn itself into a giant country-sized data centre at this point.

[james.telfer]

Funny story (now): project to move servers to a DC, told they were supplied by 2 different power stations on opposite sides of the city, 2 UPS systems on site, etc, etc.
One Monday morning, servers all offline - power cut from 1 power station caused that UPS to trip, then the other overloaded. And why weren't we told? The control room was fed from the same UPS so also went ☠️
Can you say SPOF??? 🤦🤦

[squelchedotter]

It's incredible how fast AI images went from being impressive to just looking cheap and scammy by association

[post-leftluddite]

Its called the "Rebound Effect" and its the observed phenomenon that technological improvements in effeciency NEVER result in a net decrease in power consumption. Its similar to the fact that despite being the most productive workers in human history, current workers are working MORE than ever.

A classic example from this perspective is a driver who substitutes a vehicle with a fuel-efficient version, only to reap the benefits of its lower operating expenses to commute longer and more frequently and thus resulting in a NET INCREASE in energy consumption.

It's a product of Capitalism and the paradigm of perpetual expansion. But also demonstrates how technology cannot solve the environmental and reaource crisis.

[afitzrobert]

Power and cooling: many years ago a customer selling HPC systems told me that a university had expressed their budget as “600 kW - you can split that however you need between power and cooling”

[aflury]

Damn, my whole house only consumes a couple millicybertrucks.

[FragEightyfive]

There's a reason why so many data centers are being proposed next to power plants with exclusive power deals.

[jimsubtle]

I loved this video, so cool to see the various data center solutions. Thanks !

[timchambers]

The energy per GFLOP or GIOP has greatly reduced. We just found bigger problems, more complex solutions & greater consolidation. We once had 80 cores using 40 rack units doing 128b per cycle per core, now its 1 or more 512b per cycle per core & 256+ cores in 1RU.
Network power has increased but offload has reduced CPU usage (reduces power usage) & offering higher throughput means much better per Gb efficiency.
It would be interesting to see how power per Giga FLop/INTop has changed over time vs absolute performance per rack.

[ZKIUS]

servers used to be stupidly power efficient
but when "AI" hit they all went crazy, no matter the investment the latest shiny thing brings crazy returns so shy not pump a whole power plant worth of energy through your server just to generate few images?

[adhdengineer]

At 20 cents per kWh that's $210k per year. Just to let the thing. That's mental.

[raylopez]

Nice video. More POWER to you, ServeTheHome.

[pabss]

Rather than cybertruck, compare it to a house, more relatable ;)

[Jerrec]

So true. Our AI cluster's heat is beeing used to heat the building where the Datacenter is located. No joke. It works well and the heat is at least used in Winter. And in Summer the heat isnt "destroyed" from a air conditioner, it is transfered into earth where it is cooled naturally.