NVIDIA Launches Biggest GPU Yet: Hopper H100 & DGX H100 Systems

NVIDIA's GTC 2022 event announced the company's H100 (Hopper) GPU and gave updates on its new CPU, Grace, for high-end applications. This precedes the expected Ada GPU launch.

We were able to get clarification on the bits/bytes conflict between the NVIDIA presentation statement and the slide presented: They meant to say Terabytes, not Terabits.

NVIDIA just announced its new high-end scientific computer and datacenter parts for GTC 2022. The GTC (GPU Technology Conference) event tends to bear more news for professional and enterprise use than for consumer, although there has been consumer news in the past. The current rumors indicate an RTX 40-series launch (likely an RTX 4080) towards the end of the year, but for today, we're talking about the new Hopper GPUs. NVIDIA's current GPU architecture for consumer (and datacenter) is Ampere (RTX 30-series), its new datacenter and scientific architecture is Hopper (e.g. H100), and the expected future RTX 40 architecture will be Ada or Lovelace -- currently not formally known.

Hopper, then, diverges in naming and architectural layout in at least some ways from the future consumer part. We'll also talk about the Grace CPUs coming up, also from NVIDIA, in combination with work from Arm. As a reminder, NVIDIA was attempting to purchase Arm until recently.

TIMESTAMPS

00:00 - New NVIDIA GPU & CPU
01:35 - NVIDIA Hopper GPUs & Specs
07:17 - Combining H100 GPUs & Super Computers
11:00 - Merging GPUs, CPUs, and NICs
12:39 - AI & Deep Learning Talk

** Please like, comment, and subscribe for more! **

Links to Amazon and Newegg are typically monetized on our channel (affiliate links) and may return a commission of sales to us from the retailer. This is unrelated to the product manufacturer. Any advertisements or sponsorships are disclosed within the video ("this video is brought to you by") and above the fold in the description. We do not ever produce paid content or "sponsored content" (meaning that the content is our idea and is not funded externally aside from whatever ad placement is in the beginning) and we do not ever charge manufacturers for coverage.

Follow us in these locations for more gaming and hardware updates:

Host, Writing: Steve Burke
Video: Keegan Gallick

[GamersNexus]

We were able to get clarification on the bits/bytes conflict between the NVIDIA presentation statement and the slide presented: They meant to say Terabytes, not Terabits. Re-uploaded! That should fix the bit-rate problems for the regions that had it. It was stuck at 480p/360p for a lot of people. We see that sometimes and a re-upload usually fixes it. Normally we catch it first, but didn't this time. Sorry about that! Changed the thumbnail and title a little bit just because the algorithm is a mystical behemoth that no one understands, so maybe this will satisfy our daily quota of algorithm feeding.

[RocketSlug]

I was very tickled by "Who cares" RPM and "We're only limited by OSHA" decibels. Props to whoever came up with those

[Summer_Lilac]

"When the fan rpm goes to who cares rpm and the noise levels go up to we are only limited by OSHA decibels." This is the wit I come for and I love it.

[danielchong]

Cat render performance is a very important metric going forward. We will need to see more of them in future system benchmarks.

[nimbulan]

4.9 Tb/s would be equivalent bandwidth to a 3070 Ti. It's definitely 4.9 TB/s. Also Jensen said 40 Tb/s when he talked about it, not 4.9.

[CptJoeCR]

I would like to point out that the "We're only limited by OSHA decibels" is funny...but not actually a joke! I work in an experimental remote lab and we were given $200 noise canceling headphones and given a strict warning to never go near the servers without hearing protection, 102db when working on the servers and its only getting worse as we fill out the lab.

[enthusiasticgamer]

I love how GN is simultaneously the most serious and comical tech channel. I always get a chuckle out of every video.

[WyndStryke]

The *actual* reason why cats need FP64 instead of FP8 is that they have 8 extra lives.

[fgfanta]

That's what my cat does when he disappears behind the PC!

[BROTRRer]

Really hope this new GPU will have a higher IPC (Instructions Per Cat)

[hondajacka]

The breadth of futuristic tech revealed during Nvidia’s GTC is mind-blogging.

[johnbeeck]

Fun, informative, and passionate! Great job - love this length and pace!

[addisonmartin]

For those who don't know, Hopper is named after Admiral Grace Hopper. She is an absolute legend among early programming history! Fundamental work on compilers.

[middleclasspoor]

I swear the more I see your show the more I like it! Steve, your presentations are amazing. As to how you can convey so much information so smoothly is unparalleled!

[chrisbaker]

"who cares rpm".
Yep that's my server room.

"Only limited by osha".
I have a sign on the door "Do not enter without hearing protection".
I regularly ignore it.
Might just explain my hearing problems.

But seriously folks, if you're going to work with servers, invest in, good, hearing protection.
And use it every time you enter.

[YouHaventSeenMeRight]

Small correction: The Grace module with 144 cores is the one where they combine two Grace chips on one module. The Grace/Hopper version will only have one Grace chip, so only half the number of cores (so 72). Both versions will use the NVLink C2C interconnect bus to connect both chips.

[Dr.RichardBanks]

"Looking forward to the future of hovercrafts being covered in eels" I couldn't have said it better myself. _Back_ _to_ _you_ _Steve_

[Phloat]

Youtube suggested this while it was live and I was immediately like "...Nahhh, I'll just wait for Steve to explain it to me." 😂

[MarktheRude]

6:10 Thank you Arch-Magos Steve for this explanation. You have deepened our understanding of the "4N-scale" machine-spirits.

[ProjectPhysX]

2:32 it's actually 4.9 TeraBYTES/s. But this is for the full GH100 chip (and this is the sum of memory + IO bandwidth, which is really misleading). In the 700W SXM version memory bandwidth is the full to 3.0 TB/s and in the 350W PCIe version it's "only" 2.0 TB/s. Still impressive, but at the same time not nearly enough to satisfy the demands of almost all scientific compute applications.
4:33 FP8 certainly has its use in AI, but for other scientific compute it's entirely insufficient. You only get 256 representable numbers after all. FP16 in contrast is gaining more wide adoption in scientific compute such as CFD, especially in the form of mixed precision.