How long can Nvidia stay monolithic?

Is Nvidia losing the chiplet race? AMD & Intel are already building complex chiplet GPUs while Nvidia seems to be stuck with monolithic designs. Let's take a closer look!

0:00 Intro
1:24 Nvidia Chiplet R&D
3:48 Gaming vs HPC/AI GPUs
7:05 Process Node Implications / TSMC N3E
9:24 High-NA EU Implications
10:50 Energy & Space Efficiency
11:50 Competition from AMD & Intel
12:44 Conclusion

[MickenCZProfi]

Thank you for this video, always very informative, I had no idea that the EUV-next lithiography shrinks the reticle limit by a factor of 2, that changes everything.

[pwmaudio]

Overall good analysis on this video but you forget the most important (and in fact the only) reason why NVDA didn't move to chiplets yet is the limited packaging/interposer capacity (CoWos in case of H100) and HBM TSV production machines. Currently, NV can get any quantity of dies from TSMC N4 but can't get enough HBM and package them fast enough to meet the market demand... to the point that a third packaging factory is opening in Taiwan and NV already booked the production for the next year. Samsung is also opening a new packaging fab in Korea (for HBM CoWos) to sustain NV business.
Otherwise, keep the good work. Nice channel 👍

[samghost]

Thank you very much! I'am always looking forward for new videos from your Channel

[RobBCactive]

Rather than just calling the game gfx latency dependent it's better to realise that the frames are tightly coupled in a way that HPC calculations aren't.
An algorithm bouncing rays off a surface need the texture & colour to be known for example, if these rays are scattered and reflected then you need all of that early pass data to be available.
I've seen explanations that games effectively have a global area, splitting it across dies is believed to cause problems.
The difference is that if you could pipeline frames without synchronous requirements then each could take longer than the frame time, so long as they can start early on a wide enough GPU that can process several in flight frames.
So long as the output frames respond to user input quickly latency would still appear low.

[HighYield]

This is the same video I shared on Patreon almost two weeks ago, so if you watched this, you have already seen it. Next video will come sooner, pinky promise!

[Innosos]

If I had to make a guess, the next gen will just be a small refinement of Lovelace with larger dies (a.k.a. a 50, 60, 70 and 80 class GPUs with a typical 50, 60, 70 and 80 class die sizes) since there's so much space left this generation.

[theevilmuppet]

Wow - your work has always been amazing but you're continuing to improve your presentation and focus on the critical details.

Please, keep going!

[SirMo]

Nvidia's entire origin story has always been about building the biggest chip possible. The reason they haven't went to chiplets is related to this paradigm of always having the largest chip. As you said, their margins and scale allowed for this and no one else could follow since they simply didn't have volumes which could justify the cost. But this advantage is going away. And I think companies like AMD have far more experience with chiplets.

[ramr]

good to see you back :) hope everything is going well for you

[mikelay]

They will stick with monolithic for as long as they need to, in gaming at least.
Remember NVIDIA is not one to lay on their laurels, I am 100% sure they have chiplet based chips in their RnD labs just waiting for the right time to pull the trigger.

[falsevacuum]

And you were right, Nvidia made Blackwell from 2 chiplets.

[VideogamesAsArt]

Always enjoy watching and hearing your opinions. You do very good analysis, keep up the good work!

[BecomeMonke]

Wow you made some real dry topic real interesting to listen to, thanks for the video

[Treez]

You deserve more subs. Great content.

[lahma]

First time viewer of your channel here and I really enjoyed hearing your opinion on this topic which I've been thinking a lot about lately. I look forward to exploring your past and future content!

[RealLifeTech]

I would say Hopper Next is monolithic as Nvidia tries to capitalize on the AI boom with an early release and before the competition can launch something more interesting. Big corporations aren't that willing to take risk as they have their leadership role to loose while the under dog(s) can as they don't have a brand to lose if it doesn't work. Hopper Next Next will for sure be MCM because of the reticle limit. Maybe Hopper next is an intermediary generation and we see both a monolithic chip launching first to ensure leadership which is followed up by a risky MCM on the same architecture which takes longer to develop and has the potential to beat it

[ipurelike]

make sense, thanks for being super informative!

[Alex-iipm]

Chiplets are used for cost saving, they get better yields from smaller silicon and less wastage, monolithic will always be superior for gaming gpu's.

[DJaquithFL]

**Chiplet is synonymous with Cheap.** There's no other upside. A Monolithic CPU or GPU doesn't have their chiplets separated by millions of nanometers of added unnecessary latency. Even in Intel's Tile, each tile is specialized. The GPU, SoC, and CPU are all on their own tiles to avoid the latency cost from downgrading from a monolithic design.

[dozens]

There is a lot of upcoming tech that will primarily only push logic density forward.
With high-NA halving the recital limit, and GAA + Backside power delivery increasing complexity, it might be unwise, or even uneconomical to put L3 or L2 on the die at all. Necessitating die stacking to maintain the necessary bandwidth/latency.

Hopefully the thermal reductions gained from GAA and backside power will be enough to just stack cache directly onto the logic without issues.