What runs ChatGPT? Inside Microsoft's AI supercomputer | Featuring Mark Russinovich

Get an inside look at the AI supercomputer infrastructure built to run ChatGPT and other large language models, and see how to leverage it for your workloads in Azure, at any scale.

Go behind the scenes:
-For how we collaborated with NVIDIA to deliver purpose-built AI infrastructure with NVIDIA GPUs
-How Project Forge checkpointing works to restore job states if a long training job fails or needs to be migrated
-How we used LoRA fine-tuning to update a fraction of the base model for more training throughput and smaller checkpoints
-How UK-based company, Wayve, is using Azure's AI supercomputer infrastructure for self-driving cars
-And how Confidential Computing works with Azure AI to combine datasets without sharing personally identifiable information for secure multiparty collaborations.

Mark Russinovich, Azure CTO, joins Jeremy Chapman to break it down.

► QUICK LINKS:
00:00 - Introduction
01:15 - AI innovation building specialized hardware and software
04:22 - Optimizing hardware
05:40 - Improved throughput
06:17 - Project Forge
08:01 - Project Forge checkpointing demo
10:02 - LoRA fine tuning
11:29 - Use AI supercomputer infrastructure for your workloads
12:34 - How Wayve is leveraging AI supercomputer infrastructure
​​13:47 - How Confidential Computing works with Azure AI
15:21 - Wrap up

► Link References:

► Unfamiliar with Microsoft Mechanics?
As Microsoft's official video series for IT, you can watch and share valuable content and demos of current and upcoming tech from the people who build it at Microsoft.

► Keep getting this insider knowledge, join us on social:

#Azure #OpenAI #Supercomputer #LLM

[Ionizap]

A fascinating look at the big picture, I've always wondered about that. This vid shows it . I'm an Apple Guy at home but I am so impressed with what MS has done in the last few years. I run MS Edge and Bing Ai on my desktop at home and at work. The brilliance of these people is just beyond belief.

[harriehausenman]

Finally some actual insight into the system! Thank you so much for clarifying LoRA usage, clustering and your "AIOps" apporach 🙏
Good to see someone so knowledgeable talk about it!

[AlexanderMorou]

Holy crap. The number of iterations in the technology stack that they've done in the background is massive. To make these tools usable, scalable, distributed, and likely things I don't even comprehend since it's out of my domain, is massive.

I can't imagine how much it costs to use this stack, probably an enterprise level offering.

[flwi]

Impressive to see how Microsoft has transformed.

[ankushalgudkar]

Absolutely fascinating look into the BTS of these incredibly powerful AI tools now. I was always guessing that the easier something looks and feels for the consumer, ALOT of manpower, cost, time, resources would have gone into making that a reality. Massive respect to Microsoft for the transparency and releasing these videos on how they're building their AI capabilities.

Incredible time to be alive!

[SirIsaacMewtonIII]

microsoft is just killing it in this area. seriously impressive engineering. i wonder if they'll ever release an ai agent for pc that will fix os-level issues (eventlog issues, give human-readable advice based on eventlog errors, etc)

[naipaulojar]

A very good session showing what is under the covers with Microsoft's AI computing. Microsoft is going a good job with the confidential compute which is ultra important for business who want to use AI for both the business & their customers.

[glitchedpixelscriticaldamage]

after listening to these guys.... i feel that my work is useless and only a few people over the world get to do really impressive stuff.

[shabsZA]

Love hearing about the hardware that exists behind the scenes - thanks for sharing

[..]

It's impressive how GPUs have become such a powerhouse. CUDA was a bright idea way back when. Wish AMD competed better here.

[ShareReachCommunity]

This was so nice. Cant wait to learn more about Azure.

[TomWhi]

I love it when Mark is on the show, he's an absolute tech heart throb!

[aldosansan]

Russinovich Is one of my biggest inspiration in IT!

[cowboyuniverse]

I’ve never seen comp specs as this sexy. It’s freakin insane.

[Shahawir]

The fat tree topology still used in every data center, brilliant and simple ideas live very long

[ivan_i_am]

Mark is the Azure, fascinating amount of knowledge came from this guy in short period of time. Go systernals! 😊

[tombranson]

Mark is the Master of the Microsoft Universe!!!! He has got more genius in his pinkie that I got in my entire body.

[TerragonDE]

Very interesting insight, thank you! Microsoft is back :-)

[mantrax]

Just came to my youtube this video... it is very impressive, the infrastructure.

[huytranvan]

Can't believe I spent nearly 16 minutes watching a MS Azure advert... :) So interesting