Building a GPU cluster for AI

Learn, from start to finish, how to build a GPU cluster for deep learning. We'll cover the entire process, including cluster level design, rack level design, node level design, CPU and GPU selection, power distribution, storage, and networking.

This talk is based on the Lambda Echelon GPU Cluster whitepaper. The whitepaper can be found above.

Slides for the talk can be found here:

Errata:
- Slide 46 contains an erroneous diagram with a connection from the storage server to the compute fabric network, the storage server does not connect ot the compute fabric network. The correct diagram is available in the whitepaper.

[peterxyz]

Thanks. I’m planning on building a “massive” 2 GPU system for home use.

[randahan]

Extraordinary presentation. Covered all the important topics in depth and with real teaching talent. Many thanks!!

[fundyourhustle]

One of the best presentations on GPU cluster design, even at 3 years old. Great teaching skills!

[onlooker]

Ground level details with all the critical aspects covered nice for GPU Cluster to the last cable length calculation.

[carlschumacher]

Its nice to see a holistic explanation of designing / building / installing a complex multi-rack system...As someone that has spent years working on both sides of the "analog/digital divide" (physical data center world / digital world's various segments), the un-sexy physical aspects of available rack space / power / cooling / floor loading / network uplink bandwidth are often overlooked (often assumed)...A semi arrives with a pallet: "Hey Carl, you can have this online in a couple days, right?"

[yassinebouchoucha]

Thank you for highlighting an underrated topic/options that company should re-consider within their compute infrastructure.

[lovanda]

Best of the best presentation on server clusters. Author presented deep understanding of server clusters so that he explains things in an easy way. thank you!!!

[dr.mikeybee]

Thank you. You got me started years ago with your lambda stack -- the only way I could get TensorFlow installed on Linux.

[ProjectPhysX]

Lots and lots of A100 GPUs. Every single one of them is a monster, almost 2x faster memory than the next best GPU. An entire room full of A100 racks... holy cow.

[AjaySimha-sy]

What an amazing presentation - one of the better videos I have watched. Great breadth and depth.

[randahan]

Most professional and holistic explanation I heard about this topic.
Thank you so much!!

[cyberspider]

Highly appreciated...Youtube should have a separate category called Founder's video.

[NSPK-]

Very expert suggestions for hpc and compute sizing.

[HarishN.J]

Hey Stephen, this is highly informative. I work on this clustering. Now am able to connect the dots and get the bigger picture.
where can i read about the relationship between numa topology and GPU peering capability.

[loadmastergod]

I want to build a multi dual epyc 7742 based system for goofing around learning this stuff.

[HankGallows]

My machine learning team consists of me baby

[ilyboc]

Really good analysis and presentation!

[julianfiacconi]

Still most relevant today, 2 years later. Thanks.

[glennisholcomb]

I have three computers, and a nas, and a external hub. I think that I don’t need a another server because of the NAS. As far as my architecture goes, is there anything else that you can advise?

[rosenangelow]

Tell me how difficult it is so i can buy your solution kind of talk