How do Graphics Cards Work? Exploring GPU Architecture

Graphics Cards can run some of the most incredible video games, but how many calculations do they perform every single second? Well, some of the most advanced graphics perform 36 Trillion calculations or more every single second. But how can a single device manage these tens of trillions of calculations? In this video, we explore the architecture inside the 3090 graphics card and the GA102 GPU chip architecture.

Note: We chose to feature the 30 series of GPUs because, to create accurate 3D models, we had to tear down a 3090 GPU rather destructively. We typically select a slightly older model because we're able to find broken components on eBay. If you're wondering, the 4090 can perform 82.58 trillion calculations a second, and then we're sure the 5090 will be even more.

Table of Contents:
00:00 - How many calculations do Graphics Cards Perform?
02:15 - The Difference between GPUs and CPUs?
04:56 - GPU GA102 Architecture
06:59 - GPU GA102 Manufacturing
08:48 - CUDA Core Design
11:09 - Graphics Cards Components
12:04 - Graphics Memory GDDR6X GDDR7
15:11 - All about Micron
16:51 - Single Instruction Multiple Data Architecture
17:49 - Why GPUs run Video Game Graphics, Object Transformations
20:53 - Thread Architecture
23:31 - Help Branch Education Out!
24:29 - Bitcoin Mining
26:50 - Tensor Cores
27:58 - Outro

Animation: Mike Radjabov, Sherdil Davronov, Adrei Dulay, Parvesh Khatri
Research, Script and Editing: Teddy Tablante
Twitter: @teddytablante
Modeling: Mike Radjabov, Prakash Kakadiya
Voice Over: Phil Lee
Sound Design and mix: David Pinete
Additional Sound Design: Raúl Núñez
Supervising Sound Editor: Luis Huesca

Erratum:
04:50 Ubuntu is a type of Linux
08:45 3080 has 10GB, not 12GB.

Image Attribution:

Wikipedia contributors. "Embarrassingly Parallel", "Graphics Processing Unit", "Parallel Computing" , " SIMD", " Single Instruction, Multiple Threads" , "Thread block (CUDA Programming)". Wikipedia, The Free Encyclopedia. Wikipedia, The Free Encyclopedia, Visited October 18th 2024

Textbooks and Papers
A TON of information was found in Nvidia's white papers. You can find them here:
We recommend the GA102 architecture white paper and the fermi architecture white paper.

#GPU #GraphicsCard #Micron

[BranchEducation]

One question you may have is why did we choose to feature a 3090 Graphics Card? To create an extremely accurate 3D model, we had to purchase the graphics card and do a rather destructive teardown to the extent that we desoldered all the components. To avoid wasting a perfectly good graphics card, we typically buy our models as broken or 'for parts' on eBay, and they didn't have any 4090s, so we purchased and used a 3090. Just so you know, I (Teddy) have a 3090ti in my computer, an i9-12900k, 64GB ram and a bunch of SSDs. Also, a 4090 performs 82.58 Trillion calculations a second using 16384 CUDA cores. When it comes to 5090 graphics cards, it looks like they will be 21760 cores, with 32GB GDDR7 memory, and 109.7 Trillion calculations a second, with 170 RT cores and 680 Tensor cores.

If you have any questions, feel free to ask them here!

I am speaking sincerely; This channel does not deserve to be watched for free. It's far beyond the performance of TV channels. It is an excellent detail that such content is available for free on YouTube. Thank you very much. You guys are great.

[btxh]

the animation, the research, and the voiceover just scream quality

[KingofArsenal]

This channel is why YT and the internet should exist! It explains the most complex processes in a beautiful presentation that is visually appealing and complex enough with the correct information to explain it. It is insane the amount of details in this video.

[gaderic]

Big thanks to everyone that works on this channel.

[der.Schtefan]

I am an engineer, and the level of detail, and animations like showing the eye diagram from the ternary encoding, drive tears of joy into my eyes. Thank you so much!

[eXecue]

It's funny to think that this video may already help TPUs better understand themselves.

[shashannkaggarwal]

It is strange feeling to see this video floating around on YT with others that are far less informative and of lesser quality. I am grateful for this.

[Califfo_]

As a computer engineer, I cried with joy when I saw the architecture of the 3D GPU, and I cried even more when I heard clearer explanations than those given by my professors. Thanks.

[Jem-Holograms]

I'm taking a computer architecture class this semester, and I’m so excited to have found your video! Some concepts in computer architecture are hard to grasp just from reading a textbook. Your video made everything so much clearer and helped me score a 91 on my midterm. Thank you!

[lis]

Branch Education - one of the very few channels where ad is integral part of the video and is literally unskippable if you want to experience the whole. Amazing

[guyedwards]

I believe in the goodness, utility, and craft of what you're doing with this channel. I don't have a lot of money to throw around necessarily, but this sort of content is what the world really needs.

Your voice reminds me of the Codex Entries from the Mass Effect series. Very fitting for delivering the lore of our own universe ❤

[michaelvansoest]

This is the best animated GPU explainer video I've ever seen. Explaination, the given details, animation, all 10/10. I am so impressed!

[wedusk]

Keep 'em coming!

Update: I didn't even start watching the video when when I made the small donation because I knew how good it would be. And I'm happy to say that your channel has exceeded my expectations again. Once again, I will be using your content to introduce new comers to the field. As a professional, in this exact field, I'm overjoyed to know that there is a channel like yours spreading the passion I have for this highly technical but often misunderstood domain. Hats off to your entire team 🙌

[ExploringNew]

It's basically like magic at this point

[JoseeHanma]

Its insane how we can watch videos with such amazing graphics and thorough information for free.

[bhuvaneshs.k]

As a GPU Design engineer I can confirm GPU is so complex yet this channel has presented it so eloquently

[ravager]

I work in the Medical Devices industry so not necessarily my field. But these videos are so high quality, so well researched and informative, I genuinely cannot believe I get to watch such quality for free.

[Furly]

In a different world, this wouldn't be free. I feel like it's time I do my part to help keep it free.

[Maiklerulez]

If this content would be on Netflix. It would be one of the reasons to buy a subscription. Magnificent, marvelous and mind blowing