The Coming AI Chip Boom

Nvidia's GPU evolution kicked off the neural network revolution. However, while GPUs run neural network algorithms quite well, they are not specifically designed for it.

So companies have started to develop hardware customized for running specific AI algorithms - dubbed AI accelerators.

Today, the AI accelerator hardware market is estimated to be worth over $35 billion. Venture capitalists poured out nearly $2 billion for AI chip startups in 2021.

TSMC considers AI accelerator hardware as one of their top secular drivers in revenue for the near future. In this video, we are going to look at what these weird things actually are about.

Links:

[out_on_bail]

Wish i bought NVIDIA Stock when i watched this

[tykjpelk]

I'm very excited about the silicon photonics approach. Photonic chips don't need to perform multiplications one at a time, but rather do the whole matrix multiplication in parallel, which makes it a O(1), or constant time operation. The chip needs to be configured to multiply by a certain matrix, which takes milliseconds, and can then perform matrix multiplications as fast as you can give it inputs. With 50GHz modulators and photodetectors readily available, I'm excited to see what companies like QuiX, iPronics and Xanadu will achieve.

[gotfan]

You missed 2 important AI chip companies. UK based Graphcore and US based Cerebras which has designed a wafer scale AI chip.

[rich_in_paradise]

You didn't mention one of the key aspects of Google's TPU (and other specialist AI processor) compared to a GPU is also the number representation. GPUs can process 32 and 16 bit IEEE floating point numbers. But for AI work Google found that the fractional part of the number (commonly known as the mantissa) is less important than the magnitude (the exponent) and so they changed the number of bits allocated to each in their own BFLOAT16 format. That makes their processors better for AI, but relatively useless of other kinds of numerical computation.

[AB-uvkg]

My new favourite channel. Looking forward to catching up on what you've already released and your future videos :).

[pirojfmifhghek]

This is a great video. I'd been predicting this for a while, simply because of all the gains that I'd heard about in analog chips from Mythic AI. Glad to see that more companies are getting in on this. It's also the perfect time for computing to start implementing new components for neural networks. The available bandwidth for motherboards has gotten ridiculously large lately, so there's a lot of headroom. I think it makes a ton of sense to start using dedicated AI chips for a whole host of common tasks and applications. The efficiency and speed gains would be enormous.

This change in computing is gonna happen eventually. We're all gonna be socketing a plethora of purpose-built AI chips into our computers soon. There are just so many "fill in the blank" potential uses for AI. Anyone playing around with AI art generators can see that the results are surprisingly sophisticated and sometimes spooky. But damn does it take a lot of horsepower to do that stuff with a GPU. It takes a 3090 running at full power for several minutes just to produce results. It's horribly inefficient and slow. But it does remind me of the delight people experienced with the early internet. The internet used to be meagre and slow and truly amateurish, but everyone still shared that undeniable enthusiasm for being the first pioneers in a new world. And that's where we're at with AI.

[ishan]

As a ML researcher it is interesting to watch, unless run at extreme scales, regular chips are just enough especially for inference

[AjinkyaMahajan]

8:15 MAC diagram, you won my heart.
This channel motivates me to keep learning and researching and never give up regardless of how many failures.
You are a true person who understands affection with technology.
Cheers ✨✨

[Davethreshold]

Oh drat! Another YT channel that I'll become addicted to! Seriously, I am FASCINATED with technology, mainly computer tech. You cover many aspects of things that I have not quite seen before. Good work! 🧡

[reh-linchen]

Love your AI example of taking eggs for ping-pong balls with 100% confidence. It is hilarious!

[phinguyenvan]

I think the problem is not people cannot design the AI chip that run faster than Nvidia GPUs, but the problem is the huge software stack behind Nvidia GPUs. I have tried both IPU and TPU and believe me, the software is painful as hell.

[Palmit_]

Thanks Jon :-) How you get your head around, and then write and deliver stuff of this complexity is mind-boggling! Do you even sleep? Do you work? Are you an automaton?? You're incredibly efficient and skilled in any case. You should def do a youtube live Q&A. I'm sure thousands of your viewers have lots of questions each. Thank you again.

[lumanaty]

Great Video. Extremely excited about this industry and really hope to get involved. Met up with some Cornell scientists and discussed lower-power electronic AI accelerators. This space is ripe for innovation that will lead to 10x improvements in power and inference speed. Amazing stuff out there.

[aarch]

Just a quick thing, I’m like 95% sure Xilinx is pronounced Zy-links. I grew up about a mile from the HQ, and frequently had employees from there read books to me in elementary school.

[Luxcium]

The way you talk about your topic with passion, confidence and humility but also the rhythm and the synco-tonic of your voice makes those videos not only interesting but relaxing and calming you are such an amazing person yet because of how humble you are it makes me feel so strange to give you compliments but I guess that somewhere inside of you, you know that you are doing something right and something good 😅 So I must share this with you because you deserve many compliments 😊🎉❤

[PlanetFrosty]

Dimensity, is doing a good job.

I’ve worked on silicon photonics for 25 years and we have a new design we’re now working on a solid state SOC which includes unique photo sensitive “protein” molecule, but more another time...these are now the “Wet Works” as we try to evolve to new methodology in visual and human language understanding.

[joshhyyym]

7:11 the box labelled as system processing is actually just a bench top power supply. It is supplying 1.00volt and 0.000A. It is not doing any processing. Great video btw, big fan of your channel

[artemglukhov]

Great video that presents a nice overview of the current technological scenario. Could you please add the DOI for the papers you are quoting? Just for an easier search.

[avanisoni]

Great explainer!!! I would highly suggest attaching your research source material in description.

[helmutzollner]

Very interesting! Excellent overview on the subject. Thank you