why can’t computers have thousands of cores?

If you're watching this video on any device made in the last 10 years, be it a desktop, a laptop, a tablet or a phone, then there is an extremely high chance that your device is powered by a multi-core processor. Since the release of the first dual core processor in 2005 by IBM, it has become more and more common for computer processors of all varieties to be multi-core. This is in direct contrast to laptops in the 2000's, like my iBook G4 for example, which was powered by a single core PowerPC processor at around 800MHz. Now a days, it is common for any desktop to have at least 4 cores, and clocked easily into the GHz.

But what does it mean for a processor to have multiple cores? How does a processor with multiple cores work? Why are more cores better than just one? How many cores are too many? These are all really important questions, and, like you, I was curious to find the answer.

🏫 COURSES 🏫

🔥 SOCIALS 🔥

[utubekullanicisi]

Both Intel and AMD are rumored to release server processors (codenamed Sierra Forest, and Turin respectively) with more than 200 cores in the next few years (as soon as 2024). Servers will continue to scale well and make use of as many cores as you can give them.

[davidthacher]

Technical Tradeoffs:
1. Power - Power Consumption / Thermal
2. Area - Core Size / Cache / Memory Bandwdith / IO interconnect / Yield
3. Performance - Architecture / Instruction Set / Clock Speed / FPGA / Multicore / CMT / NUMA / SIMD

Business Tradeoffs:
1. Algorithm Capability / Application
2. Market share / Manufacturing Size
3. Time to Market / Training

[larrydavis]

As a former programmer, not all functions of a program can be run in parallel. Sometimes a function needs to wait for another process to finish before it can proceed.

[veleriphon]

We see the cores to code limit already with Threadripper 64 core, 128 thread units. It's hilariously overpowered for most tasks.

[badass]

Also, a big factor is that many programs have linear logic. Amdahl's law shows how well a task scales with multiple cores depending on how parallel it is. For 50% parallel tasks above 4 cores is pointless. For 75% parallel above 16 cores is pointless. You just don't gain performance and that is baked in the logic of the task. Many cores are great when doing multiple of the same task without caring which task is completed first.

[benandrew]

I've recently started a job as a technical support engineer / technical writer working on complex digital signal processing applications. Videos like this, like yours, are exceptionally valuable to me as a non-programmer. There are limitations to design, implementation and efficiency that are contingent on factors entirely within low-level hardware programming, and having them explained so succinctly makes my job way easier, because I'm being provided with a higher level understanding that I can pass on to my reports. Props.

And, on a more personal-craft level, the quality of your videos in terms of rapidly explaining complex topics through efficient use of graphics and constrained use of jargon is very inspiring. Well done.

[AlessioSangalli]

I always categorized "asymmetric" systems the ones that, while having multiple cores, do not have cache coherency - so it's up to the programmer to synchronize the cores. I once worked on a system that was running Linux on a core and an RTOS on the other, with independent MMUs

[dannygjk]

Back in the 80's I read a magazine article about the 'Connection Machine' which had 65536 processors but each processor wasn't like a core that we think of these days. Each processor was a tiny simple device which operated in a massively parallel architecture. Such machines had a limited practical value since they were specialized for a narrow range of problems and were also limited by being a 'hard-wired' architecture. Right now I can't think of a better description but I do know I should word it differently. I vaguely remember it had clever solutions to how to break down tasks and how the machine's processors worked together. It makes me think of things like ant colonies.

[TrippTech]

(electrical engineer here)
LOVE this, great explanation!!! One thing i would have mentioned, especially when talking about single core chips is "out of order" execution where the chip executes instructions as soon as its ready, rather than everything waiting in a queue. Probably one of the biggest advances in chip design in history.

[RoelBaardman]

You're more describing the limits of the Von Neumann architecture and our current (mostly sequential) models than anything else imho.
Have a look at Erlang and the Actor model, and I think you'll argee that processors can scale just fine if we rule out shared memory.

[pwnmeisterage]

GPU and SPU cards already pack hundreds or thousands of "cores" onboard. They can only process simplex tasks, not complex tasks, but they can stream their outputs in near-realtime.
They suck a lot of power and spew a lot of heat while working at full load.

[desmondbrown]

I think it would be interesting to go into why GPUs CAN have so many cores, be parallelized more effectively and with better thermal efficiency, but CPUs cannot. I know the answer, but I do think it would be an interesting follow up video.

[RonJohn]

AMD also released their first dual-core CPUs in 2005. (Of course, not everyone instantly bought them...)
Another issue with huge core counts is cross-core communication: threads usually want to talk to each other, and the wiring between all those cores gets crazy. You effectively get a traffic jam in there...

[dmitrykargin]

Scientific computing guy here. Most often we hit RAM bandwidth limit. Sometimes we use all bandwidth by a single core with optimised AVX2 code and perfect memory layout. Using more cores will just slow down everything until you switch to a platform with more DDR channels.

[MichaelBristow]

My first computer had 48k (it was an Apple II+ with 16k extra memory). I remember learning some assembly language. Now I have a multi gigabyte memory (255 Gb SD, plus 128 internal) phone which takes 8 Mb photos... I am so amazed at how far we've come and what the computer is actually doing to even display what I'm typing right now. It's mind bogglingly amazing...

[LilacMonarch]

The "number of transistors doubling every 2 years" might already be hitting its end. The problem is in order to add more, they have to be made so small that it's impossible to keep the circuits properly separated. The gaps are so small that electrons easily jump across, causing shorts. Maybe we will see an increase in larger sized CPUs, but that will have its own problems.

[ccflan]

On of the best YouTube channels out there, it feels like you should pay to see this Content so thank you

[renchesandsords]

to be fair, in the science and datacenter space, that kind of core density can be effectively leveraged, threadripper and epyc proved it, and the development of processors like genoa and bergamo only serves to drive that point home further

[k_gold]

"If you're watching this on any device made in the last 10 years.."

Me watching this on my 2008 Windows XP Professional PC with an Athlon chipset: *You're wrong*

[xeridea]

The main issue is that, many tasks don't gain much efficiency from being split to many cores, due to having data dependencies on previous instructions. Generally, better applications for multithreading are those that have workloads easily divided up. Anything to do with graphics tends to be heavily threadable, which is why GPUs these days have upwards of over 10, 000 tiny cores, you have millions of pixels on a screen, so it is easy to split up the work. Game logic, however, isn't as easy to split up, which is why games don't generally benefit from having more than 6 CPU cores. It would be trivial to have a CPU with 1000 cores, just shrink the cores. With CPUs though, it is generally better to have a smaller number of cores, that are better at executing code fast, than it is to have a crazy amount of simple cores.

It is significantly more energy efficient to have more cores if workload can use them, which is why GPUs are so much more efficient at drawing graphics than CPUs. On the flipside, GPUs are pretty bad at general code, since to effectively use them, code needs to be what is referred to as "embarrassingly parallel". Many non graphics tasks are still able to be effectively programmed on the GPU, so they are still used for non graphics tasks, just not as CPUs.