How Much Memory for 1,000,000 Threads in 7 Languages | Go, Rust, C#, Elixir, Java, Node, Python

Recorded live on twitch, GET IN

ty piotr!

MY MAIN YT CHANNEL: Has well edited engineering videos

Discord

[thedoctor]

Using Python's asyncio for this test was the wrong thing to do. It's similar to what was done with NodeJS. Asyncio is an event loop, not a thread. Python has threading libs for threads.

[jonathan-._.-]

compaaring actual threads with async tasks seems kinda weird

[nunograca]

If I'm not wrong, C# uses a theard pool behind the scenes when using async/await and what it does is it recycles theards. That's why in the first test it was way up than the others. I think that was the threads pool being initialized with a bunch of threads.

[hansenchrisw]

As a Java apologist, it first got virtual threads in 1997 with version 1.1 (edit: later removed and recently re-added in v 19). Also, Java (and presumably.NET) pre-allocates a bunch of memory by default. Hence how mem looks high for small numbers of threads and it doesn’t increase until you hit bigger numbers.

[devotiongeo]

Creating a million concurrent "tasks" (or spawning processes as we call them in Erlang/Elixir) and allowing them to remain idle is one thing, while making those processes actually do something, such as each one of them having a persistent connection to a client and feeding it, is something entirely different. In practical terms, when it comes to real-time apps, the BEAM (Elixir/Erlang) outperforms all other languages by a significant margin.

This is precisely why Brian Action and Jan Koum chose Erlang for WhatsApp after years of experience with Yahoo Messenger and Yahoo Chat Rooms. If someone hasn't had the opportunity to work with any BEAM language, the above statement may appear to them as an empty boast, and I can't blame them for that.

[shreyassreenivas]

Go reserves 4K of memory for each thread's stack so you could do quite a bit of work on each of those threads without incurring further costs.

[casperes]

There's also the memory vs. speed tradeoff. Sometimes keeping more things in memory can also make it faster. If the managed environments that have a higher starting point in memory usage already has a bunch of kernel threads lying dormant in a thread pool that's taking up memory but speeds up spawning of threads.

[Deemo_codes]

Each elixir process spawns with a 50k heap, garbage collection happens on a per process level (you dont stop the world, you stop a process). This is because the way processes are used in elixir is like how microservices are used. Each process does a small amount of stuff then sends a message on to another service.

The erlang vm that elixir runs on will launch 1 scheduler per cpu and does pre-emptive multitasking. So if you had 1mn processes doing stuff you would get each process executing for a few ms then being switch out and added back into the queue that the schedulers pull from. So if you have more cores you get more parallelism, if you only have 1 core you still get concurrency.
Whereas async runtimes tend to be cooperative require some form of explicit yielding from a running task, elixir will just swap stuff out. Makes it good for soft realtime stuff, if you want to do cpu intensive things you can delegat to NIFs (native implemented functions) written in C or Rust. The rust ones tend to be safer since panics are caught and raised as errors in elixir. Wheras a panic in C will crash the whole VM

[TanigaDanae]

An information that has not been said in the video is that: async functions in C# are State Machines and Tasks (are part of the Task Parallel Library and) are automatically run in thread pools. So the only internal state these async functions have is the time they need to wake up, and all Tasks could theoretically have the same wakeup time.
I would've loved to see a C# Thread implementation. I suspect the C# compiler is optimizing redundant Tasks away since they lack any side effects.

[NameyNames]

As likely already pointed out, C# uses a thread pool, and will definitely not create a gazillion threads in this test, and the memory required to house all of these insignificant tasks will be very small, which is apparent in the test results.

I tried it out in LinqPad, but with one additional task whose only purpose was to keep track of the number of simultaneous threads actually in use. For 1 million tasks, the actual active thread count peak never even exceeded 50 on my system (usually much lower). No wonder, when all that the tasks are "doing" is async-waiting on a delay.

This benchmark is broken in the sense that it doesn't really do what the author thinks it does, i.e. it does NOT create a lot of threads (virtual or otherwise) in all languages/runtimes, and measuring the memory usage is thus close to pointless.

[Hallo]

C# has the lowest memory usage because it is using the threadpool, that recycles blocking threads, like when calling Task.Delay. So there aren’t actually a million threads created but rather they are queued into the threadpool. To avoid this create the threads explicitly

[SirBearingtonSupporter]

You actually pointed this out early on. In the Java and C# version, he uses "ArrayList" without specifying the size.
ArrayList in both these languages hold an actual Array object. It's why the lookup time for "get" is a memory address lookup time.

When Java needs to expand the array size, it creates a larger array that is twice the size of the current array size. I believe the default is 10.
Java also doesn't run the garbage collector unless it needs to be run or specifically invoked with System.gc.
Because the JRE doesn't plan ahead for your bad code, it just looks for a new place to put the object in memory, leaving all the old references that need to be deleted alone - because the GC will deal with it as needed.
Just to recap there are several arraylist objects each holding an array of size n (below) in memory - and if the JVM is given enough memory, all 11 of these will still be there.

So that means there are 20510 threads in memory on the test.


While his approach to joining all the threads was barbaric, it's also the accepted answer on StackOverflow, we are not measuring the speed of the execution, just the memory of it.

If you were not trying to measure the memory performance of threading on difference languages, I would actually give java more threads to manage the threads (parallelize stream).

Finally thoughts,
We aren't concerned about thread space in production equipment, we are concerned about execution time and if my entire program hangs because one calculation couldn't be done, I'm missing out on something important - it could be a trade, moving servo for a robotic (self driving cars) or producing an input for a chess game. Collecting the information that I can allows me to implement an algorithm that is capable of making educated guesses based of what was calculated.
If we do care about thread space, we would be better off doing single threaded applications since we don't have an overhead associated with the effing cost of the thread.

TL;DR
Something something short equal something something int because the JVM go fast blah blah addresses blah blah blah 4. (primitive array blah blah addresses, blah blah)

[bryanenglish]

You forgot the extra Rust thread it takes to track all the bullshit drama in the Rust community

[diadetediotedio]

C# was the winner, clearly everybody was expecting this

[baxiry.]

There is some important information not mentioned in the article. Goroutines are compared to threads, whether real or virtual, but they are not compared to an event loop. Go has event loop libraries, and since the author of the article has used the event loop in other languages, he should also use it in Go to ensure an unbiased comparison.

Additionally, the advantage of goroutines over threads is their portability; they do not depend on the operating system. If your application requires low-level operation, such as with chips or microcontrollers that do not have an operating system, a goroutine can still be executed. This is not possible with threads, as the language does not perform the task—the operating system does. Where there is no operating system, there are no threads.

One last thing: when an application uses system threads, the system reserves memory. The question is: Did the author of the article account for the memory reserved by the system?

[MyriadColorsCM]

9:30 - In the 19th century the german mathematician Georg Cantor proved that there must be more than one kind of infinity, such a the infinity of the natural numbers, and the infinity of real numbers and so on, and that there are larger infinities than others. The smallest infinity is that of the natural numbers, and its called Aleph Zero.

So yes, Buzz can indeed go to infinity and beyond, so long it is mathematical infinity.

[robfielding]

Go is definitely not a memory hog; at least for IO-intensive tasks. The main thing is that the Go libraries are always very careful to stream large inputs; rather than buffer them in memory. Java itself doesn't really have major memory issues beyond spawning threads; but in any large Java project, the code will be full of things being buffered into arrays, rather than being streamed. I tried rewriting netty to make it stop doing dumb things; and just switched (permanently) to Go. Part of Java's program is also the legal issues of shipping a JVM; and the existence of Oracle thumb-breakers and lawyers; to come punish you for shipping.

[Trekiros]

Intro: let's not compare apples to potatoes
The rest of the video: compares making threads with maintaining an event queue

[RngeRpidz]

You're 100% right about the complexity of the task.
But also, I would have stopped reading after they said they used ChatGPT to come up with the code.

You need to have these contributed by people that actually write this language and that actually understand this language.
The ambiguity between what the code was actually doing in all of these was horrible, as other commenters have also pointed out.

[wlockuz]

It should've been "To infinity and NaN" as an homage to JavaScript.