Python is NOT Single Threaded (and how to bypass the GIL)

I've encountered this misconception about Python a lot: Python is single threading and to use more then one core on your CPU you must use the multiprocessing module. In this video I show that this is utterly false and that you can be very CPU efficient with Python when calling out to something like a c-extension (numba in this case). I feel this is an important nuance to understand, hence this video.

MY OTHER VIDOES:

#python #threading #multiprocessing

[JackofSome]

To the inevitable "nuh huhh" commenters: save your collective breaths. I have lost any interest in discussing this with anyone unwilling to listen. This video isn't about language superiority or "what X was designed to do", it's about showing a way to write better code involving python. If that is helpful to you, great. If that somehow makes you angry, maybe take a step back and re-evaluate your priorities in life.

I will no longer be responding to "no this is wrong" comments, and I _will_ remove any of them that are disrespectful or offensive. You have been warned.

[JethroYSCao]

Your channel fills the much needed niche for intermediate-leveled programmers. Sometimes I'd rather watch a succinct video on a topic than read through pages of docs or the source. Got yourself a new sub :)

[StefanNoack]

5:48 you are not understimating the CPU, but the JIT compiler which just folded your for loop operating on fixed input into a constant. This also explains why it barely goes above 100% when run in parallel, since the part of the execution that runs wthout the GIL is very short for each run.

[officiallyjk]

The little detail of you speeding up typing code made my day. It only saves a few seconds but makes the video just a bit more enjoyable to watch

[laundmo]

The main issue is that a video like this will have the opposite intended effect - people are now going to go around and use threads simply because they heard that python is in fact using proper system threads, ignoring the nuance. Ive seen that happen often enough.

Ultimately, in python threads ACT as if they aren't real threads - nicely proven by spawning a few threads that all append the current timestamp to separate lists. You can very clearly see that the multiple threads are never executing at the same time. This is your pudding. It's what the threads act like. that they don't act like that in NumPy etc. is a nice detail, but ultimately not disproving the core statement: python threads are not (read as: dont act like) threads. I will continue to point this out to beginners and i will continue to recommend other ways of speeding up their code - like using numba (with gil-released threads if you want more speed).

Ultimately, its the same as telling people python is interpreted - it's also compiled, but explaining that to beginners is entirely pedantic as python acts entirely like a interpreted language.

[usingvancedplzdontban]

9:00 was mind-blowing ! ! Gonna try that some day when I do some heavy work . Thank you for sharing your knowledge !

[LoneTech]

Very good demonstration.

There are 4 distinct levels of thread execution touched on; physical processors, SMT (simultaneous multi threading, where one core has multiple contexts; "virtual core" is a bit of a misnomer as they have their own physical registers and are just as real as the sibling they share execution units with), operating system threads (which were repeatedly referred to as "proper"), and in-process coprocesses ("green" threads). The load spike moving from processor to processor in the GIL demonstration is probably execution, not threads, moving between cores. It just varies which thread acquires the GIL. Linux tends to keep threads in the same core when possible, something that can be enforced using thread affinity. This reduces overhead from warming up the caches for particular tasks. In some processor cores, e.g. Xmos XS1, a single thread can't use all the execution resources; in XS1, 4 threads are required to saturate the pipeline. The multiprocessing example probably bottlenecked in the scheduling of work or transfer of parameters and results, which are serialized due to execution on the parent control process. Larger queues are often used to mitigate this, but it won't help if the task generation takes longer than task execution.

[sendivine]

Even if python can make system threads as you demonstrated in 2:00 ~ 2:55, I think it is NOT truly multi-threaded because of the GIL.
A user who uses system threads expects those threads to be literally run parallel; but in python, thread scheduling is done by GIL which means that only a single thread is allowed to be run at a time.
You can even see this in 2:53 that only a single thread (the one that is highlighted) is in state R (Running) and others are in state S (Sleep), due to the GIL.

In the latter case, I'm not familiar with the internals of NumPy or so, but it bypasses the GIL through FFIs, and thus almost (still, not all) states are in R, but this is thanks to the libraries that use FFIs, not python's threading. So it is not a fair example for arguing that "python is not a single-threaded". All language that has GIL in it can bypass GIL by using FFIs or libraries that use FFIs like this.

So, it is true that "Python is NOT Single Threaded" literally, but many people don't think because it is still sequential.

[ben]

People say "single threaded" when they don't know what it means. What they mean is, python does not run code in parallel (multicore). Using C++ in Python (e.g. through numpy) IMHO is not a positive, its a escape hatch that has been used over and over again because of this Python limitation. Python succeeded despite its GIL, not because of it.

[karlkareemmelaimi]

Man, you make great videos. It's just at the right level. As someone who would class themselves as an intermediate Python programmer, your videos provide just the right amount of information and examples for me to immediately apply them and run with it. Thank you and keep up the great work!

[DanEllis]

Firstly, I don't think anybody who knows Python thinks it's single-threaded. If it were, it wouldn't need a GIL. Secondly, async is not green threads, it's cooperative.

[willboler]

When I was first learning Python 3 years ago, I fell into this category of being upset with Python's GIL and claiming it's "not true multithreading." This being because I came from a C++ background, where if I needed to do parallel processing on complex data types/structures, such as flying PSO particles in a distributed fashion, it was fairly straightforward in C++ but rather impossible in Python without the performance hit.

Since then, I've learned to use Python as a "wrapper" for calling more complex operations underneath. Occasionally, I still find myself using the multiprocessing library as a clutch to run a set of numpy or pandas operations in parallel, especially when I'm in a bind. But, the one thing Python seriously lacks is native performant parallel processing on complex structures. It's not a problem if you can fit your code into the interface of one of the underlying libraries, but can be a problem for novel approaches in research. It's still my biggest gripe about Python, even tho it's good at almost everything else. The whole pickle approach to passing args to processes almost made our work a no-go in a previous project.

[Sevenhens]

Great video!

I think the misconception comes from the expectation that python should make threads that run parallel. The GIL prevents them from running in parallel, but your operating system will schedule the threads, so it acts more like async as an analogy but letting the OS take the wheel.

This is all by design to prevent 90s-2000s era thread lock management like in C++ which was a mess. And anyways the web has shown the world that a single thread is all you need for GUIs (JavaScript stuff). Almost everything else like you said with numpy will bypass the GIL with c code.

[szeoretical]

Good video. Racket calls its green threads "thread" (which may not be ideal in all cases) and its multiprocessing thing "places".
I like that it calls it places, because it is a nice analogy. I started writing an explanation but it looked to much like a book, so I deleted it.
Another thing I want to mention: it also has "distributed places" which is a way to do distributed programming / writing programs that run across multiple machines (of course there is no magic to make that super easy).

[levb]

Right off the bat, The advice on using multiprocessing instead of threading came from Guido himself, it's not a "misconception", it's a quote from the creator of Python, And he said it without the context of "what if you offload some tasks in your code to a CPP code".
The second part is, nobody ever claimed that python is using green threads, of course it creates a system thread. But because of the GIL every thread waits for the GIL to free up for execution, in that making your threading tasks not truly concurrent and rendering it being single threaded in that regard, no matter what your wrote in your code. unlike NodeJS, Golang and others.

[lawrencedoliveiro]

The fundamental reason for the GIL is that you cannot manage reference-counting without it. Python, like Perl, uses reference-counting as a first resort for managing memory, only going to full-on garbage-collection à la Java when reference counting is no longer enough (like when you have reference cycles). The trouble with using garbage collection for everything is that it means your innocent little script, if it runs for long enough, can end up consuming all available memory on the system unless you impose artificial limits to constrain it. Python tries to avoid this. Hence the GIL.

There are Python implementations that have no GIL -- e.g. Jython. Most people prefer CPython with its GIL -- should that be a surprise?

[punishedsnake]

What a clickbait video. Offloading some calculations through FFI does not make python multithreaded in a way programmers think when they hear "multithreading". Nor sequential concurrency does.

[xXneorlinXx]

Thanks for the awesome video!
I have a question though, sorry of it is dumb.

On 7:28 you say "The code that we written doesn't need an interaction with inter[preter", I don't really understand this.
How it doesn't need an interpreter if it used to execude code?
Maybe you can explain what you mean, please? Or, at least, point me to a place to read more about it?

Thanks again for the content!

[samanmahdiabadi]

subbed now . I just started learning Python 3 days ago. had no background of Computer programming at all but found the Python for dummies book really helpful. Suddenly I found your channel. I love your job. Please let guide me where to start as a total noob and beginner. thanks so much.

[allyourcode]

The main thesis is pretty wrong. Nobody cares if Python threads are implemented using kernel threads or not. What ACTUALLY matters is whether you can get SIMULTANEOUS EXECUTION, because that's what's going to let you use all of the many cores that are usually available. If you can't do that easily (e.g. you need to resort to multiple processes, or some third-party facility), then Python is not amenable to a VERY WIDE range of applications. This non-amenableness explains something said near the beginning: When was the last time you saw a CPU-bound Python program? The reason this doesn't exist is because when people set out to write a CPU-intensive application, they immediately throw out Python as a possible implementation language.

Numba looks pretty interesting, and was new to me, so I did get something out of this, but the fact that you have to explicitly go out of your way to release the GIL in special sections of code proves the general wisdom correct: When using CPython, you have to jump through extra hoops to use all of your many cores, because in its "native" state, Python doesn't let you do that.