How Much FASTER Is Python 3.13 Without the GIL?

In this video, I'll discuss how Python 3.13 is revolutionizing performance by making the Global Interpreter Lock (GIL) optional! Learn what the GIL is, why it exists, and the potential impacts of its removal on your Python projects.

🎓 Courses:

👍 If you enjoyed this content, give this video a like. If you want to watch more of my upcoming videos, consider subscribing to my channel!

Social channels:

👀 Code reviewers:
- Yoriz
- Ryan Laursen
- Dale Hagglund
- Kit Hygh
- Alexander Milden
- Bean

🔖 Chapters:
0:00 Intro
0:57 What is GIL
2:12 Why the GIL is in Python
2:52 Performance increase without the GIL
4:53 GIL is becoming optional
5:34 The plan for GIL
6:44 Potential issues with removing the GIL
8:31 How much will this affect you?
9:45 Outro

#arjancodes #softwaredesign #python

DISCLAIMER - The links in this description might be affiliate links. If you purchase a product or service through one of those links, I may receive a small commission. There is no additional charge to you. Thanks for supporting my channel so I can continue to provide you with free content each week!

[spicybaguette]

We need python 3.14 to release on pi day :)

[Frank]

A small correction: The GIL does not prevent all race conditions, just those inside the interpreter code. e.g. two threads updating a dictionary won't break the internal data structure, but a computation over multiple python statements can still have context switches and thus have race conditions.

[ztowfic]

I feel like it was better to compare 3.13 with and without GIL. Not 3.12 vs 3.13. Who knows if there is a speed regression not due to GIL in 3.13.

[bn_ln]

pyenv has free-threaded (ie no gil) builds available, they're the "t" versions, 3.13.0b4t for example

[MrGb]

We should code the following versions as 3.13, 3.14, 3.141, 3.1415, 3.14159, .. until we get the GIL out, then jump to 4.0 😂

[XRay]

Check out episode 2 of the `core.py` podcast (by Lukasz and Pablo) if you are interested in the gory details of what goes into removing the GIL.
Fascinating to see how much effort was required to do this. It also gives you a better understanding of the performance hit for single-threaded programms that Arjan was observing in his benchmark. ;)

[daemotron]

Google attempted to build a GIL free version of Python (project Unladen Swallow). Today they use Go where they need massive parallel code execution. Back then I was hoping for a GIL free (and thus horizontally scaling) version of Python. Today I'm not too sure anymore it's a good idea - GIL free Python comes at a cost, and I'm by far not sure it will put Python on par with other languages focusing more on parallel execution. Therefore I tend to think that it's better to have two (or more) languages, and use them according to their core strengths.

[IMSezer]

so what we blame for slow code if they unlock gil

[cosmicaug]

1:45
Jython is sad. Their stable release is still Python 2.7 based.

[ernestterkuile]

I would suggest you also compile 3.12 with the same parameters, and run you test again. Then you would have a beter speed comparison.

[diegol_]

Python has been thought of as a language to make things "easy". Disabling the GIL greatly increases the complexity of the programs. It brings problems that many Python developers may not even know about. It is a new tool, let's hope it is used well and consciously.

[MiesvanderLippe]

I think I prefer Python being the glue, and lower level code doing the heavy lifting. Like with PolaRS.

Furthermore running into performance limitations should be a decent sign to move on to a different language / solution.

[rheumaticharm]

Hi arjan, I don't think the slower single threaded python was a problem on your side. But the side effect of the removal of the GIL itself. I don't exactly remember the specifics but there was a very famous guy who tried removing the GIL before. But later concluded that removing it is not really that difficult but the performance that it gives after the removal is quite disappointing due to how python is built. After later searching a bit I found that the reduce in performance could be due to the exact reasons that you mentioned in the beginning. The higher complexity of managing memory, garbage collection, locks and all brings overhead. Even if we are not using the threads it is still managing the memory in a very complex way which kills it's performance.

Can't say for sure but the guy who may be tried to remove it was 🤔...
Larry hastings

[danilshein]

With GIL enabled all Python's data types, including mutable ones, are guaranteed to be thread safe. Without GIL you should manage it by yourself with all that mutexes like in C++. Nobody really whant to mess with it, especially those who use Python for ML not being a professional developer.

[wdavid]

The slower multiprocess and single threaded performance in 3.13 is likely due to the GIL not just being removed but it has probably been replaced by multiple locks in the interpreter for specific things. This has been a longstanding reason not to remove the GIL because experiments in doing so have generally slowed down a lot of use cases.

Also it would have been worthwhile to attempt multiple runs in each scenario with different amount of work to separate out setup overhead from processing slowdown. I expect multi-processing to have overhead that is paid one time to get everything running but to have minimal performance losses compared to threading once executions starts, assuming shared memory is setup and an alternative approach like socket communications isn't used for multi-processing.

[filipengstrom]

I can’t imagine how to deal with arcs and mutexes etc in Python while keeping Python as simple as it is today. I assume the added complexity the true multi threading adds to Python defeats the purpose of using Python. If you’re really dependent on performance, just write it in Rust.

[humanardaki]

3 minutes into the video and you are just repeating the same thing :/

[anonymouscommentator]

so it decreased the performance in the default single threadded and the optimized parallel computation at the same time 👏

[MisterComment]

I see this mostly as something that benefits complex applications, that shouldn't have been written in Python in the first place. At my last job this would have been excellent. Of course I'd have had to take better care with the multithreading and synchronization problems but any dev should be able to figure it out.

What would have been even better would be if I'd been allowed to rewrite the application in something more suitable for a large complex system. Something compiled with static typing...

[shaikhshafeen]

I would like a video from you on the comparison of different STL libraries we got like multiprocessing, threading, concurrent. I have seen multiple stackoverflow answers for the question like "fastest way to send 100k HTTP requests?" people answer it by using different STL libraries and I am no much clear about the core difference and advantages/disadvantage. The end goal remains same but usage of different libraries for achieving the same goal confuses me.