The Python Global Interpreter Lock - Explained

To learn programming and Python - check out Datacamp!

Today, I'm revealing the worst feature Python has... The GIL (Global Interpreter Lock)! We'll be going over what the GIL is, how it compares to traditional programs, and why it's one of the worst features.

🎬 Timestamps⏱️

00:00 | What is The GIL
00:18 | How Traditional Programs Work
01:44 | The Problem With Python
02:37 | Why Use Multiple Threads in Python
04:10 | Multi-Processing

◼️◼️◼️◼️◼️◼️◼️◼️◼️◼️◼️

◼️◼️◼️◼️◼️◼️◼️◼️◼️◼️◼️

⭐️ Tags ⭐️
- Tech With Tim
- The GIL
- Python

⭐️ Hashtags ⭐️
#programming #python #coding

[TechWithTim]

To learn programming and Python - check out Datacamp!

[bereck]

For the people who are asking the question, why was GIL introduced in the first place or what advantage does it have is that the primary purpose of the GIL is to simplify the memory management and ensure thread safety in python ( cpython ). It is a lock that allows only one thread to execute Python bytecode at a time, even on multi-core systems. This means that even if you have multiple threads in a Python program, they can't truly execute in parallel and take full advantage of multiple CPU cores.

The GIL was introduced in python to simplify the memory management of Python objects. The memory management in python is implemented using reference counting, where each object keeps a count of how many references are pointing to it. When the reference count of an object reaches zero, it is deallocated. The GIL ensures that reference counting is thread-safe by allowing only one thread to modify the reference count at a time.

While the GIL simplifies memory management and provides thread safety, it also has some drawbacks. Because only one thread can execute Python bytecode at a time, it can limit the performance of multi-threaded Python programs, especially those that involve CPU-bound tasks. However, the GIL is not always a limiting factor for performance, as it can still be beneficial in scenarios where the program is I/O bound or when using extension modules written in languages like C/C++ that release the GIL during their operations.

Although you cannot enable / disable or remove GIL, it is a part of python itself, although there is a project that exist where a fork of cpython is being developed without GIL.

[raymondforbes]

I really think if you are going to make a video on the GIL it is worth exploring why it exists and how important it was for the adoption of Python. I would also point out that using IO bound applications as an example isn't a great one. It is very rare for IO issues to be CPU bound. Async and event loops work perfectly well for IO bound applications.

[SivaranjanGoswami]

I have extensively used both multi-threading and multi-processing on my codes. The thumb rule is to go for multi-processing if a process needs to perform some processor-intensive task. If it just needs to wait for a response from an API, database, or some I/O device, use multi-threading. GIL doesn't cause problems if you know how to structure your code efficiently. Python is one of the easiest languages to learn, but these are some advanced concepts of Python that needs a thorough understanding of the concepts.

[wealox]

2:05 C++, Go and Rust are not interpreted languages.

[loadeddice]

Hi Tim, could you make more videos on this topic? Maybe going further into detail with multi-processing and how to handle the separation of tasks and how to recombine the output?

[MrDowntemp]

Eehh... this video isn't very informative. (WHY is there a GIL? Why don't other languages use it? What are the benefits of having a GIL? etc.) In addition some of the simplifying goes so far as to spread misinformation. Why would you use non interpreted languages when explaining how other languages can use the same interpreter at the same time? I know there are special cases for C++, but the vast majority of the time it's compiled, not interpreted.

[jaimeduncan]

Very good explanation, some details:
1. All modern CPUs are superscalar, which means that you can, and should perform multiple operations per thread. Modern compilers will attempt to help you out, for example by unrolling a loop for an arithmetic loop.
2. The summation is not sped up by a factor of 4, it was a perfect example to explain why. The speed-up is not that big because you have to add the segments, two times.

[MedartN]

"The Guy" form Disturbed(David Draiman best vocalist our generation period!) covers make me smile, interesting way that You use him like that :) Nice explanation Tim, thank You for Your work!

[WaldoTheWombat]

Great video Tim, thank you!

Perhaps you could make another video explaining why the GIL even exists in the first place.

[aflous]

Great job on explaining the Global Interpreter Lock (GIL) in Python, but I don't agree with the final conclusion regarding multiprocessing being not as easy to handle as multithreading. Writing safe concurrent code is currently a really hard task. David Baron once said, "You Must be This Tall to Write Multi-Threaded Code, " pointing at a post-it note placed at an 8-foot height in Mozilla’s San Francisco office.

[eltreum]

Ah my old arch nemesis and frenemy the GIL. When I was new to it, I hated it and didn't understand why make an OOP language with all this flexibility then constrain it so heavy handed. The GIL is that best friend that will always call you out on your BS and stop you from doing dumb things. If you are running afoul of the GIL, you are either going about solving your problem in the worst way possible, or using Python for the wrong job and something else should be used instead. Working with/around the GIL can be an interesting exercise to deconstruct your problem into proper units of atomic work. You have to go out of your way and know a lot to write thread/memory unsafe code in Py typically.

[lowkeygaming]

When I was a newbie I use gunicorn to support multiprocessing, now I'm also using celery to support async process and multiprocessing in my project. It's additional complexity to setup but still worth the effort.

[Saadullahkhan]

SImple in GIL:
Multi-Threading(Concurrent): Not really parallel because here is multiple threads with one interpreter
Multi-Processing(Parallel): Spawn multiple interpreter for each thread

Parallel: Parallelism refers to the simultaneous execution of multiple tasks or processes, where each task is broken down into smaller sub-tasks that can be executed independently and simultaneously on multiple processors or CPU cores. In parallel execution, tasks run at the same time, truly concurrently, and can potentially speed up the overall execution time. Parallelism requires hardware support for multiple processing units.

Concurrent: Concurrency, on the other hand, refers to the ability of a system to handle multiple tasks or processes at the same time, even if they are not executing simultaneously. In a concurrent system, tasks may be started, run for a while, paused, and then resumed later, interleaving their execution. Concurrency doesn't necessarily require multiple processing units; it can be achieved through techniques like time-sharing or interleaved execution on a single processor.

Because python is simplest language GIL ensure that a beginner mistake not become a problem by limiting threads while multi-processing is more complicated to multi-threading so python give to advanced user and create a awesome balance. If you want to enjoy power of multiple works at a time but not to make it complicated use multi-threading and when actually want parallel execution use multi-procession

[leamon]

Thanks Tim. Could you also make a video about the difference between coroutine and thread? Thank you

[MakeDataUseful]

I’m kind of a fan of the GIL, it’s a level of complexity removed for new Python people

[Djellowman]

I knew all of this, but wish you went into WHY python implements a GIL.

[b]

Nice video, very informative! The style was perfect for this video, great demonstrations.

[reasonforge]

You seem to be leaving out a huge piece here: async and its predecessors green threads. They have been the typical way get around waiting for non-CPU intensive processes.

[becauseiwanttoanime]

I absolutely love the new video style! its focused to the point short and give all the information we need!
Great video! Thanks tim!