Python 3 Programming Tutorial - Threading module

In this Python 3 programming tutorial video, we cover the threading module that is a part of the standard library. With this, you can engage in threading, which allows multiple processes to take place at a time.

Bitcoin donations: 1GV7srgR4NJx4vrk7avCmmVQQrqmv87ty6

[RealMcDudu]

I think this video is a bit confusing because it mixes 2 different topics: "threading", and "queue".

As I understand, queue is not so relevant here, since it's only used to number the different threads, and kind of "indexing" them.

The important things to take out of this video (threading-wise) is, IMO (disclaimer: I'm not an expert), this:

# t =
# t.start()
this creates different threads and tell them to execute a command called "threader" (confusing name, but he could have called it "banana"), and puts it in an object called t. But first, you have to start the thread by using the <thread object>.start() command

Since this runs on a loop 10 times - it means 10 different threads are created and are executing "threader"/"banana" - command (and this is the important part) simultaneously! (!!!) Meaning, the 2nd loop thread does not wait until the first loop thread finishes before it executes - it runs off to do the command side by side.

# t.daemon = True
is defining the thread as a daemon, meaning it is not the main thread. I don't see the significance of that in this program, and maybe you can take it off.

Now what does the threader command/function do?
First - it gives the thread an index from 0-19, and then it tells him to do exampleJob, which is just to wait 1/2 a second, and print the thread name, and the index.

# print_lock = threading.Lock()
# with print_lock: ... print...
are used to lock the other threads from stepping on each other while printing. Meaning - if thread #9 is now printing, all the other threads that currently arrived at this command have to wait until it finishes.

So you will get printed a random arrangement of the 10 different threads names, and the "index" they were executing. Each thread will get 2 "indexes", resulting in 20 instances of the printed command.

[[[ This (above) is the important code for threading. Down below is using the queue code ]]]

Why 2? Why 20?

# for worker in range(20):
# q.put(worker)
the reason it will happen 20 times, is because there are 20 "indexes" being sent to the threading/banana command, and-

# worker = q.get()
# exampleJob(worker)
- the ten threads created are reaching here basically at the same time, are assigned an "index", and execute exampleJob with that index (here "worker" is actually a new local variable, that gets the other "worker" variable out of the main code, again a bit confusing, but he could have called it a different name). This is looped until-

# q.task_done()
# q.join()
-all the "indexes" are "returned".

Then the program continues to its last command (print...) and finishes. Since there are 20 "indexes", there will be 20 times exampleJob will be executed, but since there are 10 threads, they will do simultaneously the first 10 exampleJob, come back and do the second 10 exampleJob (this is why, "index"-wise, the first 10 indexes will always be 0-9, and the last 10 indexes will always be 10-19).

Since every time the 10 threads execute exampleJob they have to wait 1/2 a second, and they do this twice, the whole thing takes around 1 second, in total.

[mdzubayerrahman]

This is "The Most Complex Video on Threading" I've ever seen!

[pengli]

I renamed the variables to let everyone understand the code better.
====
import threading
import time
from queue import Queue

print_lock = threading.Lock()

def worker(job):
time.sleep(0.5)
with print_lock:
print(threading.current_thread().name, 'Task'+str(job))

def worker_thread():
while True:
job=q.get()
worker(job)
q.task_done()

q = Queue()

num_jobs=10
num_workers=20

for wk in range(num_workers):
t = threading.Thread(target = worker_thread)
t.name='Worker-'+str(wk)
t.daemon =True
t.start()

start=time.time()

for job in range(num_jobs):
q.put(job)

q.join()

print('Entire job took:', time.time()-start)
===

When num_jobs=10, num_workers=20, the code outputs
>>>
Worker-2 Task1
Worker-7 Task7
Worker-3 Task3
Worker-1 Task0
Worker-5 Task4
Worker-9 Task8
Worker-4 Task5
Worker-0 Task2
Worker-6 Task6
Worker-8 Task9
Entire job took: 0.5330305099487305

When num_jobs=20, num_workers=10, we get
>>>
Worker-4 Task2
Worker-8 Task8
Worker-6 Task5
Worker-5 Task6
Worker-7 Task7
Worker-1 Task3
Worker-9 Task9
Worker-0 Task0
Worker-2 Task4
Worker-3 Task1
Worker-4 Task10
Worker-8 Task11
Worker-6 Task12
Worker-5 Task13
Worker-7 Task14
Worker-1 Task15
Worker-9 Task16
Worker-0 Task17
Worker-2 Task18
Worker-3 Task19
Entire job took: 1.0650608539581299

Make sense?

[nsuryhr]

@sentdex i was joking my teacher learn from you alot of and i got a job cause your php love you brother

[sagarbhamare]

How some one can dislike this video, he is sharing knowledge it's up to listener to get it or leave it.

[adrianmxmx]

Thank you for your time. I've used this awhile ago to multi-thread many ssh connections to routers, which has saved a ton of time!!

[bbmanpwns]

This is definitely not a beginner video, but this is the only video so far that has actually mentioned why queues are important (issue I was running into). All the other videos section queues off from threading and make it extra confusing.

[jspatterson]

I did some itterative testing and found that the best possible performance I could get is with making the count of Workers (threads) the same as the count of Jobs. I tested up to 20, 000. With 20, 000 jobs and 10 threads it took my laptop 1018.36 seconds. With 20, 000 jobs and 20, 000 threads it took 2.81 seconds. That 2.81 dropped to .89 when I commented out the Printing section. I am running an AMD Ryzen 9 5900HS CPU.

[lucajaich]

omg this guy has a tutorial for everything, i love him

[GHOST-qxwi]

i know this is before 8 years but this guy better explain then whole semestre

[markd]

Good video of complex topic for beginner, but also good explanation by Refaeli below. This code makes more sense to me to think of 'worker' as workjobs needing to be done (as there are 20 of these jobs), and 't' as threads (or actual workers, 10 of these, who can do the jobs which require to be done).
Interestingly, running and then re-running this code in jupyter does not reset my 'thread' ID numbers to 0-9, but increments range by 10 on each re-run (0-9, then, 10-19, then 20-29...)
Great series Sentdex - thanks

[DanielLewisDance]

For people who are confused, I feel some of the confusion also comes from how he has named his threads / workers.

May help if you think of SentDex's 'workers' as 'jobs', and his 'threads' as 'workers'.

Makes more sense to me that way!

[bilelmalek]

to be honest i never understood this thread on py until i see your video thanks

[aboutmount]

Another example of a thread is a browser. It runs one thread and its processes in one tab for one processed thread with many queue's. When you open another tab it begins another active thread with its processes also in queue. And at times one thread process may need to work with another process. To do this we have to use a Queue Lock in that thread to grab a variable from another thread. Example of these are passing a variable or objects from one too the other behind the scenes. Example of this is when t5he main browser program on one thread the user may close one tab then close the second. Means both of those tab threads of web-pages are linked to the main browser program and can be killed by it through using variables Queue locked in their respective threads to do so.

[MrFedX]

Great tutorial! I´m glad you mention concurrency issues. I guess that´s what´s causing most headaches. Now I´ll try to do some threading and the get into functional programming... :)

[pinkalpatel]

can you explain threading with socket running two different file client.py and server.py in one machine ?

[haydensmith]

Hey sentdex, I have this problem with parallel programming, I am trying to make a raspberry pi light up while being able to control the thread at any time. I'm so close and this video helped a lot.

[yuqingwang]

I think doing time.sleep() isn't quite good for performance illustration of threading. Because Python (Cpython) has GIL which will be released when time.sleep() is called, so the sleep work can be done concurrently and the performance is surely better than running one thread. However, in most job cases, the operations in exampleJob() will not release GIL, which means only one thread can have the GIL and running at one time. So the performance won't be improved much (or maybe decreased) when using multi-threading. Just replace 'time.sleep(.5)' with 'a = sum([x for _ in and have a comparison between the run time of multi-threading and one-threading.

[khushbuparakh]

this is really helpful for me even after an year of this video

[aboutmount]

He is explaining how it works basically. And it is much simpler than how he is trying to explain it. Better would be a Loop while running a Tkinter process. The program runs and holds its visible objects in a Queue or in a lock on a time in the loop. Because the loop starts at the top and works down from top to bottom at 32 to 64 million iterations per second repeating itself from top to bottom of the program. Everything could not be viewed at that speed without a lock/queue on a time in it;s Thread. And when a function or process needs to work another thread outside of itself is essentially a separate loop and interacting with the first loop. Example is when Tkinter uses Top Window Widgets. These run in a separate Thread for each new window, these also holding on a set Queue/time. Images in these Queue cant hold themselves in Queue without re-assigning their variable to itself in the Thread. image=image.