How to Run Multiple Functions Simultaneously in Python3 Using Threading

Learn how to effectively run multiple functions concurrently in Python3. Discover why `Threading` is often the best approach over `multiprocessing` due to the Global Interpreter Lock (GIL).
---

Visit these links for original content and any more details, such as alternate solutions, latest updates/developments on topic, comments, revision history etc. For example, the original title of the Question was: How to run two or more functions at the same time in Python3?

If anything seems off to you, please feel free to write me at vlogize [AT] gmail [DOT] com.
---
Running Multiple Functions Simultaneously in Python3

If you've ever tried to run multiple functions at the same time in Python3, you may have encountered some challenges. Particularly, if you're dealing with functions that are resource-intensive, you might want them to execute concurrently to save time. A common question that arises is how to execute two or more functions simultaneously without having the processes run serially. In this guide, we’ll explore a practical solution using threading, which is often a better choice compared to multiprocessing, especially due to the Global Interpreter Lock (GIL).

The Problem

The user in our example wanted to train a model while simultaneously monitoring system resource usage. However, they encountered issues when attempting to use the Process class from the multiprocessing module, which led their functions to run one after the other instead of concurrently. They also faced errors when trying to retrieve return values from the processes. If you're facing similar challenges, read on for an effective solution.

The Solution: Using Threading

After experimenting with multiprocessing, the user found that switching to Threading provided the solution they needed. Here’s why:

GIL (Global Interpreter Lock): Python's GIL allows only one thread to execute at a time. This can lead to complications when trying to execute heavy tasks concurrently using multiple processes.

Lightweight: Threads are lightweight compared to processes. They share the same memory space, which makes data sharing between them simpler and causes less overhead.

Step by Step Code Explanation

Importing the Required Module: Begin by importing the Thread class from the threading module.

[[See Video to Reveal this Text or Code Snippet]]

Define Your Functions: Make sure your functions are ready to be called. In the user’s case, they had a function to train a model and another to check system resources.

Creating Threads: Create two thread instances, one for each function.

[[See Video to Reveal this Text or Code Snippet]]

Starting the Threads: Use the start() method to begin executing both functions at the same time.

[[See Video to Reveal this Text or Code Snippet]]

Joining the Threads: Lastly, ensure both threads complete before your program terminates. This is achieved with join(), which waits for the threads to finish running.

[[See Video to Reveal this Text or Code Snippet]]

Final Revised Code Example

Here’s how the complete code looks with threading implemented:

[[See Video to Reveal this Text or Code Snippet]]

Conclusion

In Python3, running multiple functions at the same time can effectively be accomplished by using the Threading module instead of multiprocessing. This approach not only simplifies your code but also enhances its efficiency by allowing concurrent execution of tasks without the slope of process overhead.

By adapting to threading, you ensured that both the model training and the system usage checks ran as intended, significantly improving your application's performance. Now, you can harness the power of concurrent execution in your Python programs!