MultiProcessing in Python | Creating Multiple Processes in Python | Python MultiThreading

In todays world with the availability of multiple cores, it's highly likely that we end up creating multithreaded applications more often than not for our day to day programming tasks. Almost all programming languages support creating and multithread applications and more often than not we end up creating more kernel level threads than user level threads.

Welcome to this video series on multithreading in python programming and in this video we're talking about Multi Processing in python programming languages. You might be wondering why I am talking about multiprocessing in the video series of python multithreading. Well, its because of the presence of Python GIL - The Global Interpreter Lock.

The Python GIL prevents multiple python threads to execute at a single point of time and this creates as serious problem for CPU Intensive jobs. If you have multiple CPU Intensive jobs which can run in parelle then you can use python multiprocessing to use multiple cores of the machine

Before going ahead with python multiprocessing, please remember that creating multiple threads are heavy as compared to creating multiple threads and you must do it only for CPU intensive jobs.

In this video, I've explained how to create multiple processes in python using python multiprocessing and how we can use the same. However, as far as the syntax is concerned, its very similar to creating threads in python

I hope and believe that this video will help you in expediting your learning journey. Here are the contents of this video

Timecodes
0:00 : Python Multiprocessing
0:42 : Python Multiprocessing for CPU Intensive Jobs
1:42 : Recap Python MultiThreading
2:12 : Entry Point Function for Python Threads
3:22 : Multiprocessing in Python for CPU intensive Jobs
3:48 : python multiprocessing module
4:07 : Python Multiprocessing - Entry Point Function
5:14 : Creating and calling python process in python multiprocessing
5:30 : Python Multiprocessing in action
5:45 : Python Multiprocessing changing the process name

#python #multithreading #multiprocessing #multicore #cpu #pythonprogramming #threads #pythonmultiprocessing

About Me i.e. An Insightful Techie

Greetings and Thanks a lot for checking out my YouTube Channel. I’m Deepak K Gupta (Daksh). This channel is a medium for me to share my learnings and journey as a Techie so that you not only learn from my experience but also from my mistakes

As a techie I am closely associated with software development, so most of my content will revolve around Programming Languages, AI, Machine learning and Databases. I share things which will help you learn, remember and use these things in an effective and efficient way.

As a Techie, I also know that there is a life beyond software development and it has a profound impact on our career as well as on our well being. I also share those learnings which I feel are worth sharing with the intention that it MAY help you in taking better decisions in your life.

Last but not the least, I'm a traveler and explorer by nature. I would like to take you with me on some of the interesting journeys around the world. Believe me there is more to learn outdoors than indoors

Hope you’ll like my contents and will be part of my journey 🙏

[rahulnapa]

I have 2 procedures in the Postgres database that take 2 hours each to complete, but I want to run them parallelly without affecting any kind of db performance so that they both can finish in 2 hours and they are not dependent on each other. I have 4 cores allocated to that server. if I run multiple procedures through concurrent.futures, will it be able to complete it in 2 hours?
Which will be helpful multi threading or processing?

[pratyushkumarbiswal]

Sir, Why output is not showing in jupyter notebook ?? When I use mp.Process() my code runs but I am unable to see my output. I want to know why this problem occurs in jupyter notebook. It would be absolutely great if you could help me over that. 🙏

[ayushsingh]

Hi Sir, I have a question for you based on your comparison of multithreading and multi processing.
So, As GIL applies during multithreading, we can never use more than 1 core no matter how many threads we create and run?
Am I right?

[sudhirkaushik]

Hello, i have one case can you help me with that, i get near about 10k XML files(in each xml data is so much, many tags present ) per hour and I want to read all XML files, get data from them and also perform some mathematical calculations on some XML tag values and insert data into database and also make csv file for each xml files , can you please help that, which approach i take for this scenario, becuase of i am getting huge amount of files in per minute with large amount of data in every xml