Why thread pools even exist? and how to implement them?

Build Your Own Redis / DNS / BitTorrent / SQLite - with CodeCrafters.

In the video, I explained the importance of thread pools in managing multiple concurrent requests efficiently without overwhelming the system. I discussed the drawbacks of spinning up new threads for each request and how thread pools help mitigate these issues by capping the maximum number of threads. I highlighted real-world use cases and provided a quick prototype implementation in Go. Tuning thread pool size based on hardware and workload characteristics was emphasized, along with practical tips for implementation. Watch the video for a deeper understanding of thread pools and their implementation.

# Recommended videos and playlists

If you liked this video, you will find the following videos and playlists helpful

# Things you will find amusing

# Other socials

I keep writing and sharing my practical experience and learnings every day, so if you resonate then follow along. I keep it no fluff.

Thank you for watching and supporting! it means a ton.

I am on a mission to bring out the best engineering stories from around the world and make you all fall in
love with engineering. If you resonate with this then follow along, I always keep it no-fluff.

[LeoLeo-nxgi]

A very IMP question I had.
1. So we know that usually when the run() method of a thread is executed, the thread dies automatically after it's work is done.
2. Then why is it said that we put the same thread back to the pool or how does ThreadPool does that (i.e reuse the same thread), I think we are again creating a new thread right? and putting that back to the pool? and not reusing the original thread (which did the earlier task).

Would be great if you could guide here. Thanks for the knowledge share !!

[AsliEngineering]

What all topics do you want me to cover? Do add them as a reply to this comment ⚡

[shawkishor]

In one of the faangs, there was an interview question: one method (class method/REST call etc) is being called by 1 million users, all users get the response at no delay. What is happening behind the scene?how come that method is getting executed million times concurrently.

[reudrax]

This concept is same as connection pooling. saving the time to make connections. putting a limit on numbers of concurrent db connections that can be made, thus handling the load effectively

[keerthi]

Another main reason for thread pool is "creating new thread and deleting then itself is very costly in terms of CPU".
Also what is the max thread for a n-core cpu ?

[burionyt]

Hey awesome video but here is a tip on "thread pool in Golang".

In go thread pool is an anti-pattern and you should generally use semaphores instead. One can implement a basic semaphore using a buffered channel. There is a weighted semaphore implementation in the golang/x pkg if you wanna checkout or could make one yourself(I generally do this)

[_sudipidus_]

just a nitpick: you demonstrated with an example of go routines which are not actually threads (even using just go routines without keeping a worker pool would perform better than threads)

but on a positive side, this was more about generic pooling concept (pooling of threads, HTTP connections, db connections or anything whose creation/bookkeeping is expensive)

[jakubfrei]

Best explanation i came across, thank you

[jspnser]

For the topics in future: I wish you could explain internals of go routines more in depth, (why they're light weight etc), thank you for the video

[hiteshbitscs]

You mentioned for network bound workload we can have more threads, for CPU heavy workload we have few threads... Isn't it should be opposite?

1. For network I/O workload because you wait more on network packet to send/arrive due to it's millisecond latency.. we can have few threads. so that thread has works rather than doing short stuff and wait
2. For CPU heavy workload : because you have more works we have more threads.. (Here I guess in hyper threading each core has 2 threads per core.. hence max we can go is approx/. 2 *number of cores) at max...

[HarshitSingh-jwo]

Have you pushed your code on github, if yes, could you share the repository link?

[joelphilip]

if thread pool is 5 and no of tasks > 5, doesn't it execute parallelly (not concurrently) ?

[BhanuReddyputtareddy]

What is the font you have used in vscode looks nice

[SlimShady-op]

Hi Arpit thank you for the knowledge, just one question here. The implementation mentioned in the video is through go routines. In this case what does the thread pool signifies in terms of operating system threads, more briefly If i create a thread pool of size 5 (similar to mentioned in the video) will it reserve 5 go routines or 5 platform threads ? As afaik we can have more than one go routine running on one platform thread, due to the fact that go scheduler itself takes care of that. And a follow up on this is, If I have to implement this In java do I use the normal thread pool or virtual threads ? Thank you, cheers

[Ðogecoin]

Thank you for this insightful explanation Mr armpit

[sachinmalik]

Hey Arpit, I know it a old video, but if you do see this and have a seconds to spare, I have a doubt.

Why do we have a unbuffered channel, here like if we are adding jobs, to a workQueue, Should our channel have some buffer, in above code/scenario, are we blocking the putting of the job on workQueue, because it is unBuffered, so I add a job, I have to wait for some thread/go routine to pick up that job? before adding new Jobs to queue

I am new to Go, and I am loving what you have to teach, Thanks

[sharoonaustin]

Isn't it something similar to the batch processing (Correct me If I am wrong) For example: let's say you have a slice which contains 40 elements and at a time you want to process 20 of them, so you'll fetch the first 20, iterate over them and keep adding to the wait group, and perform the task using go routines and outside the loop you added wg.wait(). This way you're creating 20 threads and running them for a batch isn't it?

[damn]

For our server example there is also another way to handle client, epoll/kqueue, so my question is what is the tradeoff of using thread pool over epoll or vice versa? Specially for the I/O bound task.

[neo_otaku_gamer]

Thank you for explaining the concept on thread pool. How can we implement similar concept in python, such as server less functions on cloud. Python inherently doesn't have support for threads and thread pool due to Gil.

[MarathiNationOne]

If cpu cores are 4 and threads are 10 is it worth? Or can we have at most treads equals to no of cores