Java Concurrency Interview Question: Multi-threaded Message Queue like Kafka, SQS, RabbitMQ

In this video, we are going to create multi-threading based message queue. This message queue acts as a broker between publishers and subscribers. Publishers publish message in a topic in the queue and the subscribers who are subscribed to that topic will receive it. We need to take care of multi-threading in this problem since requirement is that multiple subscribers should receive the message in parallel.
This is one of the most popular interview questions which is asked by many companies.

Thanks for watching. Hope you find this video helpful. Let me know your feedback in the comments.

Do subscribe to the channel!!

You can follow me on:

#system #design #interviews #interview #coding #programming #faang #tech #technology #developer #coder #code #java #systemdesign #architecture #software #sde #sdi #systemdesigninterview #software #development #program #lowleveldesign #lld #oops #designpatterns #designprinciples

[sagartyagi]

Have an uber sde2 interview in few days, can't even imagine of solving this in an interview.

[vishnusingh]

Amazing content! This is what differentiates you man. Pressed like before even watching it, and not disappointed at all. Keep up the great work. More hands-on, work related industry best practices videos please. More power to you!

[vaibhav]

Fantastic breakdown of LLD! Just came across your LLD videos, and I must say, your explanations and implementations are top-notch—haven't encountered such clarity and detail before. Looking forward to more LLD content from you! Cheers! :)

[kanishkverma]

Thanks for making such awesome and easy to follow videos . One suggestion is if you could add a class diagram also then it would help to understand flow in between but otherwise thanks for the efforts :)

[anujpratapsinghyadav]

Really Great Content for LLD! Subscribed before watching :)

[ayushupadhyaya]

Overall the code and explaination I found is good, but what I found missing was the overal explanation on the UML relationship between different classes. Why a particular class is having a "is-a" and "has-a" relationship. What is the underlying low level design decison that was made due to which we arrived a certain class structure. If explanation regarding this is added, then I beleive this would be a great video for multi threaded message queue.

[alphaninja]

Please add a Class Diagram also here. It will be easier to follow with different mappings also.

[amanpandey]

Hi Udit! Awesome video man, this was long awaited. One question though:
1. at 18:27, can you explain the reason why in SubscriberWorker wakeUpIfNeeded() method is made synchronized? In what case will this method be accessed by multiple threads?
Follow up question: Why does it have synchronized block then? I mean, why both synchronized block and method level synchronization is needed here?

[snehasishroy]

Hey Udit, Great video. Couple of questions.
1. Ideally shouldn't you return a Future/CompletableFuture from the publish() method so that the client can choose to wait on the future until all the messages have been published?

2. In subscribeWorker, why does the while loop have >= condition? Shouldn't it be just curOffset == topic.getMessages().size()
Because curOffset can never exceed the no of messages in a topic as currently you don't support deletion of messages in the topic.

Also I think you should have highlighted the importance of using while loop for wait() check, most people tend to forget putting wait() check in a while loop.
Nice explanation of compareAndSet though :)

[yuganderkrishansingh]

Thanks for the video. Learned a hell lot of things from your videos man.. Keep up with these..A million Thanks

[greenmug]

God bless you, for creating this channel

[lokeshratusaria]

Hey udit ! Great video man
One Doubt:
If i am not wrong you are creating a new thread every time you are publishing a message in the topic
new Thread(() ->

so multiple threads are publishing a message in the topic at the same time but your subscriberWorkers Map is not synchronized that means multiple threads can see that a worker is not created for a subscriber and end up corrupting subscriberWorkers Map

[kritikasomani]

Hi, thanks for the video. However, I have a doubt. The run() method in SubscriberWorker has a synchronized block and all the functioning happens in that. Now synchronized means that only one thread can have access to that block at any point. Then why do we need AtomicInteger to deal with offsets? Won't a normal Integer value be sufficient since we've wrapped this code inside synchronized block?

[codetech]

Great great content. Please keep posting machine coding questions.

[mujahidpasha]

Amazing content just what I wanted, one request it would be great if you code along while explaining even if video goes long that would really help us understand the thought process as well. Thank you very much for the efforts. Keep up🙏

[priyankamishra]

once again great quality content!! Please continue this series on multithreading scenarios.

[Subudhdh]

Seems very little synchronization was needed, how are messages synchronized between producer and consumer, can you help me understand how many different threads are getting created here; and how producer and consumer are ensuring the message is used only by one of them at any given time

[sudhiryadav]

Hi Udit, first of all thank you so much for creating such a meaningful content. But I have doubt when you are using compareandset method in order to increase the offset value but if you are using the synchronized block then there would be only one thread at a time to change the offset right? So do we stillneed to use the compareandset mehtod or can we simply increase the value of offset

[developerabhishek]

Your content is very unique. Thanks for sharing the knowledge

[akashgoyal]

Instead of taking lock on the topic during write can we use mutex so that our reads don’t need to be synchronised since we are supporting only adding messages in the topic not updating it