Kafka vs. RabbitMQ - who wins and why? | Systems Design Interview 0 to 1 with Ex-Google SWE

I'm keeping the moustache sorry not sorry.

Stay posted for this Saturday, ultimate video incoming.

[VijayJain]

The key thing to mention is that If sensor metrics are spread across multiple partitions, they can still be read out of order, so in-order is only guaranteed within the partition.

[Mtlstorm]

RabbitMQ has durable queues, where the mesages are written to disk, they can also be replicated to other nodes. RabbitMQ also has "Streams" which are queue-like in that you can pop and read messages in order, but the messages are not deleted once read, meaning another consumer can start from the beginning (or any point in the stream) and read the messages again.

[rhodyborn]

Throwing together a design doc and this video was quite helpful, thanks. Also, P!

[Mactwinza]

I'm interviewing several SWE for a position in my company and this video helps with preparing questions. Thanks!I am interviewing multiple software engineers for a position at my company, and this video is helpful for preparing interview questions. Thank you!

[comingfall]

You helped me so much with this video, your explanations are clear and made me really understand the core of it

[dariashevchenko]

just started the video and already heard 2 jokes! well done, this is IT I am looking for!)

[tienat]

I really enjoyed this video, it clear the burden about the different between them. Can you make another video about how RabbitMQ and Kafka solve there's cons like those in your conclusion? i really appreciate it.

[indraneelghosh]

This was a really helpful overview of the messaging systems. Can you do a video that goes more into the implementation details of the message brokers you discussed? For instance, answer to questions like how and under what conditions does Kafka guarantee exactly once message delivery and what sort of guarantees these message brokers give in case of failures etc.

[quirkyquester]

amazing topics, and video been thinking about learning more about rabbit mq and kafka for awhile. wanted to know the diff. this answers my question in mind.

[misamee]

Interesting video.
As for the advantages of the log based systems, the metrics example seems a bit of a stretch since one could use a memory based system and include a precise timestamp as part of the message's payload.
The second example is more appropriate.

[kpboy]

Low latency backbone used by trading firms beat these two by wide margins.

I guess these are designed for high throughput whereas LLBs are designed for low latency.

[Dozer]

It's common practice with memory based brokers to have deadletter queues. If a message fails to process N times, it fails over to a 2nd queue, that is a "failed messages" queue. These can be re-driven at any point. Kind of mitigates some of the downside you were talking about with them.

[devops_junkie]

Ah, this is amazing, I have soome junior developers that I am training on MS Architeture we are now at the integration point and wanted to see which is better from our case. It seems we might be using both options for us. Thanks

[vadimc]

RabbitMQ has a different storage options that are log-based. Calling it an "in memory" doesn't sound right. Not sure about others in a list but it could be the same case with them.

[sourcerer]

6:20 For log-based message broker as set up in the video, both consumers read from m1 to m4? So it's not a job dispatch where consumers share the workload?

[beecal]

Why having separate message queue for each consumer, would reduce throughput for in-memory MQ 4:00, but increase throughput for log-based MQ 6:22 ?

[Bruh]

Great videos + How someone so young can have so much knowledge .. but bro why do you have named it -> Systems Design Interview 0 to 1 ? Systems Design 0 to 1 could have been an apt name .. don't you think? its more useful than just for interviews..

[rodfrazier]

For a visual learner like me, I understand but the way you scribbled all over the board. It can throw people off. Just a bit of advice maybe you can take some goodness from. Great video though man!

[tungthanh]

I think SQS must be a log-based message broker, it allows replay, and also has message retention, 1 message by 1 handler. Could u explain why it is considered an in-memory broker?

[david]

Wait I think RabbitMQ has some failover strategies? I think they also have quorum queues and classic queues where quorum queues actually have a persist log on disk? Not 100% sure about what I am saying though, maybe a bit too specific to RMQ. someone correct me if I am wrong.