When to Use Kafka or RabbitMQ | System Design

Lots of engineers mistake Kafka and RabbitMQ for interchangeable systems, but they serve very different purposes. This video goes over the fundamental differences between these two systems and when you should use one over the other.

Table of Contents:
0:00 - Introduction
1:29 - Consumer Patterns
2:55 - Message Routing
5:24 - Acknowledgements
6:43 - Use Cases

Socials:

[pratikshitsingh]

Awesome explanation on the difference of use cases. I've worked with kakfa but kind of new to RabbitMQ and this video helps me getting the brief insights on these. Thanks

[Fikusiklol]

Both systems are designed to decouple.
Both systems have routing/filtering/processing capabilities, that either built in or part of their eco system.

Main difference is that kafka is designed for events, rabbit is for commands.
And ordering/persistence associated with those.

[atulshah]

Great breakdown of the queues and side-by-side comparison. And I can’t believe I’m the first one to comment here!

[muhammedashkar]

Nice content. I was doing a microservice project where I had chosen Kafka over RabbitMQ without actual knowledge of the use of kafka. I needed the traditional RabbitMQ behaviour for my current microservice project, and I thought Kafka would be better for that as I heard it had high throughput. now I know the why's. gonna go with rabbitMQ.

[debangshubanerjee]

Excellent video, one request can u compare Solace Queue also with Rbbit and kafka?

[augustinemunene]

would you kindly do how to make a backend as a service system design

[whoknows]

Kafka can be used as queue tho, sharing the same consumir group and increasing the partitions as many consumers needed to balance messages

[matveyshishov]

I really like your videos (and even bought your course because of that), but this one seems to be lower level than your usual standards.
You make it seem like the difference is mostly topological, or maybe API-wise.
But in reality the systems are very different in purpose and use cases.
RabbitMQ is an implementation of AMQP, full stop. Just go read John O'Hara's article "Toward a commodity enterprise middleware". Being an AMQP, all it does, and nothing else, is delivery of messages to the specified recepient. Again, full stop. As soon as the message is in the memory buffer - it's done its job.
Kafka, on the other hand, is a CDC on steroids. A distributed WAL. There are no messages, therefore, no addressee, there are only "data changes", which we usually call "events".
I could wax poetic for much longer, and do comparisons with this and that, but honestly, I don't see the point, as we live in the time when you can literally ask the person who created the system what they wanted to do ;) .