Kafka vs. RabbitMQ vs. Messaging Middleware vs. Pulsar

Animation tools: Adobe Illustrator and After Effects.

Checkout our bestselling System Design Interview books:

ABOUT US:
Covering topics and trends in large-scale system design, from the authors of the best-selling System Design Interview series.

[MobileComputing]

Cool, some names of MQ tools were mentioned. Where is the "vs" in "Kafka vs. RabbitMQ vs. Messaging Middleware vs. Pulsar"?

[CalifornianViking]

@ByteByteGo - Thanks for another good video.
A word of caution. While message queues are impressive, they are often misapplied. I have seen message queues make systems slower and less reliable while consuming a lot of resources that could have been used to speed up other parts of the system.
It would be great if you could create a video that explains when (and when not) to use a message queue.

[chetankhandave]

Very good information, but it can much better if you explain how Stream processing works in Kafka, RabbitMQ, Pulsar. Polling and pushing in each techniques is very important to know.

[miran]

Worth mentioning is also (jetstream) nats, trivial to scale, has routing, partitioning, queueing, pub sub; it also has key value store, object store.
Imagine if kafka and redis had a baby.

[kervinpeguero]

Hi, I do not know your name, but I really enjoyed your videos and learned so much from them. In the diagram, (0:17min) I think there is a typo in the flow from passenger management to notification shouldn't be "Notify Driver" instead of "Notify Passenger" or invert the flow from the notification queue to passenger. If I'm wrong, please let me know if you have time of course. Hope you have a really nice day or night Mr. and keep going.

[jpzhao]

The animated flow charts and diagrams are very impressive, which tool did you use to create those?
BTW all the videos you created are super high quality in both content and presentation, great job!

[NielsBasjes]

Love your videos. You mentioned exactly once semantics as a feature of Kafka. I have found that another game changer is the capability of building systems that also maintain a guarantee of message order per key (usually a session id) which makes all of it (often in combination with Apache Flink) suitable for a whole new class of applications.

[fadthegreat]

wow, within 4min we already get and understand the high level of each message queueing from old to latest. Thank you

[madhujai_shiksha]

Its good know abt latest tech trends and tools but challenging part to learn where to use them in overall system. Breaking a system into different apps is most challenging thing in real world which can only be learn by experience

[szsz]

This is great information! Thanks for the video. Can you do a video explaining and reviewing some of the top Platforms as a Service options? Eg. Heroku, Render, Digital Ocean, Vercel, Fly, etc. When to use them and which ones are better for which job? Thank you.

[ManticoreRoko]

Why are you forget Redpanda(and Redpanda ecosystem)?? Redpanda is most powerful streaming solution today working C++ and GO under hood + Kafka Api. Please talk about Redpanda too

[hentie]

What is XXX would be better title. There is no 'vs.' in the whole video. If I just throw this video's content to my tech design spec as explanation of why I picked Kafka over RabbitMQ or another way around, I will probably have to look for another job again.

When we talk about 'vs.' we should go from advantages, disadvantages, use case (this was mentioned). No advantage or disadvantage for one over another has been mentioned throughout the video.

[ament]

Each diagram seems so informative
Thank you

[artemvasilyev]

what service do you use to draw these animations?

[JonathanDieu]

Thank you for continuing to create such helpful resources! 🐐

[maverickmaverick]

If we buy ByteByteGo subscription will you share your powerpoint clip arts ?

[jopa]

While Pulsar is evolution of a message queue in one way, it lacks exactly once delivery policy, which is Kafka is famous for the most

[sanapramesh]

Should the arrow from driver management to notification be labeled as "NOTIFY DRIVER"?

[graymcd]

Don't forget about Microsoft Message Queue (MSMQ), it's still there.

[Flako-dd]

I looked into Pulsar recently, looks really powerful. Anybody got real-world experience already?