Kafka vs RabbitMQ Performance

RabbitMQ vs Kafka Performance Comparison.
RabbitMQ (4.0.2) vs Apache Kafka (3.8.0)

▬▬▬▬▬ Experience & Location 💼 ▬▬▬▬▬
► I’m a Senior Software Engineer at Juniper Networks (12+ years of experience)
► Located in San Francisco Bay Area, CA (US citizen)

▬▬▬▬▬▬ Connect with me 👋 ▬▬▬▬▬▬

▬▬▬▬▬▬▬ Timestamps ⏰ ▬▬▬▬▬▬▬
0:00 Intro
1:22 Test Design
2:32 1st Test
6:22 2nd Test

▬▬▬▬▬▬▬ Source Code 📚 ▬▬▬▬▬▬▬

#kafka #rabbitmq #devops

[sweetcapitan]

I think a battle between NATS and Kafka would be a good continuation of this comparison.

[xtrn]

NATS vs KAFKA vs Pulsar vs AutoMQ should be a great addition to this series. This will be helpful for a lot of audience.

[jm-alan]

My company uses RabbitMQ as our communication backbone for an IOT-type deployment right now, so this is actually super interesting to see

Also, I might have a PR - it looks like you're instantiating a connection for every RabbitMQ consumer, when Rabbit generally prefers that you try to use only one connection and multiple channels to dole out multiple logical connections to the broker. There might be performance to gain/some wasted CPU resources there, depending on how many individual consumers you're actually constructing

[supermamoru]

Hello, just come through a few videos of you and I LOVE them immediately. Seeing moving charts and benchmark stuff made me happy

[Demodude]

Redpanda vs kafka would be an interesting test

[Qrzychu]

As usual, great video. I never expected you to add a face cam, but it gives your videos a bit more personality, good job!

[longshin]

Why you test Rabbit MQ with option keep msg memory but Kafka written to disk? That's is not fair. Can you test RabbitMQ ( Amazon MQ in AWS) with config mode lazy( written to disk).

[artursradionovs]

Thanks for a video.
Can we also check Kafka vs Redpanda?

[joswayski]

I don’t know about rabbit MQ streams, but the catch with things like Redis streams and even NATS is that you can process things out of order on the same partition or message subject if you have multiple consumers which makes it a non starter for a lot of projects

As always, thanks for making these videos :)

[unom]

Nice, please try NATS
Would love to see IBM message queue as well

[yunocode]

Thank you for the videos, very useful
Would love to see some kind of websocket benching for number of connections and throughput with go, rust, js, erlang/elixir

[TweakMDS]

Love this, I would also love to see NATS thrown into the mix.

[wety]

Would love to see comparison of kafka against NATS and/or redpanda

[ucretsiztakipci]

Great job. My deducation after video is this, Kafka is too big to start but perfect for the large scale in case of long term.

[uwontlikeit]

Yes!🎉 Glad you got your talented hands on queues finally :) great job. Can't wait to see benchmarks of Apache Pulsar, NATS and RedPanda

[krissukoco]

This channel keeps getting better and better! Kinda my breakfast companion at this time.
A bit curious, do you have Indonesian or south east asia parents? Due to the "Putra" last name.

[ttrel]

It would also be excellent to see a comparison when using production best practices, being 3 replications and min isr of 2. Not sure if rabbit mq streams has an analogue but it would be really interesting if so

[mitchellmnr]

So, I do feel your comparison and wording is a bit misleading.
Since both can be clustered
Its not a true apples to apples comparison.
Now although we do know Kafka can push more due to the way it is designed.
Rabbit can handle 50k msgs per second on a single node - ive tested and seen that.
But it also depends on the node specs.
However when you cluster, which is what most people would do (HA, reliability and scalability) - then we can see a really good test.
Although I do get having the single nodes - but that should really be said.
Since its not rabbit vs kafka - its a single rabbit and kafka node vs eachother - since both are designed to be clustered

[PrestonThorpe-dx]

I have to +1 asking for the NATS v. kafka test next!

[jansyren]

What you have in Kafka which is quite essential for many due to security is the append only and immutable lugs, the fact that they are stored to disk also retains the documents even in case of a crash. So for a banking system etc it is very important that you know the last transaction and that it isn't lost.
RabbitMQ is more for less serious workloads, maybe in a web application backend but nothing I would use for anything that needs security.