How to handle message retries & failures in event driven-systems? Handling retires with Kafka?

How to handle message retries & failures in event driven systems?

In event-driven architecture if your services are running and processing without a problem, event driven architecture is great but handling failures can be hard.

How do you handle retires in Apache Kafka?

#eventdrivenarchitecture #danieltammadge #ApacheKafka

-

[IsabelPalomar]

Great video! I really like your conclusion and final comments. I have been working with Kafka a lot this year and event driven is definitely complex.

[StephenTD]

Awesome it cleared up my questions around how to handle retries using a event streaming platform like Apache Kafka and thank you for part 2, where you went into how to keep ordering. Again amazing videos!!!!

[tibi]

Nicely explained - I really liked the presentation :)

[MrBillJDavis]

This is great, thank you. It would be really helpful to talk about what issues might lead to a message getting retried and how that might dictate deciding on X number of retry topics.

[ricardo.fontanelli]

Great video. I would just add one small thing to the retry mechanism: think about event order! Do you really want to consume event 5 after consumer event 7? In many cases, if you already consumed event 7, for example, to update an entity copy in a microservice, all that you need is to discard event 5. To do so, you need to record the id/offset of the last event successfully processed.

[doganaysahin]

I agree. You can use this kind of implementations. But you should be also careful when you retry. Because you can loose the order and some stale data could be happen.
I have a question . How you can handle, exception occurs when you try to send retry topic ?

[eduardleroux]

Great video! Would love to get your take on using Kafka vs AWS SNS / SQS.

It would be great if kafka had a built in retry mechanism (one that does not require additional topics) and once that fails then it's moved to a DLQ.

[cuongnguyenmanh]

Thanks for sharing 👍. I have a question for waiting time in retry topic. how to config it. Thanks

[rajapattanayak]

Hi Daniel great video indeed. I have a question. How can we manage if there is any unhandled exception? If we handle the exception then we can send to retry topic.

[kevinding]

Great video! I'm interested in the design and would like to dive a little bit, we usually would have different schedule retry in 2nd/the 3rd topic, for example, we want to retry 2nd time after 5 mins/3rd retry after 10mins, but Kafka didn't support a delay queue, how should the producer handle produce a 2nd/3rd retry event so it can be executed with scheduled waiting time?

[jincyv]

Hi, how can we handle persistent error in producer side with spring cloud stream

[Danieltammadge]

Part 2 is uploaded so after you watch this one be sure to check it out. Link is at the end of the video

[amseager]

Really wanted to implement a monolith after all of that lol

[chessmaster]

Any code or only this. Anybody can write code but only some can talk

[manideepkumar]

if handson is also there it would have been better, cant get most out of it