Kafka retry consumer and error handling - Spring boot

You will learn about Error Handling and Retry pattern from Kafka topic(s) using spring boot so as not to lose any message.

#1 Publishing to the Kafka topic using Spring KafkaTemplate and REST endpoint using spring boot.

#2 Listening to the topic using Spring Kafka Listener and publishing the failed messages to another retry topic.

#3 Listening and retrying the messages and saving the failed messages to a database.

Find the link to the git repo:

If you like the content, do like and subscribe, and do share the feedback.

#apache #kafka #latest #springkafka #errorhandling #retry #java #pattern #springboot #softwaredesign #softwaredevelopment #eventdrivenarchitecture #decoupled #microservices #QuickLearningHub

[sudheerkumar-tpmg]

One of the indepth video on Kafka in which it is really need for interview thanks a lot for sharing such a great content

[shyamagrawal]

Excellent stuff sir .. Thanks for your valuable time and efforts ..

Request and wish you to continue to work on this type of topics ..

[smilysarat]

This is what i was looking for. Thank you so much for the video! You explained it very well. You deserve more likes and subscribers.
I am also looking for publisher error handling. Do you have any video on this.

[rajivsharma]

Good Explanation !!! You have chosen a really good topic. It will be really highly effective if you plan to make a video of doing live coding rather than showing the code. Your subscription would increase very fast.

[siddaramkj]

Good explanation, as you said this architecture is for suitable when order of messages is not important, what are ways we can handle when order is important?

[agusmartinez]

does anyone knows if there is some way to not have the exception thrown by the error handler? Even though it properly retries or whatever, you can see the exception being thrown in the console

[appstude]

thanks for the video. one question please. how to catch a real error while consuming a message ? because offset %0 != 0 was really confusing. regards

[niladrisekharnath]

Great indepth video but instead of using this approach through the console you could have used the debugger but anyways good

[shahjahanazmi]

If the data read by consumer has already failed, how can we save the failed record in a database and then re-read it from there again? Kafka already persists data, we just need to start consuming and reading again from the topic and partition offset where we had left or we stopped reading in the previous read when network failed. Kafka takes care internally of the technical implementations as I understand.

[mrgv]

Good video. I want to implement Retry mechanism at kafka consumer level when particular condition met, i want retry kafka consumer (i will throw an error, at that time i will call consumer to read topic again with 2mins of gap) i want to be implement everything in a Singleproject, not like 2 projects. Spring boot 3.x version with jdk17. Can u help me how to do.and it is non block

[milindchaudhari]

video is blur hence can not able to see the code, can u please provide a github link

[gontlamalli]

Nice Explanation. Similar kind implementation i want in One Project (Both main and retry should be in Same project). Actually my requirement is I want to communicate 3rdparty URL, I am pushing the data to 3rd party API (1st API), they will provide one unique id, i am pushing this id to my Kafka Consumer project which is communicate 3 rd api to check status of my pushed data(1st API Related data to know the status), there side it is taking some 5 to 10mins, in between i need to call the 3rd API for status, i need to use retry in a single project. when data available from 3rd party API, i need to stop retry and need to remove from kafka topic. Plz let me know how to do

[sinurao]

The concept was very good but video is blur difficult to understand please increase the font size