What is the Dual Write Problem? | Designing Event-Driven Microservices

The dual write problem occurs when you try to write to two separate systems and need them to be atomic. If one write fails, and the other succeeds, you can end up with inconsistent state. This is an easy trap to fall into, and it can be difficult to avoid. We'll explore what causes the dual-write problem and explore both valid and invalid solutions to it.

RELATED RESOURCES

CHAPTERS
00:00 - Intro
00:23 - How to emit events in an Event-Driven Architecture
00:55 - What is the Dual Write Problem?
02:19 - Can you avoid the Dual Write problem by emitting the event first?
02:35 - Can a transaction help avoid the Dual Write problem?
03:54 - What is the Change Data Capture (CDC) pattern?
04:12 - What is the Transactional Outbox pattern?
04:29 - What is the Event Sourcing pattern?
04:42 - What is the Listen to Yourself pattern?
05:43 - Closing

--
ABOUT CONFLUENT

#confluent #apachekafka #kafka

[ConfluentDevXTeam]

Wade here. Just sharing my own experience with this problem. In my early days of building event-driven software, I would take the approach of ignoring the dual-write problem. We tried all sorts of things to make it not matter. However, we often found ourselves in situations where events were missing, or we had extra events that didn't make sense. We'd try going back through the records to understand how that happened and usually ended up with no logical explanation. While I can't prove today that this was the dual write problem, it seems like a pretty logical candidate.

[nitingaur]

Dual Write problem is part of larger distributed transaction handling which is a very complex process with a lot of moving parts. In microservices architecture, where services operate independent of each other, distributed transaction management is replaced by series of local transactions known as saga pattern.

In this video, presenter is illustrating issue when implementing saga with choreography. The presenter did not mention about 2PC (Two Phase Commit) which is an alternate option. It is less preferred and viable because it requires XA compliant transactions resources to participate. The patterns mentioned in the video favours eventual consistency approach.

[akashagarwal]

best concise, crisp, to-the-point video on this topic till date ever by anyone.

[gimmebytes]

When engineering teams start saying "We assume that...", than this often a sign of not knowing the fallacies of distributed systems and hence, building software that contains the mentioned flaws. Nice and crisp summary!

[jewpcabra]

have definitely been working through this problem but never had the right name for it. Thankfully got to one of the approaches for solving it - but its definitely a hairy problem that requires forethought!

[zxborg]

I've never written a microservice (whatever that even is). But having done hardware design and embedded realtime software for the last 30 years, this stuff is just another spin on the same universal problems. Very familiar in general terms.

[rubenvervaeke]

A difficult problem, which I find difficult to explain to collegues. Thank you for such a clear and instructive explanation!

[kevinding]

3:41 certainly could be worth exploring further as a separate topic! Especially if instances of derived events being mishandled, overlooked, or redirected to DQL occur, as this could reintroduce the dual write problem.

[VincentJenks]

Great vid! We ran into all the same issues in 2017 when designing and architecting my first microservices platform. We used Kafka with Node and the Listen to Yourself pattern, and we were quite successful with it! It seems like the path of least resistance and complexity.

[omernacisoydemir]

This is a very descriptive video. I would like to have a question about the message relay layer where the outbox table is read and events are forwarded. I had implemented The Polling publisher pattern specific to my needs. What are the disadvantages when we compare with CDC(Debezium)?

[Lexaire]

Great video and really appreciate your discussions in the comments!

[dimitrikalinin]

Brilliant! Thanks a lot! Based on my experience, it's much harder to explain it to the stakeholders to get additional resources for the proper implementation. Especially in the cloud environment. Components are reliable - they say... It will never happen - they say...

[chessmaster]

IBM Websphere + IBM MQ has been supporting container managed transaction where DB write + MQ send can become part of transaction or consumer + DB + send can become part of transaction and all of it can be rolled back together since very long time

[kaushikmitra]

Hi, what if we enable transactions in the kafka producer and defer the e-mail logic to a kafka consumer which is set to read_committed isolation level?

[warpmonkey]

Great video. Something though is wrong with the levels on your mic, it's crackling on louder sounds much more than it should.

[RicardoSilvaTripcall]

Can we use the SAGA pattern in this case? If something goes wrong, we prepare compensations and execute them on all previous and executed steps int the workflow ...

[ThePlaygroundFire]

Regarding 2:36, what about using Apache Camel? In that scenario, you have a transacted route that writes to the database and then emits an event to a Kafka topic? That would ensure both events are wrapped in a transaction.

[alhassanreda]

why there are no solution that confirm status. so added the status in db and after even confirm the status .

[IIIxwaveIII]

many messaging systems are XA complient and as such supports transactions that span multiple systems. Kafka as far as i know does not support XA but many others do...

[sarwan.surchi]

Wade don't you think independent should be used instead of "dependent"? when mentioning that the two writes should be separated since the scanner simply checks for new events periodically and is independent of the state writer and doesn't really case about what's being stored to be emitted.