Using sagas to maintain data consistency in a microservice architecture by Chris Richardson

The microservice architecture structures an application as a set of loosely coupled, collaborating services. Maintaining data consistency is challenging since each service has its own database to ensure loose coupling. To make matters worse, for a variety of reasons distributed transactions using JTA are not an option for modern applications.

In this talk we describe an alternative transaction model known as a saga. You will learn about the benefits and drawbacks of using sagas. We describe how sagas are eventually consistent rather than ACID and what this means for developers. You will learn how to design and implement sagas in a Java application.

[benp]

Very interesting talk, but set playback to 1.25x speed. It transforms this talk.

[zhouyuan]

Agenda [3:58]
ACID is not an option [5:04]
(Scenario: Customer has a credit limit) [5:13]
Transactions in a monolithic architecture [5:58]
Concurrent transaction for the same customer will be serialized [7:15]
Microservices [9:00]
Microservice architecture [10:30]
Loose coupling = encapsulated data [11:13]
2PC is not an option [13:41]
Overview of sagas [15:01]
Use Sagas instead of 2PC [15:14]
Create Order Saga [16:22]
Rollback using compensating transactions [18:20]
Saga: Every Ti has a Ci [19:08]
Create Order Saga - rollback [20:30]
Sagas complicate API design [22:03]
Request initiate the saga. When to send back the response?
Option #1: Send response when saga completes
- Response specifies the outcome
- Reduced availability
Option #2: Send response immediately after creating the saga (recommended)
- Improved availability
- Response does not specify the outcome. Client must poll or be notified
Revised Create Order API [24:41]
createOrder()
returns id of newly created order
NOT fully validated
getOrder(id)
called periodically by client to get outcome of validation
Minimal impact on UI [25:22]
UI hides asynchronous API from the user
Saga will usually appear instantaneous (<= 100ms)
If it takes longer => UI displays "processing" popup
Server can push notification to UI
Sagas complicate the business logic [26:33]
Coordinating sagas [28:33]
How to sequence the saga transactions? [28:41]
After the completion of transaction Ti "something" must decide what step to execute next
Success: which T(i+1) - branching
Failure: C(i-1)
Option#1: Choreography-based coordination (distributed) [29:45]
Option#2: Orchestration-based coordination (centralized) [30:16]
Saga orchestrators are state machines [30:48]
Implicit vs. explicit orchestrator [31:47]
Event-based, implicit orchestrator [32:38]
Explicit orchestration [34:05]
Create Order Saga (code sample) [35:12]
Initializing the saga [37:20]
Handling a reply [37:58]
Customer Service - command handling [38:28]
Transactional messaging [39:18]
About Saga orchestrator ⇔ participant communication [39:22]
Messaging channels [40:52]
Create Order Saga messaging [41:31]
Messaging must be transactional [41:59]
2PC still isn't an option [43:06]
Use database table as a message queue [43:22]
Publishing message using polling [44:33]
Transaction log tailing [46:17]
About transaction log tailing [46:57]
MySQL master-slave replication protocol
DynamoDB table streams
Summary [48:13]

[Suamere]

I incorporated a company in 2007 who's goal was simply to proclaim the wonderfulness of MicroServices. Since (and before) then, I have never seen such a flawless talk related to MicroServices. Every other "concern" about distributed systems, Service-Oriented Architecture, Statelessness, Events, Administration, Security, and so forth have all been so mundane. And frankly, all of those other concerns can be mocked out in fairly generic frameworks. This talk touches on the actual MEAT of Software Development which is different for every customer and is the actual fun challenge. It may be stuff that experienced MicroService people know, but it has never been spoken so eloquently. I appreciate this talk very much.

[vishalsheth]

Great talk, one thing I realized after this talk is that 2PC is an option.

[GarriAndOkroSoup]

Had already been implementing this principle in some of my designs before learning that it had a name . what a saga :D

[anildatt]

Old wine in new bottle. I have been doing this design pattern of State Machine Orchestrator in my SOA EDA, with compensating transactions to rollback what services had done till the point of failure. The failure event is either broadcast to all the services or to the Orchestrator so it can co-ordinate the cancellation. As I listened to the talk it was like going through my design document.
Message table to message broker has a drawback when the load is high. As it is DB spefic to pull data out to push on message broker it introduces a bottleneck in the DB. Ideal for this is using a distrbuted cache DB or like he mentioned the replication and use the replicated data to message broker.

Excellent talk. It touched all the touchpoints

Newbies to Micorservices without much experience may think this is something rocketscience, but it as been there for long in a different avatar.

[kevinkkirimii]

Great speaker, articulate and simplifies the concept

[landkasupada]

This is a real talk on real use cases on real problems

[mohammadbarbast]

excellent, I have looking for something like this for days. really thanks to Chris Richardson

[chang]

Actually I'm readying that book. very informative book!

[jsignuy]

I'd only add some minute or two about idempotency in some key moments of the process. Despite of that, this is an excellent talk.

[mjmsxnh]

I readed alot about Saga design pattern and I developed two years a system than implement Saga design pattern and I still learned new things from this video. Thank you.

[talivanov]

Great Speaker!
Knows how to simplify ideas.

[brainoverflow]

I feel like "Saga Orchestration" is completely against the logic behind the microservices which is building independent components and teams. In this scenario a team has to maintain the orchestration point of two services which also has to be aware of the business logic of both services. I know sometimes orchestration is inevitable but I think in this situation it's better to use choreography.

[volkerreichel]

Very good explanation of the concepts.

[greneroom]

Very interesting talk!

One question I had near the end of the talk — it seems like using the DB’s change log as the mechanism to publish to the message broken has a large drawback: it tightly couples the internal DB scheme to the API a service exposes to the Saga (which defeats one purpose of microservices, since you want to avoid leaking your DB).

I didn’t quite follow why having each individual service coordinate a 2PC between its own DB and a message broker was problematic. This seems much less bad than using distributed TXNs to implement the entire Saga, since the 2PC is an implementation detail of each service. The only single point of failure is the message broker, and I think this is true regardless of your approach?

[alexknn]

Great talk, like that you explained very important part as processing data and sending event atomically. Other resources usually do not explain this important part. But I want to mention one moment, as usually messaging systems guarantee “at least once” delivery, we have to think about idempotency. And if we implement it, we don’t need complicated atomic save and send mechanism anymore, we now can save data and then try to send message to the next queue and ack/commit current queue. For example If we have server failure after saving to DB and before sending to next queue and acking current, then the other instance will get and process current message (here idempotency come into play as we already saved data to DB and another instance will do the same processing cycle with the message - save to db and send to next queue)

[donotreportmebro]

13:08 correct me if I'm wrong but that transaction example with a single database doesn't work unless you have specified a serialized isolation level (one cpu core to execute all transactions). Otherwise it suffers from write skew even with Snapshot Isolation transaction isolation level which is by default I guess on many relational DB systems. You still might have two transactions that would confirm that there is enough credit at the same time, and at the same time execute insertion of two new orders.

[alexright]

Hi, i understand that 2PC is out of the scope between two microservices, but inside a single microservices you can use it. For example when you have to write a record on DB and send a message to a broker to start another micorservice.

[hamedhatami]

It's been really informative demonstration about how the Saga works entirely, when it comes to publish a message to the message broker (queue) in order to make them transactional, we can design them to get ACK back then consider it as done thus give it a go