Using sagas to maintain data consistency in a microservice architecture by Chris Richardson

 April 6, 2021

Here is the link. 

The microservice architecture structures an application as a set of loosely coupled, collaborating services. Maintaining data consistency is challenging since each service has its own database to ensure loose coupling. To make matters worse, for a variety of reasons distributed transactions using JTA are not an option for modern applications. In this talk we describe an alternative transaction model known as a saga. You will learn about the benefits and drawbacks of using sagas. We describe how sagas are eventually consistent rather than ACID and what this means for developers. You will learn how to design and implement sagas in a Java application. Chris Richardson is a developer and architect. He is the author of POJOs in Action, which describes how to build enterprise Java applications with frameworks such as Spring and Hibernate. Chris was also the founder of the original CloudFoundry.com, an early Java PaaS for Amazon EC2. Today, he is a recognized thought leader in microservices. Chris is the creator of http://microservices.io​, a website describing how to develop and deploy microservices. He provides microservices consulting and training and is working on his third startup http://eventuate.io​, an application platform for developing microservices.

Agenda [3:58] ACID is not an option [5:04] (Scenario: Customer has a credit limit) [5:13] Transactions in a monolithic architecture [5:58] Concurrent transaction for the same customer will be serialized [7:15] Microservices [9:00] Microservice architecture [10:30] Loose coupling = encapsulated data [11:13] 2PC is not an option [13:41] Overview of sagas [15:01] Use Sagas instead of 2PC [15:14] Create Order Saga [16:22] Rollback using compensating transactions [18:20] Saga: Every Ti has a Ci [19:08] Create Order Saga - rollback [20:30] Sagas complicate API design [22:03] Request initiate the saga. When to send back the response? Option #1: Send response when saga completes - Response specifies the outcome - Reduced availability Option #2: Send response immediately after creating the saga (recommended) - Improved availability - Response does not specify the outcome. Client must poll or be notified Revised Create Order API [24:41] createOrder() returns id of newly created order NOT fully validated getOrder(id) called periodically by client to get outcome of validation Minimal impact on UI [25:22] UI hides asynchronous API from the user Saga will usually appear instantaneous (<= 100ms) If it takes longer => UI displays "processing" popup Server can push notification to UI Sagas complicate the business logic [26:33] Coordinating sagas [28:33] How to sequence the saga transactions? [28:41] After the completion of transaction Ti "something" must decide what step to execute next Success: which T(i+1) - branching Failure: C(i-1) Option#1: Choreography-based coordination (distributed) [29:45] Option#2: Orchestration-based coordination (centralized) [30:16] Saga orchestrators are state machines [30:48] Implicit vs. explicit orchestrator [31:47] Event-based, implicit orchestrator [32:38] Explicit orchestration [34:05] Create Order Saga (code sample) [35:12] Initializing the saga [37:20] Handling a reply [37:58] Customer Service - command handling [38:28] Transactional messaging [39:18] About Saga orchestrator ⇔ participant communication [39:22] Messaging channels [40:52] Create Order Saga messaging [41:31] Messaging must be transactional [41:59] 2PC still isn't an option [43:06] Use database table as a message queue [43:22] Publishing message using polling [44:33] Transaction log tailing [46:17] About transaction log tailing [46:57] MySQL master-slave replication protocol DynamoDB table streams Summary [48:13]