The Digital Looking Glass pattern: Eventual consistency, transactions and compensation
In the principles on data latency I mentioned the importance of compensations. To that end I think that there is a standard pattern that I’ve used, its in some way just formalizing a cache, but in large scale data environments it becomes a bit more than that. In honour of the king of safety critical, I’ve gone for a tribute to another of his books.
Pattern: Digital Looking Glass
Intent: A service/microservice model with high reactivity that is designed to integrate compensation into the core flow, where the transactional functionality is contained within an existing transactional system
Motivation: For operational purposes we need to start moving data control out of being purely a transactional system to becoming data driven.
Applicability: This is intended to be a standard pattern for building digital services on top of an existing transactional solution.
Structure:
Participants: A requestor, the digital service, an existing transactional service and its database.
Collaboration
The Digital Looking Glass (DLG) is made of 3 core parts
- A data store which replicates state from the transactional service, this includes the history of change for that state
- A digital service providing the interactional and data driven capabilities
- A compensation service
When a requestor requests the current state from DLG it returns the latest state available from replication. When the requestor then decides it wants to act on that state, the request is passed back to the transactional system for confirmation. If the confirmation is successful then this is returned, if however the confirmation is not successful then a compensation action is automatically triggered to provide an alternative solution to the requestor.
Example
An example of the DLG is a website which allows the ordering of cheese, the inventory of cheese is replicated to the DLG from the transactional system, a user searches for “Cheddar” and the DLG searches across the replicated store to identify those cheddars which are currently in stock. The user selects an 18 month aged Australian cheddar and makes the request. Unfortunately in the time between the initial request and the commit request from the user there has been a run on Australian 18 month cheddar, instead of just saying ‘denied’ the system instead makes a temporary hold on Canadian 18 month Cheddar, suggesting this as an alternative to the requestor and giving them a window within which the request can be guarenteed to succeed. This avoids a Cheese Shop issue where each subsequent request may also be denied. Because this ‘hold’ needs only to be a rare event and is after a formal request to buy it minimizes the risk of holding items that are in abandoned baskets.
Consequences
This pattern is based on the idea that there are more requestors and more interactions than transactions, and that eventual consistency of the replicated store is sufficient to meet those operational needs. This means that the speed of replication from the transactional system to the DLG must work within data latency boundaries which are acceptable to the use case that the DLG aims to address. It also requires that the design of the DLG includes the concept of compensation as an integral part of any attempt to change system state.
Implementation
A Change Data Capture feed from the transactional system to the DLG, the DLG then updates its current state view and appends the update to the historical view. The current state could be a calculation on the history, depending on the performance required and the performance of that calculation. The current state store is then used for all queries and data access within the DLG.
When a request is made to the DLG it makes the appropriate request to the transactional system, if the transactional system confirms the request it returns. If however the request fails a compensation action, specific to that request, is made. This takes the original request, the reason for failure and the data store within the DLG and comes up with a valid alternative strategy. This could be as simple as posting a redirect request back to the requestor which initiates a different request.
Uses
The intention of this pattern is to create a data driven layer above a transactional infrastructure, enabling a more data oriented way of working and placing the concept of AI and analytics more actively within the engagement between the requestor and the transactional system. This also enables multiple transactional systems to be engaged by a single or group of DLGs, for instance if looking at bundling or aggregation, and undertaking compensations across those systems. An example would be if there are multiple warehouses that contain inventory, the replicated data could indicate that Warehouses 1, 2 and 3 have stock, but Warehouse 1 returns ‘fail’, an immediate request to Warehouse 2 would mean the transaction wouldn’t actually fail. This could also be done in parallel with 3 concurrent requests being made, with ‘success’ being defined as ‘only one succeeds’, if 2 or more succeed the compensation would then cancel all but one of the orders, ideally leaving the order with the shortest delivery cost to the business.
Another example of this pattern is where a transactional system can be augmented with things like dynamic pricing, subscription pricing, adding new interactional facilities above the transactional level, for instance increasing the price if demand is high, decreasing if demand is low. Adding in algorithms to better optimize price to improve both margin and through-put. The DLG is a pattern used to construct data driven systems that leverage the technical capabilities of systems but which operates at a digital operational level rather than only a technical process level.
The advantage of this pattern is that it removes the need to customize transactional systems, using replication as the way to communicate state and then compensation to handle the eventual consistency challenge.
