Outbox Pattern
The outbox pattern is one of the most popular patterns to achieve data consistency. It helps to process outcoming messages. Want to know more? Read to find out.
Don’t forget to check out other similar articles:
- Inbox Pattern
- Outbox Pattern
Meanwhile, let us start.
Problem
It is common for the application to send HTTP requests (or messages).
Let’s say the server executes an operation that includes the following actions:
- send a user an email
- publish an event in the service bus
- update the user’s name in the DataBase
Each of the actions can fail:
- email service is unavailable
- the service bus is down
- operation violates a DB constraint
In case we fail at any step, we need to roll back all the previous ones, which is not always possible (e.g. DB-update operation fails, you need to remove the message from the event bus and cancel the email sending).
Get iamprovidence’s stories in your inbox
Join Medium for free to get updates from this writer.
This leads to data inconsistency and unpredictable behavior.
Solution
Instead of executing all external requests right away, the server serializes and stores them in the database as a single atomic transaction.
Later, those stored external requests can be retrieved and executed:
Pros & Cons
The advantages are:
- the outbox pattern ensures that a request/message was sent successfully at least once
- it ensures data consistency, because either the transaction and requests are persisted, or neither are, preventing inconsistencies in distributed systems
- stored requests can be processed by a different server, releasing the load from the main one. The server that processes messages can scale separately
- stored requests can be retried upon failure until completion
Remember about those disadvantages:
- it leads to eventual consistency
- messages or tasks held in an outbox table require memory and storage
- outbox messages are sent according to at least once semantic. It is possible for a message to be published multiple times
Examples
I suggest checking out how Microsoft implemented this pattern in eShopOnContainers to publish integration events
Summary
The Outbox pattern is a design strategy primarily used in distributed systems to ensure reliable message delivery and data consistency between services.
It works by storing messages in an “outbox” table within a transactional boundary. Messages are then later sent to external systems.
The Outbox pattern is especially useful for systems requiring high reliability and consistency across services.
That’s all folks 🐷 I hope you learned something new here 🤓 Give this article a clap 👏 Support me with coffee ☕️ And do not forget to follow to receive more on event-driven architecture patterns ✅
