Your Database Commit Succeeded. Your RabbitMQ Message Failed. Now What?
Building reliable event-driven systems with .NET, SQL Server, and the Transactional Outbox Pattern.
Imagine building an e-commerce platform where a customer places an order. The application saves that order to SQL Server and publishes an OrderPlaced event to RabbitMQ.
Everything works perfectly — until RabbitMQ becomes unavailable.
The database transaction commits successfully, but the event never reaches the message broker.
Your order exists, yet the inventory, notification, and delivery services may never know about it.
This is one of the fundamental reliability challenges in event-driven architectures.
The dual-write problem
Consider a simplified implementation:
await dbContext.Orders.AddAsync(order);
await dbContext.SaveChangesAsync();await messageBus.PublishAsync(
new OrderPlacedEvent(order.Id));At first glance, this looks reasonable.
However, there are two independent operations:
- Commit the order to SQL Server.
- Publish an event to RabbitMQ.
What happens if the application crashes between these operations?
The order is already committed, but the event is lost.
Reversing the order doesn’t solve the problem either. RabbitMQ might successfully receive an event while the subsequent database transaction fails.
Neither operation participates in the other’s transaction.
That is the dual-write problem.
Why a distributed transaction isn’t always the answer
One possible solution is coordinating both operations through a distributed transaction.
However, distributed transaction support varies across databases and messaging systems. RabbitMQ publishing is not ordinarily enlisted in the same SQL Server transaction.
Introducing distributed coordination can also increase operational complexity and coupling.
For many event-driven applications, a more practical solution is the Transactional Outbox Pattern.
Introducing the Transactional Outbox Pattern
Instead of publishing directly to RabbitMQ during the business operation, the application writes an event record to an Outbox table.
Both the business entity and its corresponding event are committed within a single SQL transaction.
A separate background worker then publishes the pending events.
Architecture
Client / API
|
v
Application Service
|
SQL Transaction
/ \
v v
Orders Table Outbox Table
\ /
Commit
|
v
Background Worker
|
v
RabbitMQ
|
+------+------+
| |
v v
Inventory Notification
Service ServiceThe critical principle is simple:
Either the business change and its outbox event are committed together, or neither is committed.
This removes the failure window between committing the business data and recording the intent to publish an event.
It does not guarantee that a message will be delivered exactly once.
Implementing the pattern with .NET and EF Core
A simplified outbox entity might look like this:
public sealed class OutboxMessage
{
public Guid Id { get; set; }
public string EventType { get; set; } = "";
public string Payload { get; set; } = "";
public DateTime CreatedUtc { get; set; }
public DateTime? PublishedUtc { get; set; }
}When creating an order, persist the order and the outbox message in the same SQL transaction.
await using var tx =
await dbContext.Database
.BeginTransactionAsync();dbContext.Orders.Add(order);dbContext.OutboxMessages.Add(
new OutboxMessage
{
Id = Guid.NewGuid(),
EventType = "OrderPlaced",
Payload = JsonSerializer.Serialize(
new { OrderId = order.Id }),
CreatedUtc = DateTime.UtcNow
});await dbContext.SaveChangesAsync();
await tx.CommitAsync();
This example assumes the order ID has already been generated and both entities are mapped to the same transactional database.
If the transaction fails, neither record becomes committed.
If it succeeds, the outbox event remains available for later publication — even when RabbitMQ is offline.
Publishing events with a background worker
A .NET BackgroundService can periodically query unpublished outbox records.
Get Rajaram Perumal’s stories in your inbox
Join Medium for free to get updates from this writer.
The processing workflow is:
- Fetch a bounded batch of eligible outbox messages.
- Claim the records to prevent competing workers from processing them concurrently.
- Publish messages to RabbitMQ.
- Wait for broker publisher confirmation.
- Mark confirmed messages as published.
- Retry failed messages using a controlled retry policy.
Publisher confirmation matters. A successful call to a client library does not always mean the broker has durably accepted the event.
What happens when the worker crashes?
Consider this scenario:
- The worker publishes an event.
- RabbitMQ confirms acceptance.
- The worker crashes before updating the outbox record.
When the worker restarts, the same event can be published again.
This is expected.
The outbox pattern usually provides at-least-once publication, not exactly-once end-to-end delivery.
Consumers must therefore be idempotent.
Handling duplicate events
Each published event should carry a stable event identifier.
A consumer can use that identifier to detect duplicate processing.
For example, an inventory consumer can maintain a table of processed event IDs with a unique constraint.
Within a local database transaction, it can:
- Register the event identifier.
- Apply the inventory update.
- Commit both changes atomically.
If the same event arrives again, the consumer recognizes that it has already processed it.
For external side effects such as payment calls, an idempotency key or additional reconciliation mechanism may also be necessary.
Production considerations often overlooked
The outbox pattern becomes more interesting when moving beyond a single worker and a small message volume.
Concurrency: Multiple workers should use atomic claiming, leases, or an equivalent database-coordination strategy.
Ordering: Events that belong to the same business aggregate may require ordered processing. Multiple workers can otherwise publish events out of sequence.
Retry strategy: Use bounded exponential backoff and distinguish transient failures from permanently invalid messages.
Dead-letter handling: Messages that repeatedly fail should be isolated for investigation, with a safe mechanism for replay.
Schema evolution: Include event version information so consumers can handle changes without breaking existing integrations.
Observability: Monitor pending message count, oldest unpublished event age, publish latency, retry frequency, and dead-letter volume.
Cleanup: Archive or purge successfully published events according to a defined retention policy.
These details determine whether an architecture remains reliable under real production conditions.
When should you use the Transactional Outbox Pattern?
The pattern is particularly useful when a business transaction must reliably trigger asynchronous downstream work.
Examples include:
- Order creation followed by inventory reservation.
- Payment confirmation followed by fulfillment.
- Account registration followed by notification.
- Delivery assignment followed by real-time dispatch.
- Invoice generation followed by reporting workflows.
It may be unnecessary for applications that have no durable messaging requirement or can tolerate losing a noncritical event.
Architectural patterns should solve identifiable failure modes, not simply add complexity.
Final thoughts
Reliable distributed systems are not built by assuming that databases, message brokers, networks, and workers will always remain available.
They are built by planning what happens when one of those components fails.
The Transactional Outbox Pattern solves a specific and important problem: reliably recording the intent to publish an event alongside a business database transaction.
Combined with publisher confirmations, retries, idempotent consumers, observability, and operational recovery procedures, it provides a strong foundation for event-driven applications.
The goal is not to eliminate failures. It is to make failures recoverable without silently losing critical business events.
Technical topics: .NET, C#, SQL Server, RabbitMQ, Distributed Systems, Software Architecture.
Author: Rajaram Perumal — Technical Architect
