One key, one order
Exactly-once delivery cannot be achieved over an unreliable network, so an order pipeline must reconstruct an exactly-once effect at the endpoint, keying each instruction to a stable client identifier so any duplicate resolves to the original order rather than a second one.
In an order pipeline, the link between a system and the venue it trades through is never perfectly reliable, and no transport removes that uncertainty. The guarantees actually on offer are at-most-once delivery, which may silently drop an instruction, and at-least-once delivery, which may present the same instruction more than once. Exactly-once delivery, the property a naive design quietly assumes, cannot be had in the general case, because an acknowledgment can itself be lost, and a sender that receives none cannot distinguish a message that never arrived from a reply that never returned. The rational response to that ambiguity is to retry, which is why duplicates are not an exceptional event but an ordinary consequence of building on real infrastructure.
A duplicated order is indistinguishable, by its contents, from a deliberately separate one. If the receiving system treats the arrival of a message as the act of intent, one intended order becomes two working orders. In an execution context, that is not a benign inefficiency. It is unintended exposure: doubled notional, a position no one authorized, and a plausible breach of a limit that every individual instruction respected. The failure is costly precisely because each copy carries the full authority of a legitimate instruction, so nothing downstream has any reason to question it.
Get Zahava Levy’s stories in your inbox
Join Medium for free to get updates from this writer.
What I want is an exactly-once effect, which is a weaker and achievable goal than exactly-once delivery. The instruction carries a stable identifier generated by the client, an idempotency key, and I define the order’s identity in terms of that key rather than in terms of receipt. The first message bearing a given key creates the order and records its outcome. Any later message bearing the same key returns that recorded outcome rather than beginning new work. Processing becomes idempotent in the exact sense of the word: applying the same instruction any number of times has the same effect as applying it once.
Three details carry the discipline, and each is easy to underweight. The key must be the client’s, minted before the first attempt and held constant across every retry, because only the originator knows that two transmissions express a single intention; a key assigned on receipt cannot tell a retry from a genuinely new order. The deduplication must sit at the endpoint that owns the order’s identity rather than in an intermediary, which is the end-to-end argument in its original form: a check placed in the transport can be circumvented by a retry that re-enters above it, so the guarantee has to live at the same binding control the venue’s own identifier passes through, holding from submission through to the fill. And the record of seen keys is a store with a finite retention window, which must exceed the longest plausible retry horizon, or a late duplicate arriving after its record has expired will be admitted as new.
Concurrency sharpens all of this. Two copies of one order can arrive closely enough together to race, so the deduplication itself has to be atomic, a compare-and-set or a uniqueness constraint under which exactly one arrival commits the order, and the others observe that commitment and return its result. Without atomicity, the check simply passes twice, and the duplicate survives the very mechanism meant to catch it.
The principle underneath the mechanics is that correctness under repetition is a primary property of a trading system, not a refinement to be added once the happy path works. A design that is correct only when every message is delivered exactly once is not correct under the conditions it will actually meet. When identity is decided by the key, the system can absorb every duplicate the transport produces and still place exactly what I intended, once.
A retry is not a new order.
