A timeout does not mean nothing happened. Without idempotency, a second attempt leads to a double payment, two accounts or the same e-mail sent to a customer twice.
Idempotency is the property that lets a system safely try the same thing again.
In a distributed system we will sooner or later receive the same request twice. The question is not whether it happens, but what happens to the data when it does.
Contents
- What idempotency is
- Why it matters in distributed systems
- What happens without it
- How to ensure idempotency
- Idempotency in HTTP APIs
- Typical mistakes
- Five questions for your system
What idempotency is
An operation is idempotent when performing it repeatedly has the same result as performing it once.
The difference between two commands shows it best:
- “Set the balance to CZK 5,000.” Whether I run it once or three times, the balance is CZK 5,000.
- “Add CZK 5,000.” Every further run changes the result.
The first command is idempotent, the second is not. A lift button works the same way: when I press it a second time, the lift does not come twice.
In software we do not care about the mathematical definition but about the effect on the state of the system. An idempotent operation may return a different response or write a log entry. What matters is that the second run does not change the data differently from the first.
Why it matters in distributed systems
As long as everything runs in one process and one transaction, we can see whether an operation went through. As soon as a network sits between two parts of the system, we lose that certainty.
Duplicates have three typical sources:
- A lost acknowledgement. The service processes the message, but the acknowledgement is lost on the way. The broker delivers it again.
- A client retry. The call times out and the client repeats it. It does not know whether the first attempt went through.
- Broker redelivery. After a connection or node failure, the broker delivers messages the consumer may already have seen.
The RabbitMQ documentation says so directly: messages can be redelivered and consumers must be prepared to handle deliveries they have seen in the past. It recommends designing consumers to be idempotent [1].
A broker guarantees “at least once”, not “exactly once”.
In our Event-Driven Architecture training we show this on a mortgage payout. The Disbursement service calls the payment system and the call times out. The money may have left, or it may not. Sending the payout again is a risk; not sending it is a risk too.
What happens without it
Take a service that opens a loan account once a mortgage is approved. It receives the message “open an account for application 42”, creates the account and acknowledges the processing.
The acknowledgement is lost. The broker delivers the message again. The service processes it as new and creates a second account.
There are usually two intuitive reactions, and both are wrong:
- Turn off retries. Then, in a real outage, the message is lost entirely and the account is never created.
- Retry “more carefully”. A longer timeout or fewer attempts only make duplicates rarer; they do not remove them.
Reliable delivery and correct data do not go together without idempotency. In practice, “exactly once” is therefore made of two parts: at-least-once delivery plus idempotent processing.
How to ensure idempotency
Idempotency key
The client adds a unique identifier to every request. The server stores the result of the first processing against the key. When the same key arrives again, it does not perform the operation and returns the stored result.
Payments in Stripe work this way, for example. The server stores the status code and body of the first request made with a given key, and repeated requests with the same key receive the stored response. Keys can be removed once they are at least 24 hours old [2].
What matters is who creates the key: the client, before the first attempt. A key generated on the server would be different for every retry.
Inbox: recording processed messages
For messages from a broker, the key is the message ID. The consumer records in its own database which messages it has already processed and discards duplicates. The pattern is called Idempotent Consumer [3].
The condition is that the processing record and the data change are written in one transaction:
public async Task Handle(OpenMortgageAccount message)
{
await using var tx = await db.BeginTransactionAsync();
// Duplicate: the message has already been processed, do nothing more.
if (await db.ProcessedMessages.AnyAsync(m => m.Id == message.MessageId))
return;
db.Accounts.Add(Account.Open(message.ApplicationId, message.Amount));
db.ProcessedMessages.Add(new ProcessedMessage(message.MessageId));
await db.SaveChangesAsync(); // both, or nothing
await tx.CommitAsync();
}
The check and the write must be in one transaction, and the processed messages table needs a unique index on the ID. Otherwise two concurrent deliveries pass the check at the same time.
Naturally idempotent operations
Sometimes idempotency can be ensured without any record:
- Set a state instead of changing it by a value. “Application status = approved” instead of “move the status one step further”.
- A unique constraint in the database. An account can exist only once per application. A second attempt fails on uniqueness and the service treats it as “already done”.
- The aggregate version. Optimistic concurrency rejects a write that expects an old version of the data.
Where idempotency ends
An idempotent receiver protects only its own data. If the service calls another system while processing, for example to send an e-mail or a payment, it has to pass the idempotency key on. Otherwise the duplicate just moves one floor down.
On the sender side the Transactional Outbox pattern helps: the message is stored in the same database as the data change and sent only afterwards [4]. The Outbox ensures the message is not lost. The idempotent receiver is responsible for the duplicates that sending creates.
Idempotency in HTTP APIs
The HTTP specification defines idempotent methods: a repeated identical request has the same intended effect on the server as a single one. PUT, DELETE and safe methods such as GET are idempotent. POST is not [5].
That is why payments, orders and other POST requests use the Idempotency-Key header. The IETF is standardising it precisely so that non-idempotent methods can be retried safely [6].
In practice:
- the client generates the key (for example a UUID) and sends the same one on retry;
- the server stores the key together with the result and the request parameters;
- the same key with different parameters is a client error, not a new request.
Typical mistakes
- Checking for duplicates outside the transaction. Two concurrent deliveries pass the check at the same time and both write data.
- A key generated on the server. Every retry then looks like a new request.
- Deduplication in memory only. After the service restarts, the record is gone and duplicates get through.
- Keeping keys too briefly. A retry after an outage can arrive hours later. The retention period must be longer than the longest possible retry.
- Idempotency only in the first service. The duplicate moves to the downstream system that did not receive the key.
Five questions for your system
- What happens when this message arrives twice?
- Who creates the idempotency key, and does it stay the same on retry?
- Is the processing record written in the same transaction as the data?
- How long do we keep keys, and is it enough for the longest outage?
- Do we pass the key on to the systems we call?
Idempotency is not an optimisation. It is the condition for a retry to be safe.
You can try the duplicate yourself in our Event-Driven Architecture training, in the part on idempotency and in exercise 12, where the simulator shows the same scenario without and with an Inbox. The article Event Sourcing as the memory of a system builds on the same topic.
References and sources
Sources verified on 26 October 2026.
[1] RabbitMQ. Reliability Guide. RabbitMQ documentation. The section Data Safety on the Consumer Side: redelivery and the recommendation of idempotent consumers.
[2] Stripe. Idempotent requests. Stripe API Reference. Storing the response of the first request with a key, repeated requests and key retention.
[3] Richardson, Chris. Pattern: Idempotent Consumer. Microservices.io. Recording the IDs of processed messages in the database and discarding duplicates.
[4] Richardson, Chris. Pattern: Transactional Outbox. Microservices.io.
[5] Fielding, R.; Nottingham, M.; Reschke, J. RFC 9110: HTTP Semantics, section 9.2.2 Idempotent Methods. IETF, 2022.
[6] IETF HTTPAPI Working Group. The Idempotency-Key HTTP Header Field. Internet-Draft, IETF. A draft standard, not a final RFC.