Part 04 of 16 · Day 1 · acknowledgement (ack)
Delivered but not processed
Problem and experience
Delivered does not yet mean processed
A service receives a message, crashes, and nobody knows whether the work was done.
- Message
- Servicereceived
- Process crash
- Work?done or not
From practiceWhen did you find out a message had “gone” but the work wasn't done?
TakeawayThe “done” decision must belong to the recipient that actually finished the work.
What it means
An acknowledgement (ack) is a signal from the recipient that it has processed the message and the broker can delete it.
Why we use it
Without an acknowledgement we don't know whether the message was lost or someone is processing it.
When it makes sense
For every asynchronous subscriber.
Where it breaks
- Acknowledging immediately on receipt, before processing.
- Never acknowledging, so the message is processed over and over.
- Having no timeout after which the message returns to the queue.
Imagine
Customer Identity receives the message but crashes before creating the identity. If the broker has already deleted the message, the identity is never created.
Delivery and processing are two different events.
Principle, application and exercise
We acknowledge only after the work
A file-based broker shows what is going on: a message is a file moving between folders. We acknowledge only after processing.
Happy path
- inbox/waiting
- processing/processing
- done/potvrzeno
Error
- processing/chyba
- wait and try againretry with backoff
- dead/after exhausting attempts (DLQ)
Trade-offAcknowledging after processing means a message may be processed twice. We address that with idempotency in parts 09 and 10.
TakeawayA message must not be lost and a permanent error must not block the rest.
What it means
A broker that stores messages as files in folders is illustrative: we can see where a message is. The principles of retry, dead-letter and redrive are the same in real brokers. In the lab the broker is simulated in the browser.
Why we use it
We want to see that the message wasn't lost, and have something to do when it is “poisoned”.
When it makes sense
For every subscriber with side effects.
Where it breaks
- Retry storm: everyone retries at once. Jitter helps.
- Retry without idempotency produces duplicates.
- A DLQ nobody looks at.
Imagine
The identity can't be created. After three attempts the message ends up in dead/. Once fixed, operations sends it again (redrive).
Time is often the cheapest fix, and a DLQ is honest.
Training for your team
Would you like this training for your team?
We run the training on your code and your domain problem. We agree the scope, dates and format with you.
Discuss the training →