The Message Lifecycle

One message, from SendMessage to final disposition, as an explicit state machine.

This is the map you keep returning to. Every remaining module in the course is a deep dive into a single transition in the diagram below — visibility timeout is one arrow, retries are one loop, dead-letter queues are one terminal state. Learning the machine first means those modules become detail rather than new material.

Most production SQS bugs reduce to one sentence: somebody was wrong about which state a message was in.

1. What you will learn

• Draw the full message state machine from memory
• Name every transition and what triggers it
• Explain the four possible endings, including the one that is silent data loss
• Predict what happens to a message when a consumer crashes at any given point

2. Why a state machine

In Module 04 you sent, received and deleted a message. That is the happy path, and it is three states long.

The interesting part is everything else. A message can be delayed before anyone sees it. It can be received and never acknowledged. It can be received eleven times. It can be quietly deleted by AWS because it got too old. It can be moved to a different queue entirely.

Scattered facts about visibility, retries and DLQs only cohere once you can see them as transitions of a single machine. So here it is, and then we will walk each arrow.

3. The machine

stateDiagram-v2
    state "In flight (invisible)" as InFlight
    state "Moved to DLQ" as DLQ
    state "Expired — silent loss" as Expired

    [*] --> Delayed: SendMessage + DelaySeconds
    [*] --> Available: SendMessage
    Delayed --> Available: delay elapses

    Available --> InFlight: ReceiveMessage
    InFlight --> Available: visibility timeout expires
    InFlight --> InFlight: ChangeMessageVisibility

    InFlight --> Deleted: DeleteMessage
    InFlight --> DLQ: count > maxReceiveCount
    InFlight --> Expired: retention elapses
    Available --> Expired: retention elapses

    Deleted --> [*]
    DLQ --> [*]
    Expired --> [*]

Two transitions carry side effects the labels do not have room for:

• Available → InFlight — ReceiveMessage also starts the visibility timer and increments ApproximateReceiveCount.
• InFlight → Available — besides the timer expiring, you can force this immediately with ChangeMessageVisibility(0).

(A message sitting in Delayed can expire too, if DelaySeconds plus retention conspire. It is rare enough that the edge is omitted to keep the diagram readable.)

Reading the diagram. Four things worth noticing immediately:

1. There are three terminal states, not one. Deleted is success. DLQ is managed failure. Expired is unmanaged failure — and it is the only one with no event, no metric spike and no record.
2. InFlight → Available is a loop. A message can go round it many times. That loop is the entire retry mechanism; there is no separate retry feature.
3. Expired is reachable from every non-terminal state. Retention is wall-clock time from SendMessage, and it does not pause while a message is in flight or being retried.
4. InFlight → InFlight is the visibility extension, and it is the only self-transition.

📌 The one-line version: Receiving hides a message. Only deleting removes it. Everything else is a timer.

4. The states

Delayed — Simple: the message exists but nobody can see it yet. Technical: stored and durable, but not returnable by ReceiveMessage until DelaySeconds elapses. Set per queue (DelaySeconds) or per message (a message timer). Example: "retry this webhook in 5 minutes." Maximum 15 minutes (aws-facts.md §2).

Available — Simple: waiting to be picked up. Technical: eligible to be returned by ReceiveMessage. Counted by ApproximateNumberOfMessagesVisible.

In flight — Simple: somebody is working on it. Technical: returned to a consumer and hidden from all others for the duration of the visibility timeout. Still stored; not deleted. Counted by ApproximateNumberOfMessagesNotVisible. ⚠️ "In flight" means a consumer received it, not a consumer is healthy. SQS cannot tell the difference — which is exactly why the timer exists (Module 08).

Deleted — Simple: done, gone. Technical: removed by an explicit DeleteMessage with a valid receipt handle. Terminal.

Moved to DLQ — Simple: given up on, and set aside for a human. Technical: after the receive count exceeds maxReceiveCount, SQS moves the message to the configured dead-letter queue, where it becomes Available on that queue (Module 11).

Expired — Simple: it got too old and AWS deleted it. Technical: MessageRetentionPeriod elapsed since the message was sent. Default 4 days, range 60 s to 14 days. Terminal, silent, and irreversible.

5. The transitions, one at a time

SendMessage → Delayed or Available

Which one depends on DelaySeconds:

• Queue-level DelaySeconds applies to every message (a "delay queue")
• Per-message DelaySeconds on SendMessage overrides it (a "message timer")
• ⚠️ Per-message timers do not work on FIFO queues — only the queue-level delay does

The retention clock starts now, at send — not when the delay ends.

Available → In flight: ReceiveMessage

Three things happen atomically:

1. The message becomes invisible to other consumers
2. The visibility timer starts — from the moment SQS returns it, not from when your code starts working on it
3. ApproximateReceiveCount increments

📌 The receive count counts receives, not failures. A consumer that receives a message and deletes it successfully still incremented it to 1. A message with a count of 4 has been handed out four times, for whatever reason.

In flight → Deleted: DeleteMessage

The only clean ending. Requires the receipt handle from the most recent receive (Module 04 §6).

In flight → Available: the timeout expires

The retry mechanism, in full. Nobody called an API; a timer ran out.

This transition fires for three different reasons that SQS cannot distinguish:

That indistinguishability is the whole difficulty of Module 08, and the reason Module 09 is mandatory rather than optional.

You can also trigger it deliberately with ChangeMessageVisibility(0) — the fail-fast retry.

In flight → In flight: ChangeMessageVisibility(n)

Extends the lease. The new timeout runs from now, not from the original receive. Useful when processing takes longer than expected; the basis of the heartbeat pattern (Module 08).

The absolute ceiling is 12 hours from the original receive, however many extensions you chain.

In flight → DLQ: maxReceiveCount exceeded

⚠️ The move happens on a receive attempt, not at the moment of failure.

That is a subtle and frequently-misunderstood point. With maxReceiveCount = 3, a message is not moved when it fails for the third time. It is moved when SQS goes to hand it out for the fourth time and notices the count has been exceeded. If nothing is polling the queue, a doomed message sits there indefinitely — it does not proactively move itself.

Anything → Expired: retention elapses

Covered in §6, because it deserves its own section.

6. The ending nobody plans for

Three of the four endings are visible. One is not.

⚠️ Retention expiry is the only way SQS loses a message, and it is completely silent.

The mechanics that make it sneak up on people:

• The clock runs from SendMessage, not from the last receive. A message that has been retried for three days has one day left on a 4-day retention.
• It does not pause while the message is in flight. A long visibility timeout does not extend retention.
• It applies inside the DLQ too, and for standard queues the original enqueue timestamp is preserved on the move. A message that spent 3 days in the source queue arrives in a 4-day DLQ with 1 day left — not 4. (FIFO queues reset the timestamp instead.) See aws-facts.md §7 and Module 11.

Why doesn't AWS warn you? Because expiry is not an error from SQS's point of view — it is the retention policy working as configured. SQS has no way to know whether a four-day-old unprocessed message is a disaster or a deliberately abandoned one. The judgement is yours, which means the alarm is yours.

The defence is a single alarm, and it is the most important one in Module 17:

📌 Alarm on ApproximateAgeOfOldestMessage, with a threshold well below your retention period. It is a latency metric — it tells you how long the oldest waiting message has been waiting, which is exactly what a customer would feel. Depth cannot tell you this.

7. Success and failure, side by side

The happy path:

sequenceDiagram
    autonumber
    participant C as Consumer
    participant S as SQS
    participant D as Downstream

    C->>S: ReceiveMessage
    S-->>C: msg + handle (receiveCount=1)
    Note over S: state: In flight<br/>visibility timer: 30s

    C->>D: do the work
    D-->>C: ok
    C->>S: DeleteMessage(handle)
    S-->>C: 200
    Note over S: state: Deleted ✅

The failure path:

sequenceDiagram
    autonumber
    participant C1 as Consumer A
    participant S as SQS
    participant C2 as Consumer B

    C1->>S: ReceiveMessage
    S-->>C1: msg + handleA (receiveCount=1)
    Note over S: In flight · timer 30s

    C1->>C1: processing…
    Note over C1: 💥 OOM-killed at t=12s

    Note over S: t=30s — timer expires.<br/>No delete arrived.
    S->>S: state: Available again

    C2->>S: ReceiveMessage
    S-->>C2: msg + handleB (receiveCount=2)
    Note over S: In flight · timer 30s<br/>NEW receipt handle

    C2->>C2: process ✅
    C2->>S: DeleteMessage(handleB)
    S-->>C2: 200
    Note over S: Deleted ✅

Reading the failure diagram. Two details that matter in practice:

• Consumer A's work was lost, and that is correct behaviour. SQS is designed so that a crashed consumer's message returns. If A had completed the work before crashing, B would now do it again — which is why idempotency is not optional (Module 09).
• handleA is now useless. If A somehow recovered and tried to delete with it, nothing useful would happen.

8. Redelivery is not prompt

A reasonable assumption: "visibility timeout is 30 seconds, so a failed message retries 30 seconds later." On a standard queue this is not reliably true.

⚠️ On a standard queue with maxReceiveCount > 3, a message received 3 or more times without being deleted may be moved to the back of the queue (aws-facts.md §7).

Two consequences:

1. Retry latency becomes unpredictable after the third attempt. Any design that depends on "retry approximately every N seconds" is wrong past that point (Module 10).
2. ApproximateAgeOfOldestMessage is distorted. Once a struggling message is moved back, the metric reports the age of the next message under the threshold — so the genuinely oldest message is not the one being measured. Worth knowing during an incident, when the numbers do not seem to add up.

Why does SQS do this? A poison message at the head of the queue would otherwise be re-served immediately and repeatedly, consuming receive capacity that healthy messages need. Moving it to the back is head-of-line-blocking avoidance.

9. Configuration that shapes the machine

Four settings, each controlling one part of the diagram.

All values from aws-facts.md §2 and §7.

They interact

The relationship people miss:

time before a message reaches the DLQ  ≈  maxReceiveCount × visibilityTimeout

With maxReceiveCount = 5 and a 30-second timeout, a permanently-failing message reaches the DLQ in about two and a half minutes. Raise the timeout to 10 minutes "for safety" and the same message now takes fifty minutes to be quarantined — during which it consumes a worker slot on every attempt.

📌 Visibility timeout and maxReceiveCount are two knobs on one outcome. Tune them together, and sanity-check the product against your retention period. If maxReceiveCount × visibilityTimeout approaches retention, messages will expire before they ever reach the DLQ — losing exactly the evidence you needed.

A worked check

visibilityTimeout   = 5 minutes
maxReceiveCount     = 10
retention           = 4 days (default)

time to DLQ  ≈ 10 × 5 min = 50 minutes          ✅ well inside retention
DLQ retention = 4 days (default)                 ⚠️ but standard queues preserve the
                                                    enqueue timestamp, so the message
                                                    arrives with ~4 days minus 50 minutes
                                                    remaining — set DLQ retention to 14 days

10. Production considerations

Retention expiry is silent data loss. Alarm on ApproximateAgeOfOldestMessage at a threshold far below retention. This is the single most valuable alarm on any SQS queue.

ApproximateReceiveCount is your per-message debugging signal. Log it on every receive (Module 04 §9). A rising distribution across many messages is the earliest warning that a downstream is degrading — earlier than depth, earlier than the DLQ.

Deployments are a lifecycle event. A rolling restart terminates consumers holding in-flight messages. Without graceful shutdown, each of those messages waits out its full visibility timeout and is then redelivered — so every deploy produces a burst of duplicate processing (Module 22).

A message can be in flight far longer than the visibility timeout suggests, if a consumer is heartbeating. Do not infer "stuck" from a long-lived in-flight message without checking.

Set retention deliberately. The 4-day default is a decision somebody at AWS made, not a decision about your system. Ask: how long could our downstream plausibly be unavailable? If the answer exceeds retention, a queue is not sufficient protection on its own.

DLQ retention should be 14 days. You want the maximum possible time to investigate, and standard-queue messages arrive with their original clock already partly spent.

11. Common mistakes

Thinking receive means removal. Why it's wrong: the message is invisible, not gone. A consumer that does not delete reprocesses forever. Instead: internalise the state machine — InFlight is a lease, not a deletion.

Assuming redelivery happens promptly after the visibility timeout. Why it's wrong: after 3 receives, standard queues may move the message to the back of the queue. Instead: never build timing assumptions on retry latency.

Leaving retention at 4 days on a queue that has a DLQ. Why it's wrong: standard queues preserve the enqueue timestamp on the move, so messages can expire shortly after arriving in the DLQ — deleting the evidence. Instead: source retention set deliberately, DLQ retention at 14 days.

Deleting before processing ("delete-then-work"). Why it's wrong: a crash loses the message permanently. You have traded at-least-once for at-most-once without meaning to. Instead: process, then delete.

Raising the visibility timeout without re-checking maxReceiveCount. Why it's wrong: time-to-DLQ scales with the product of the two. A 10-minute timeout with maxReceiveCount = 10 means nearly two hours before a poison message is quarantined. Instead: tune them together and compute the product.

Treating a rising in-flight count as normal. Why it's wrong: NotVisible that rises and never falls means messages are being received and neither deleted nor released — and at ~120,000 the queue stops delivering entirely. Instead: alarm on it (Module 17).

12. Real-world example

One payment message, traced through a bad afternoon. Visibility timeout 30 s, maxReceiveCount 5, retention 4 days, DLQ attached.

Four things this trace teaches that a diagram alone does not:

1. Worker A's crash and the gateway's 503 are indistinguishable to SQS. Both are "no delete".
2. The 14:01:36 → 14:03:11 gap is the back-of-queue move. Nearly two minutes instead of 30 seconds. Any monitoring that assumed a 30-second retry cadence would have reported this wrongly.
3. If worker A had completed the charge before being killed, workers B through E would each have attempted to charge the card again. Only an idempotency guard prevents five charges (Module 09).
4. Total time to quarantine was about seven minutes, not the 5 × 30s = 2.5 min the formula suggests — because of the back-of-queue delay. The formula is a lower bound.

What the team changed afterwards: a circuit breaker so a 503-ing gateway stops consuming rather than burning receive counts (Module 10); an alarm on DLQ depth; and an idempotency key on orderId, because the trace made it obvious how close they had come to a quintuple charge.

13. Interview questions

🟢 Beginner

What happens to a message after a consumer receives it? It becomes invisible to other consumers for the visibility timeout, but stays in the queue. If the consumer deletes it, it is gone; if not, it becomes available again and is redelivered.

What are the ways a message can leave a queue? Four: deleted by a consumer, moved to a dead-letter queue after too many receives, expired at the end of the retention period, or purged manually.

🟡 Intermediate

Walk through the full lifecycle of an SQS message. Sent → optionally Delayed → Available → In flight on receive (timer starts, receive count increments) → then one of: Deleted on acknowledgement; back to Available when the visibility timeout expires; moved to a DLQ once the receive count exceeds maxReceiveCount; or silently Expired when the retention period elapses. The Available ⇄ In flight loop is the entire retry mechanism.

What does ApproximateReceiveCount actually count? Receives, not failures. A successful receive-and-delete still incremented it to 1. It is what maxReceiveCount is compared against.

🔴 Advanced

A consumer crashes after processing but before deleting. Exactly what happens, and when?

SQS observes only the absence of a delete; it cannot tell a crash from slowness. The message stays in flight until its visibility timeout expires — measured from when SQS returned it, not from the crash. At expiry it becomes available and its receive count increments on the next delivery.

The important consequence: the work was already done, so the redelivery causes duplicate execution. Whether that matters depends entirely on whether the consumer is idempotent. Note also that the retry may not be prompt: past three receives on a standard queue, the message may be moved to the back of the queue first.

How can a message be lost in SQS without ever reaching a DLQ?

Three ways, in order of likelihood:

1. Retention expiry. The clock runs from SendMessage and does not pause for anything. A message retried for four days on a default-retention queue is deleted, silently, having never exceeded maxReceiveCount. This also happens inside a DLQ, because standard queues preserve the original enqueue timestamp on the move.
2. A consumer deleting on failure — a finally block, or a catch that swallows and deletes. The DLQ stays empty and the system looks healthy.
3. PurgeQueue, usually run during an incident to make an alarm stop.

Only the first is SQS's doing, and the defence is an alarm on ApproximateAgeOfOldestMessage.

Why does the DLQ move happen on a receive attempt rather than on failure? Because SQS never sees your failure — it only sees the absence of a delete. The receive count is the only signal it has, and it can only act on it at the moment it would otherwise hand the message out again. A practical consequence: if nothing is polling the queue, a doomed message will sit there rather than moving itself to the DLQ.

⚫ System design

Design a system where losing a message is unacceptable. What do you change about retention, DLQs and monitoring?

Start by naming the three loss paths from the question above — that framing is most of the answer.

Configuration:

• Retention at 14 days on the source queue, and 14 days on the DLQ. Maximum time to notice and to investigate.
• maxReceiveCount low (3–5) so failures quarantine quickly rather than burning days of retention.
• Verify maxReceiveCount × visibilityTimeout is orders of magnitude below retention.

Code:

• Never delete on the failure path. Ban finally-block deletes in review.
• Idempotency, so redelivery is always safe and you never feel pressure to delete-on-uncertainty.
• Graceful shutdown, so deploys do not create redelivery storms.

Monitoring:

• Alarm on ApproximateAgeOfOldestMessage at a threshold far below retention — this is the only detector for the silent path.
• Alarm on DLQ depth > 0, because a DLQ nobody watches is a slower kind of loss.
• Alarm on redelivery rate (NumberOfMessagesReceived ≫ NumberOfMessagesDeleted).
• Restrict sqs:PurgeQueue by IAM to an operator role.

Beyond the queue — and this is what distinguishes a strong answer: a queue alone is not a durability guarantee, because of the 14-day ceiling. If "unacceptable" really means unacceptable:

• Persist the intent to a database first, then enqueue — the outbox pattern (Module 24). The database is the source of truth; the queue is a delivery mechanism.
• Run a reconciliation sweeper that finds records in a non-terminal state older than some threshold and re-enqueues them. This catches every loss path at once, including the ones you did not anticipate.
• Consider whether you actually need a log rather than a queue (Module 27) — replay is the property you are really asking for.

14. Summary

• Stored → Available → In flight → { Deleted | Available | DLQ | Expired }.
• Receiving hides a message; only deleting removes it. Everything else is a timer.
• Three terminal states. Deleted is success, DLQ is managed failure, Expired is silent loss.
• The Available ⇄ In flight loop is the entire retry mechanism — there is no retry API.
• Retention runs from SendMessage and never pauses, including while in flight and including inside a standard-queue DLQ.
• Alarm on ApproximateAgeOfOldestMessage — it is the only detector for silent expiry.
• ApproximateReceiveCount counts receives, not failures. Log it on every receive.
• The DLQ move happens on a receive attempt, not at the moment of failure.
• maxReceiveCount × visibilityTimeout ≈ time to DLQ — a lower bound, since standard queues may move a struggling message to the back.

← Previous: 04 — Sending and Receiving Messages · Index: Course home · Next: 06 — Standard Queues →