Introduction to SQS

Module 01 derived the need for a durable buffer between producer and consumer. This module introduces the concrete thing that fills that role — what Amazon SQS is, what it deliberately is not, and the mental model that the rest of the course refines rather than replaces.

Two facts do most of the work here. Everything surprising about SQS follows from them, so they are worth holding onto from the start:

📌 SQS is pull-based. It never initiates a connection to your code. 📌 Deletion is explicit. Receiving a message does not remove it.

1. What you will learn

• State what Amazon SQS is in one accurate sentence
• List what SQS does not do — and why each omission is a design choice
• Distinguish queue name, queue URL and queue ARN, and know which API takes which
• Build the baseline mental model the whole course extends

2. The one-sentence definition

Amazon SQS — Simple: a queue that AWS runs for you, so you never see a server. Technical: a fully managed, distributed message queuing service that stores messages redundantly across multiple servers within a Region, delivers them to polling consumers with at-least-once semantics, and removes them only on explicit deletion. Example: arn:aws:sqs:us-east-1:123456789012:sqs-course-payments

Every clause in that sentence is load-bearing. Unpack it:

3. Why a managed queue exists at all

Running your own broker — RabbitMQ, ActiveMQ, a Kafka cluster — means owning:

• Capacity planning and instance sizing
• Disk management, and what happens when a disk fills with an undrained queue
• Replication and failover configuration
• Version upgrades with no message loss
• Monitoring the broker itself, separately from monitoring your application
• An on-call rotation for a component that is pure infrastructure

None of that is your product. It is the tax you pay for having a queue.

SQS removes all of it. In exchange, you give up:

• Control over storage — you cannot tune replication, choose a disk, or inspect the internals
• Replay — consumption is destructive; there is no "rewind to yesterday"
• Retention beyond 14 days — a hard ceiling (aws-facts.md §2)
• Rich routing — no exchanges, bindings or topic hierarchies inside the queue itself
• Strict global ordering at high volume — FIFO scopes ordering to a message group (Module 07)

🎯 Interview point. "Why SQS?" is not answered by "it's managed". It is answered by naming the trade: you give up storage control, replay and retention in exchange for eliminating an entire operational domain. Then say whether your workload needs the things you gave up.

4. The architecture, at the altitude that matters now

flowchart LR
    subgraph YOURS1["Your infrastructure"]
        P1["Producer<br/>(Order Service)"]
        P2["Producer<br/>(Admin tool)"]
    end

    subgraph AWSBOX["☁️ AWS — you operate none of this"]
        Q[("SQS Queue<br/>sqs-course-payments<br/><br/>stored redundantly")]
    end

    subgraph YOURS2["Your infrastructure"]
        C1["Consumer 1"]
        C2["Consumer 2"]
        C3["Consumer 3"]
    end

    P1 -->|"SendMessage<br/>HTTPS"| Q
    P2 -->|"SendMessage<br/>HTTPS"| Q
    Q -.->|"ReceiveMessage<br/>(consumer asks)"| C1
    Q -.->|"ReceiveMessage"| C2
    Q -.->|"ReceiveMessage"| C3

Reading the diagram. Three things are deliberate:

1. The AWS box contains exactly one thing you can name, and it is the queue. There is no cluster, no broker process, no storage tier that you configure.
2. The producer arrows are solid, the consumer arrows are dotted. The solid arrows are pushes into SQS; the dotted ones are responses to requests the consumers made. SQS did not initiate them. This is the pull model drawn literally.
3. Producers and consumers are plural and unrelated. Neither side knows the other exists. Adding a fourth consumer requires touching nothing — that is spatial decoupling from Module 01 made concrete.

And the same picture reduced to the three API calls that do almost everything:

sequenceDiagram
    autonumber
    participant P as Producer
    participant S as SQS
    participant C as Consumer

    P->>S: SendMessage(queueUrl, body)
    S-->>P: MessageId
    Note over P: Producer is done.<br/>Work is accepted, not completed.

    C->>S: ReceiveMessage(queueUrl)
    S-->>C: Message + ReceiptHandle
    Note over S,C: Message is now hidden from<br/>other consumers — but NOT deleted

    C->>C: process the message
    C->>S: DeleteMessage(queueUrl, receiptHandle)
    S-->>C: 200
    Note over S: Only now is the message gone

Reading the diagram. Note step 5 and step 8. Between them the message still exists in the queue, merely invisible. If the consumer dies at step 7, nobody deletes anything, and SQS eventually hands the message to another consumer. That gap is the whole subject of Module 08, and the reason Module 09 exists.

5. Pull, concretely

SQS has no callback, no webhook, no persistent connection, no "on message" handler. There is one way to get a message out: ask for it.

// This is the entire delivery mechanism. There is no other one.
ReceiveMessageResponse response = sqs.receiveMessage(r -> r
        .queueUrl(queueUrl)
        .maxNumberOfMessages(10)
        .waitTimeSeconds(20));

⚠️ Common mistake. "SQS pushes to Lambda." It does not. The Lambda service runs pollers that call ReceiveMessage on your behalf and then invoke your function. Same for Spring's @SqsListener — a poll loop behind an annotation. Every "push-like" SQS integration is something else doing the polling. Module 19 and Module 23 show both.

Why does this matter now, before you have written a consumer? Because it explains things you would otherwise find arbitrary:

• Why there is a WaitTimeSeconds parameter at all (Module 12)
• Why you are billed for requests that return nothing (Module 18)
• Why scaling consumers requires no registration anywhere
• Why SQS never needs to know whether your consumer is alive — and therefore why a timer must stand in for liveness (Module 08)

6. Explicit deletion, concretely

This is the fact that catches everyone.

Message msg = response.messages().get(0);

process(msg);                          // your work

sqs.deleteMessage(d -> d               // ← without this line, the message comes back
        .queueUrl(queueUrl)
        .receiptHandle(msg.receiptHandle()));

Omit the delete and the message is redelivered — not once, but repeatedly, until it reaches a dead-letter queue (Module 11) or expires at the retention limit.

Why doesn't SQS delete the message when the consumer receives it?

Because SQS cannot know whether your processing succeeded. Consider the alternative: delete on receive. A consumer receives a payment message, crashes before charging the card, and the message is gone forever. That is Module 01's failure 2 — work lost, not delayed — reintroduced at a different layer.

The only party that knows whether the work is done is your code. Explicit deletion is how SQS delegates that judgement to the one participant capable of making it.

This has a direct consequence worth stating now, because it shapes how you write every consumer in this course:

📌 Delete is part of the success path, not cleanup. Process, then delete. Never the reverse, and never in a finally block that runs on failure too.

7. Naming: queue name, URL and ARN

One queue, three identifiers. Beginners lose time to this, and the fix is thirty seconds of reading.

Name : sqs-course-payments
URL  : https://sqs.us-east-1.amazonaws.com/123456789012/sqs-course-payments
ARN  : arn:aws:sqs:us-east-1:123456789012:sqs-course-payments

All three encode the same three facts — Region, account, name — in different shapes.

The rule of thumb: URL for talking to the queue, ARN for talking about it.

// Name → URL, when you only know the name
String queueUrl = sqs.getQueueUrl(r -> r.queueName("sqs-course-payments")).queueUrl();

// URL → ARN, when a policy or integration needs it
String queueArn = sqs.getQueueAttributes(r -> r
        .queueUrl(queueUrl)
        .attributeNames(QueueAttributeName.QUEUE_ARN))
    .attributes()
    .get(QueueAttributeName.QUEUE_ARN);

⚠️ Common mistake. Calling GetQueueUrl on every send. It is a billed API call that returns a value which never changes. Resolve the URL once at startup and cache it — or better, pass it in as configuration.

Naming rules (aws-facts.md §3): up to 80 characters, alphanumerics, hyphens and underscores only, case-sensitive, and FIFO queues must end in .fifo (the suffix counts toward the 80).

⚠️ After you delete a queue you must wait 60 seconds before creating one with the same name. This bites in integration tests that create and tear down per run — use a unique suffix instead.

8. Creating a queue

CreateQueueResponse created = sqs.createQueue(r -> r
        .queueName("sqs-course-payments")
        .attributes(Map.of(
            // How long a received message stays hidden. Default 30s — Module 08.
            QueueAttributeName.VISIBILITY_TIMEOUT, "30",

            // How long an unconsumed message survives. Default 4 days — Module 05.
            QueueAttributeName.MESSAGE_RETENTION_PERIOD, "345600",

            // Long polling. Defaults to 0 via the API — set it. Module 12.
            QueueAttributeName.RECEIVE_MESSAGE_WAIT_TIME_SECONDS, "20"
        )));

String queueUrl = created.queueUrl();

The CLI equivalent:

aws sqs create-queue \
  --queue-name sqs-course-payments \
  --attributes VisibilityTimeout=30,MessageRetentionPeriod=345600,ReceiveMessageWaitTimeSeconds=20

Three of those attributes are set deliberately, and the third is the one people miss:

⚠️ ReceiveMessageWaitTimeSeconds defaults to 0 when you create a queue through the API — which means short polling, which means more empty responses and a bill that can be orders of magnitude larger than it needs to be. The console presents different defaults, so a queue created by hand and a queue created by your IaC can behave differently. Module 12 quantifies this.

CreateQueue is idempotent for identical attributes: calling it again returns the same URL. Calling it with different attributes for an existing queue raises QueueAlreadyExists. This makes it safe to call at application startup, and unsafe to use as a configuration-management mechanism — use infrastructure as code instead.

9. What SQS is not

Each of these is a design choice, not a gap. Knowing why is worth more than knowing that.

Not a database

No queries, no updates, no scans, no indexes, and a 14-day retention ceiling. You cannot ask "is there a message for order 4471?" — you can only take the next available one.

Design rule: put intent in the message and state in a database. A message says "charge order 4471"; the order's status lives in Postgres. If your consumer needs to look something up, it looks it up — it does not expect the queue to hold it.

Not a log

Consumption is destructive. There is no offset, no rewind, no replay. If you deploy a bug that mis-processes 100,000 messages, those messages are gone — the only copy of what happened is whatever you wrote down yourself.

This is the single most consequential difference from Kafka (Module 27), and it is a genuine reason to choose something else.

Not a broadcast mechanism

One queue serves one logical consumer group. Two services polling the same queue compete — each message goes to one of them, not both.

flowchart LR
    subgraph WRONG["❌ Two services, one queue"]
        Q1[("Queue")] --> S1["Inventory Service<br/>gets msg 1, 3, 5…"]
        Q1 --> S2["Analytics Service<br/>gets msg 2, 4, 6…"]
    end
    subgraph RIGHT["✅ Fanout — one queue each"]
        T(["SNS Topic"]) --> QA[("Queue A")] --> S3["Inventory Service<br/>gets every message"]
        T --> QB[("Queue B")] --> S4["Analytics Service<br/>gets every message"]
    end

Reading the diagram. On the left, each service silently sees roughly half the traffic — and the bug is invisible in development, where you only send one message at a time and it happens to land on the service you were watching. On the right, SNS delivers an independent copy to each queue. Module 20 builds this.

🎯 Interview point. "Two consumers on one queue, both expecting every message" is a classic interview trap. The answer is fanout, and the reason is that a queue's consumption is destructive.

Not a scheduler

DelaySeconds maxes out at 15 minutes (aws-facts.md §2). For "run this in three days", use EventBridge Scheduler or Step Functions and have that enqueue the message.

Not a priority queue

There is no priority field. SQS delivers what is available; it does not reorder by importance. The standard pattern is separate queues with separate consumer fleets (Module 25).

Not push, and not push with extra steps

Covered in §5, and worth repeating because it gets re-learned as a misconception in every Lambda tutorial.

10. Production considerations

Things that are true from day one, even before you know the mechanisms behind them.

Default settings are rarely the right ones. A freshly created queue has short polling, a 30-second visibility timeout, 4-day retention, and no dead-letter queue. Essentially every production queue changes at least three of those. The production checklist is the full list.

A queue is a Regional resource. There is no multi-Region SQS. Disaster recovery means an explicit design — active/passive with producer failover, or dual-write with deduplication — and a stated RPO (Module 25).

Queue names are not a security boundary. Anyone with sqs:* on * can read your queue regardless of what you called it. IAM is the boundary (Module 16).

Queues are cheap; mixed-purpose queues are not. Creating a queue costs nothing until it carries traffic. One queue carrying three unrelated workloads cannot be tuned, scaled, alarmed or owned independently — and you will want all four.

Plan for message-schema change on day one. You cannot migrate messages already in flight. A version field in the body, and consumers that tolerate unknown fields, cost nothing now and save a deployment freeze later (Module 04).

11. Common mistakes

Expecting SQS to push. Why it's wrong: there is no delivery callback; nothing will ever call your code. Instead: poll — or let Lambda's event source mapping poll for you, understanding that it is doing the polling.

Expecting a message to vanish when received. Why it's wrong: receiving hides the message; deletion removes it. A consumer that never deletes reprocesses forever. Instead: process, then DeleteMessage.

Expecting ordering on a standard queue. Why it's wrong: ordering is best-effort by design, because there is no global sequencer across SQS's storage servers (Module 03). Instead: FIFO if you truly need it, or order-independent processing.

Two services polling one queue, both expecting every message. Why it's wrong: they compete; each message is delivered to one of them. Instead: SNS fanout, one queue per consumer group.

Treating SQS as a database or a log. Why it's wrong: no query, no replay, 14-day ceiling. Instead: intent in the message, state in a database, history in a log or S3.

Calling GetQueueUrl on every operation. Why it's wrong: a billed API call returning a constant. Instead: resolve once at startup, or configure it.

12. Real-world example

A notification service, which will be reused for concrete numbers throughout the course.

The requirement. When an order ships, email the customer. Peak 2,000 orders/minute. The email provider's API is rate-limited to 100 requests/second and has a p99 of 400 ms. It has an SLA of 99.5%, which is three and a half hours of downtime a month.

Without a queue. The order service calls the provider inline. At peak, 2,000/min is 33/sec — under the rate limit, fine. But the provider's monthly 3.5 hours of downtime becomes 3.5 hours of failed shipments, because the shipment write and the email are in the same transaction path. And at 400 ms per call, the order service holds a thread for every email it sends.

With a queue.

flowchart LR
    O["Order Service"] -->|"SendMessage<br/>~8 ms"| Q[("sqs-course-notifications")]
    Q -.->|"poll"| W["Notification Workers<br/>rate-limited to 100/s"]
    W --> E["Email Provider"]
    W -.->|"repeated failures"| D[("DLQ")]

• The order service's involvement is one SendMessage, about 8 ms. Shipment no longer depends on email at all.
• Workers self-limit to 100/sec, so the provider's rate limit is respected by construction rather than by luck.
• During the provider's downtime, messages accumulate. A 30-minute outage at 33/sec is about 59,000 messages — well within a queue's capacity — and they drain once the provider returns.
• Messages that fail repeatedly land in a DLQ for inspection rather than being retried forever.

What it cost. An email is now sent shortly after shipment rather than during it, so the customer-visible ordering of "order marked shipped" and "email arrives" is no longer guaranteed. The team had to decide that was acceptable — and it was, but somebody had to ask.

13. Interview questions

🟢 Beginner

What is Amazon SQS? A fully managed, distributed message queue. Producers send messages; consumers poll for them and delete them after processing. AWS operates all the infrastructure; you configure behaviour and write both ends.

Does SQS push messages to consumers? No. SQS is strictly pull — consumers call ReceiveMessage. Integrations that look like push, such as Lambda triggers, have another component doing the polling.

🟡 Intermediate

Why does SQS require an explicit DeleteMessage? Because SQS cannot observe whether your processing succeeded. If receiving implied deletion, a consumer crash between receiving and completing the work would lose the message permanently. Explicit deletion puts the acknowledgement in the hands of the only party that knows the outcome — and it is what makes at-least-once delivery survive a consumer crash.

What does "fully managed" actually remove from your operational burden? Capacity planning, instance sizing, disk management, replication configuration, failover, version upgrades, and monitoring the broker itself. What it does not remove: choosing the configuration, designing for duplicates, scaling your consumers, and monitoring your pipeline.

What is the difference between a queue URL and a queue ARN? Both identify the same queue. The URL is the endpoint that data-plane API calls target; the ARN is the identifier that IAM policies, queue policies, redrive policies and service integrations reference. URL to talk to the queue, ARN to talk about it.

🔴 Advanced

What can SQS not do that a self-hosted broker can? When does that matter?

Four things, in rough order of how often they matter:

1. Replay. Consumption is destructive. If you deploy a bug that mis-processes a day of messages, there is no rewind. This matters whenever the messages represent facts you might want to re-derive from — which is most event-sourced designs.
2. Retention beyond 14 days. A hard ceiling. If a downstream can be out for longer than that, you need a different durable store.
3. Rich routing. No exchanges, bindings, or header-based routing inside the queue. You compose SNS or EventBridge in front instead — which works, at the cost of another moving part.
4. Strict global ordering at high volume. FIFO scopes ordering to a message group, and the throughput available depends on how many groups you have.

The honest framing: for a task queue, SQS gives up almost nothing you will miss. For an event backbone, replay is often decisive.

Why is SQS pull-based when push would have lower latency? Three reasons. A pull consumer can never be overwhelmed — it receives exactly what it asked for, which is backpressure at the delivery layer. SQS needs no registry of consumer addresses or health, so you can scale consumers with no coordination. And push requires the broker to determine consumer liveness, which is not reliably knowable in a distributed system — SQS sidesteps the question entirely by using a timer (the visibility timeout) instead of a health signal.

⚫ System design

Your team debates SQS versus running RabbitMQ on EC2. Argue both sides on cost, reliability and operational load.

For SQS:

• Operational load is the decisive term and it is usually underweighted. RabbitMQ on EC2 means patching, clustering, quorum queue configuration, disk alarms, upgrade rehearsals, and an on-call rotation for infrastructure that is not your product.
• Cost is usage-shaped with no floor. For spiky or low-average workloads SQS is cheaper in total even where its per-message price is higher, because RabbitMQ's instances run 24/7 at peak sizing.
• Reliability is AWS's regional durability versus your cluster configuration. Most teams do not operate a broker better than AWS does.

For RabbitMQ:

• Routing. Exchanges, bindings and header-based routing are genuinely more expressive than SNS filter policies for complex topologies.
• Latency. Sub-millisecond, in-VPC, no polling interval.
• Portability. No cloud lock-in; the same broker runs anywhere.
• Cost at very high sustained volume. Per-request pricing loses to a fixed fleet once volume is both enormous and steady — this is a real crossover, and it is worth computing rather than assuming (Module 18).
• Existing expertise. A team that already runs RabbitMQ well has already paid the fixed cost.

The synthesis an interviewer is listening for: the decision is rarely about the broker's features. It is about whether operating a broker is a good use of this team's time, and whether the workload needs any of the capabilities SQS gives up. Name the crossover conditions rather than declaring a winner.

14. Summary

• SQS = a durable, distributed, pull-based queue with explicit deletion, operated by AWS.
• Pull-based and explicit delete are the two facts that explain nearly everything else.
• Standard queues give at-least-once delivery and best-effort ordering.
• Three identifiers: name to create, URL to call, ARN to reference.
• ReceiveMessageWaitTimeSeconds defaults to 0 via the API — set it to 20.
• SQS is not a database, a log, a broadcast mechanism, a scheduler, or a priority queue. Each omission is deliberate, and each has a standard workaround.
• One queue = one consumer group. Two services polling the same queue compete; use SNS fanout.
• Default configuration is almost never production configuration.

← Previous: 01 — Messaging Fundamentals · Index: Course home · Next: 03 — How SQS Actually Works →