Expert outline

This chapter is an outline. The curriculum, learning objectives and structure are settled; the prose, diagrams and code are still being written. What is below is the plan for the chapter, not the chapter.

Capstone Project: Order Processing Platform

One system that exercises every mechanism in the course at once. Not a tutorial you follow — a specification you build against, with a production checklist you have to actually pass.

The system

A retailer's order platform. A customer places an order; three independent workers react to it; nothing may be charged twice; the system must survive a downstream outage and a deploy.

flowchart LR
    C[Client] --> API[Order API]
    API -->|outbox| DB[(Orders DB)]
    API --> SNS[SNS: OrderPlaced]
    SNS --> Q1[orders-queue]
    SNS --> Q2[notifications-queue]
    SNS --> Q3[audit-queue]
    Q1 --> W1[Order Worker]
    Q2 --> W2[Notification Worker]
    Q3 --> W3[Audit Worker]
    Q1 -.-> D1[orders-dlq]
    Q2 -.-> D2[notifications-dlq]
    Q3 -.-> D3[audit-dlq]
    W1 --> PAY[Payment Gateway]
    W2 --> EMAIL[Email Provider]
    W3 --> S3[(Audit Store)]

Fanout so a fourth consumer can be added without touching the API. One DLQ per queue so a broken notification worker cannot hide a broken order worker.

Deliverables

#	Document	Contents
1	Requirements	Functional and non-functional requirements, SLOs, traffic profile
2	Architecture	The 20 design questions answered; diagrams; what was rejected and why
3	AWS resources	Every queue, topic, role, key and alarm, with the requirement behind each setting
4	Implementation	Java 21 / Spring Boot services: API, three workers, outbox relay
5	Testing	Unit, integration against LocalStack, and a deliberate duplicate-delivery test
6	Failure scenarios	Six injected failures and the expected system behaviour for each
7	Monitoring	Dashboard, alarms, correlation IDs, structured logs
8	Scaling	Capacity math, autoscaling policies, load test results
9	Security	IAM per role, queue policies, encryption, PII handling
10	Cost model	Itemised monthly cost at the stated traffic, and the optimised version
11	Runbook	What to do when each alarm fires
12	Production checklist	The gate. Every item must pass before "done"

Requirements summary

Functional

POST /orders returns 202 Accepted with a tracking id in under 100 ms p99
Order, notification and audit processing happen independently
A customer is never charged twice, under any failure
An order is never silently lost

Non-functional

500 orders/sec sustained, 5,000/sec peak (Black Friday)
End-to-end processing p99 under 30 seconds
Survives a 30-minute payment-gateway outage with no data loss
Zero-downtime deploys with no duplicate charges
Per-customer ordering for the order worker; no ordering constraint for the others

Constraints

Java 21, Spring Boot, AWS SDK v2
Everything in IaC
No credentials in source
No PII in message bodies

What this exercises

Course module	Where it appears
05, 08	Visibility timeout sized from measured p99; heartbeat on the payment worker
07	Per-customer ordering — FIFO with `MessageGroupId = customerId`, or standard + idempotency. You must justify the choice
09	The no-double-charge requirement, including the external gateway
10, 11	Backoff with jitter; per-queue DLQ; redrive runbook
12, 13	Long polling and batch delete everywhere
14, 15	The capacity math for 5,000/sec, and the ceiling you hit
16	Per-role IAM, queue policy for SNS, SSE-KMS
17	Correlation IDs surviving the SNS→SQS hop
18	The cost model, before and after optimisation
20	Fanout, so the fourth consumer needs no producer change
22, 23	Graceful shutdown that survives the zero-downtime requirement
24	The outbox, because the API writes a row and publishes an event
25	The 20 questions, answered in writing
26	Six injected failures, diagnosed from the dashboard

The hard parts

These are deliberately included, and they are where the learning is.

The dual write. The API must persist the order and publish OrderPlaced. There is no transaction spanning both. Solve it (Module 24) or lose orders.
The external charge. Idempotency inside your database is not enough when the side effect is at a third party. You need the gateway's idempotency key and a local record of intent.
Per-customer ordering at 5,000/sec. FIFO's throughput math (Module 07) decides whether this is even possible, and the answer depends on how many customers are active.
Zero-downtime deploys. Graceful shutdown must complete within the visibility timeout, or every deploy is a duplicate-charge event.
The 30-minute gateway outage. Retention, maxReceiveCount and backoff interact. Compute whether anything expires before the gateway returns.

Definition of done

The production checklist passes in full, and you can answer, with numbers:

How many workers at peak, and what is the binding constraint?
What is the monthly cost, itemised?
What happens to an in-flight message during a deploy?
What happens on the 31st minute of a 30-minute gateway outage?
How would you know, from the dashboard alone, that the notification worker is broken?

← Previous: 29 — Hands-on Labs · Index: Course home · Next: 31 — Interview Preparation →