This chapter is an outline. The curriculum, learning objectives and
structure are settled; the prose, diagrams and code are still being written.
What is below is the plan for the chapter, not the chapter.
Lab 17 — Break it and fix it
Goal
An injected failure, diagnosed from metrics alone, then fixed.
Setup
To be written — CLI commands and an IaC equivalent for every resource, all prefixed sqs-course-.
Steps
- Deploy the full pipeline from Lab 15 with the monitoring from Lab 16
- Have a partner (or a script) inject one of five failures without telling you which
- Diagnose using only the dashboard and the triage flowchart
- State your hypothesis and the check that would confirm it
- Verify, then fix
- Confirm recovery by watching oldest-message age fall
- Write the prevention: the alarm or config change that would have caught it earlier
What you should observe
- Each failure has a distinct metric signature
- Verification before action is faster than guessing
- Every incident should end with a prevention
Why it happened
To be written — ties each observation back to the mechanism in 26 — Troubleshooting.
Experiments to try
To be written — deliberate variations that change the outcome.
Teardown
To be written — every resource created above, in dependency order.
← Back to: 29 — Hands-on Labs