reliability

A canary release sends a slice of real traffic to a new version and asks whether it is healthy. This lesson covers what to measure, why comparing the canary against the current version beats comparing against history, the statistics problem that makes small canaries weak evidence, and how automated promotion and rollback turn a judgement call into a rule.

Deploying is the moment a working system is replaced by a different one while people are using it. This lesson covers what actually goes wrong at that moment, the research finding that shipping fast and shipping safely are not opposites, and the four deployment strategies as answers to one question: how many users meet a bad version before you find out.

Every deployment strategy assumes you can go back, and that assumption is the one most often false when it matters. This lesson covers what actually makes a rollback work, the database migration pattern that keeps schema changes reversible, the one-way doors that no amount of tooling can undo, and how to tell which kind of change you are about to ship.

How reliable should the service be? Wrong question: the right one is how much unreliability you can afford, spent deliberately. This lesson builds the SLI, SLO and error budget machinery from Google's SRE practice, does the arithmetic of nines, explains burn-rate alerting, and shows why 100 percent is the wrong target.

Constrained decoding guarantees syntax and can degrade reasoning. Published work found strict formats hurt reasoning tasks while helping classification, because a schema demanding the answer first denies the model room to derive it. This lesson covers where the damage comes from, why field order is thinking order, the enum trap that manufactures confident errors, and how to measure the cost.

A configuration that fits is not a run that finishes. Llama 3 405B training saw 419 unexpected interruptions in 54 days on 16,384 GPUs, one every three hours, and still kept over 90 percent effective training time. This lesson covers model FLOPs utilisation, the optimal checkpoint interval and where it comes from, loss-spike triage, silent data corruption, and the habits that finish runs.

The concrete patterns that separate an automation that works from one that keeps working: validating input at the boundary, retrying only what is safe to retry, handling rate limits and batches, testing something you cannot easily test, and keeping secrets out of the flow.

No-code automation makes building easy and running reliably hard. This lesson covers what these platforms actually are, the failure modes that appear once something runs unattended, why partial failure is worse than total failure, and the idempotency problem behind most real damage.

Move beyond toy ReAct loops. Learn how to build a production-grade agent harness with a robust control loop, tool registry, schema validation, retry logic, token budgets, abort signals, and a persistent journal that survives crashes.

Why "the same request twice should produce the same result" is one of the most useful properties you can give an API, the standard patterns for implementing it (keys, dedupe tables, natural idempotency), and what goes wrong when you don't.