Lessons

Vector search did not replace keyword search, because the two fail in opposite places: BM25 cannot see that laptop and notebook mean the same thing, and embeddings cannot see that SKU-4471-B is not approximately anything. This lesson maps the two failure surfaces, then builds the production answer: run both retrievers and fuse the rankings, with reciprocal rank fusion done by hand.

Matching finds candidates; scoring orders them, and the ordering is the product. This lesson builds BM25, the default ranking function of Lucene, Elasticsearch and OpenSearch, from its three ingredients: rare terms count more, repeated terms saturate, and long documents get discounted. With the formula, the two tuning knobs, and the saturation curve computed by hand.

Search does not scan documents; it looks up precomputed answers. This lesson builds the inverted index from first principles, then covers the pipeline that feeds it: tokenization, normalisation, stemming and synonyms, and why an analysis mistake made at index time cannot be fixed at query time. Includes the classic failure where a product SKU becomes unfindable.

How reliable should the service be? Wrong question: the right one is how much unreliability you can afford, spent deliberately. This lesson builds the SLI, SLO and error budget machinery from Google's SRE practice, does the arithmetic of nines, explains burn-rate alerting, and shows why 100 percent is the wrong target.

Tracing every request through every service produces the most useful telemetry you have and more of it than anyone can afford. This lesson covers how context propagation actually stitches a trace together, the head versus tail sampling decision and why it determines which incidents you can debug, and the collector pipeline where all of it is enforced.

Two pieces of arithmetic decide whether your telemetry is useful and affordable. Percentiles, because averages hide exactly the users who are suffering, and you cannot average a p99. Cardinality, because metric cost is not per event but per label combination, and one careless label can multiply your bill by the size of your user base. This lesson does both mechanisms by hand.

Observability is not a product you buy but a property your system has: can you explain a behaviour you did not predict? This lesson defines the three telemetry signals, what question each answers, why their costs grow along completely different axes, and why the difference between monitoring and observability is the difference between known and unknown failure modes.

Payments engineering is mostly failure engineering. This lesson covers the decline taxonomy and what a merchant may do about each kind, how strong customer authentication and 3-D Secure actually work including the exemptions with their 0.13 and 0.06 percent fraud thresholds, who eats a fraudulent transaction and when that liability shifts, and the chargeback machine from dispute to representment.

Every card fee argument eventually reaches interchange: the fee the merchant's side pays to the cardholder's bank on every transaction. This lesson decomposes the merchant discount rate, explains why rewards cards exist and who really funds them, covers the EU caps of 0.2 and 0.3 percent and their loopholes, and shows why the same purchase costs a merchant triple on a corporate card.

The tap is instant; the money is not. A card payment runs on three separate clocks: authorization in about a second, clearing in overnight batches, settlement as netted bank transfers. This lesson walks the message flow, explains holds and why they linger, shows why refunds are slow when charges are fast, and follows one transaction through all three phases.

A card payment is a trust machine between four parties who have mostly never met: cardholder, issuer, merchant, acquirer, with a network in the middle that routes messages and sets the rules. This lesson maps who does what, who carries which risk, what a PSP actually is, and the surprising answer to whose money pays the merchant on the day you tap.

A model that has stopped working returns answers with the same confidence as one that still does. This lesson covers the difference between data drift and concept drift, what to monitor when labels arrive months late, the triggers that should start a retraining run, and the cases where retraining is the wrong response.

The most common way a good model fails in production is that the features it is served differ from the ones it was trained on. This lesson covers where that divergence comes from, why point-in-time correctness is harder than it looks, what a feature store actually solves, and when you do not need one.

Google's MLOps guidance describes three maturity levels, from a fully manual handoff to a pipeline that tests and deploys itself. This lesson covers what is automated at each level, the six stages of an ML CI/CD pipeline, and why level 2 is the wrong target for most teams.

A trained model is a small box inside a large system, and the system is what decays. This lesson covers the failure modes that are specific to machine learning: why changing one feature moves every weight, why hidden consumers break silently, and why data dependencies cost more than code dependencies.

An aggregate metric tells you a model is worse than you hoped and nothing about why. Evaluation that writes results back onto each sample turns a number into a set of images you can look at. This lesson covers the evaluation methods and their protocols, per-sample true and false positive counts, how a confusion matrix cell becomes a view, and where this workflow stops.

A query can only ask about what is recorded, so the harder half of curation is generating fields that surface where to look. This lesson covers the analysis methods FiftyOne bundles as the Brain: embedding visualisation and its four reduction methods, uniqueness and representativeness, mistakenness and hardness, and similarity indexes for near-duplicate detection and text search.

A view is a filtered, sorted or sliced window onto a dataset, built by chaining stages and computed lazily without touching the underlying data. This lesson covers what that buys, the distinction between filtering samples and filtering the labels inside them that catches nearly everyone, how views compose with the App and with evaluation, and when a view should be turned into a real subset.

Most computer vision projects are limited by their dataset rather than their architecture, and most teams cannot see their dataset. This lesson covers why a tool for inspecting visual data exists, the data model FiftyOne uses to make a dataset queryable rather than a folder of files, why it sits on a database, and where curation fits against annotation and experiment tracking.

Speech in, text, model, text, speech out is the standard architecture and it discards everything not in the words: emphasis, emotion, hesitation, overlap. End-to-end models keep it by never routing through text, and pay with a token rate roughly 185 times higher and far less training data. This lesson covers the trade, and why interleaving is the pragmatic answer.

Once audio is a sequence of tokens, generating speech becomes the same shape of problem as generating text, and the whole language modelling toolkit transfers. That reframing produced zero-shot voice cloning from about three seconds of audio. This lesson covers the codec language model approach, why so little reference suffices, the flow-matching alternative, and what the capability implies.

Speech recognition's hard problem is that audio and text have different lengths and nobody labelled which frame goes with which letter. CTC, RNN-T and attention encoder-decoders are three answers, and which one a system uses decides whether it can stream. This lesson covers all three, why Whisper's weak supervision worked, and the failure that follows from a recogniser containing a language model.

Before a model can process speech it has to be discretised, and audio resists that harder than text does. This lesson covers why raw waveforms are the wrong representation, how neural audio codecs learn a discrete one, what residual vector quantization actually does, and the arithmetic that governs every speech model: a minute of talking is around 195 text tokens or 36,000 audio tokens.

The exchange is a case study with transferable rules. A conclusion resting on failures needs a failure taxonomy. Every instance must be verified solvable before anyone is scored against it. Output format is a confound whenever answers get long. And when two explanations fit the same data, the productive move is to find the prediction on which they differ, then test it.