metrics
7 free lessons tagged metrics across Computer Science, Programming, Business, AI. Each one is a short sequence of focused steps with narration and a five-question quiz at the end — take them in any order, no signup required.
Measuring Relevance: Judgments, NDCG, and the Click Trap
Search quality arguments end when there is a number, and begin again over whether the number is honest. This lesson builds offline evaluation from its atoms: a judgment set, precision and recall at k, MRR for known-item queries, and NDCG computed by hand for graded relevance. Then the online half: clicks, position bias, and why the top result gets clicked even when it is wrong.
Percentiles and Cardinality: The Two Numbers That Run Your Bill
Two pieces of arithmetic decide whether your telemetry is useful and affordable. Percentiles, because averages hide exactly the users who are suffering, and you cannot average a p99. Cardinality, because metric cost is not per event but per label combination, and one careless label can multiply your bill by the size of your user base. This lesson does both mechanisms by hand.
Metrics, Logs, Traces: Three Signals, Three Cost Models
Observability is not a product you buy but a property your system has: can you explain a behaviour you did not predict? This lesson defines the three telemetry signals, what question each answers, why their costs grow along completely different axes, and why the difference between monitoring and observability is the difference between known and unknown failure modes.
The Five Analyses That Answer Most Business Questions
Almost every business question reduces to one of a small number of analytical shapes. This lesson covers trend, breakdown, funnel, cohort and distribution, what each is good for, how each misleads, and what you may and may not conclude from any of them.
The Fairness Metrics, and Why You Must Choose
Fairness has several formal definitions that sound equally reasonable and cannot all hold at once. This lesson covers demographic parity, equal opportunity, equalized odds and calibration, the impossibility result proved independently by Chouldechova and by Kleinberg and colleagues, and how to choose one deliberately and defend it.
What the algorithm optimizes, and why it drifts
A feed does exactly what it was told to maximize, which is rarely what anyone wanted. Learn which signals rankers actually weight and why implicit ones beat likes, how proxy objectives produce clickbait and rage-bait as correct answers to badly posed questions, and the mechanisms platforms use to pull an optimizer back toward what people value.
Evaluating RAG Pipelines with RAGAS
A rigorous guide to measuring RAG quality using RAGAS metrics — faithfulness, answer relevancy, context precision, and context recall — plus how to build a golden dataset and recognize where automated metrics fall short.