llm-as-judge
3 free lessons tagged llm-as-judge across AI. Each one is a short sequence of focused steps with narration and a five-question quiz at the end — take them in any order, no signup required.
Filtering: The Half That Decides Quality
Generation is the cheap half. What you discard determines what the student learns. This lesson orders the filters by strength: machine verification where an answer can be checked, self-consistency where it cannot, LLM-as-judge with its known position and length biases, and cheap heuristics. It ends on contamination, the failure that invalidates results rather than degrading them.
Evaluating AI Agents: From Final Answers to Full Trajectories
A rigorous look at how to measure agent performance — trajectory-level vs final-answer evals, canonical multi-step benchmarks (SWE-bench, WebArena, OSWorld, GAIA), LLM-as-judge pitfalls, and why your eval is your spec.
LLM evaluation: how to know your model output is actually good
Why traditional software testing falls apart on LLMs, the four evaluation regimes that work in practice (golden sets, LLM-as-judge, human review, online metrics), and how to wire them together without drowning in ungrounded scores.