All lessons
llm-as-judge
2 lessons tagged llm-as-judge: free, quiz-checked micro-lessons.
AI
intermediateEvaluating AI Agents: From Final Answers to Full Trajectories
A rigorous look at how to measure agent performance — trajectory-level vs final-answer evals, canonical multi-step benchmarks (SWE-bench, WebArena, OSWorld, GAIA), LLM-as-judge pitfalls, and why your eval is your spec.
12 steps·~18 min
AI
intermediateLLM evaluation: how to know your model output is actually good
Why traditional software testing falls apart on LLMs, the four evaluation regimes that work in practice (golden sets, LLM-as-judge, human review, online metrics), and how to wire them together without drowning in ungrounded scores.
8 steps·~12 min