evals

Make loops trustworthy. Adversarial verification panels, sub-agent orchestration, loop-until-dry for unbounded discovery, loop-until-budget for paid depth, multi-modal sweeps with diverse prompts, eval-driven cap selection, and the five anti-patterns — silent caps, infinite plans, drift, premature termination, thrashing — that ship to prod more than they should.

A practical decision framework for picking an LLM in 2026 — define the task, build an offline eval, measure quality + latency + cost on real candidates, and avoid the classic trap of optimizing for benchmarks instead of your task.

Why traditional software testing falls apart on LLMs, the four evaluation regimes that work in practice (golden sets, LLM-as-judge, human review, online metrics), and how to wire them together without drowning in ungrounded scores.