harness-engineering
4 free lessons tagged harness-engineering across AI. Each one is a short sequence of focused steps with narration and a five-question quiz at the end — take them in any order, no signup required.
Measuring a Harness Change: Trials, Error Bars and pass^k
Every harness change is a hypothesis. Learn how evaluation harnesses record agent runs, why one run is one noisy sample, how paired comparisons and repeated trials separate real gains from luck, and when to report pass@k versus pass^k, with computed simulations.
Permissions, Sandboxes and Hooks: Containing an AI Agent
An agent with a shell can make destructive mistakes or follow injected instructions. Learn the four layers of control, from instruction files to OS sandboxes, why command-pattern rules are not a boundary, how hooks add deterministic checks, and how to design out the lethal trifecta.
The Repository as the Agent's Environment
Coding agents only know what the repository shows them. Learn to write instruction files as maps, turn conventions into checks with fix-it messages, give agents sensors, and structure work that spans many sessions, using OpenAI's, Anthropic's and ETH Zurich's findings.
Model Plus Harness: Why the Same Model Scores Differently
An agent is a model plus everything around it. Using controlled evidence from SWE-agent and Terminal-Bench, see how interface design, harness choice and even container limits move the results of a fixed model, and why the model still sets the ceiling.