AnyLearn
All lessons
AIintermediate

The Repository as the Agent's Environment

Coding agents only know what the repository shows them. Learn to write instruction files as maps, turn conventions into checks with fix-it messages, give agents sensors, and structure work that spans many sessions, using OpenAI's, Anthropic's and ETH Zurich's findings.

Updated · AI-authored, review-gated · how lessons are made

Not signed in: your progress and quiz score won't be saved.
Progress1 / 10

If it is not in the repo, it does not exist

A coding agent starts each session knowing nothing about your team. It cannot read the chat thread where you agreed a naming rule, or the reviewer who always rejects a certain pattern. Whatever it needs has to be discoverable from the files and tools in front of it.

OpenAI's Ryan Lopopolo put it bluntly in Harness engineering: leveraging Codex in an agent-first world: "if it isn't discoverable in the repo, it effectively doesn't exist for the agent." His team built an internal product in which, by their account, agents wrote all of the code: roughly a million lines and about 1,500 merged pull requests over five months, with three engineers growing to seven, averaging 3.5 pull requests per engineer per day.

Key idea: In an agent-first codebase the engineer's main output shifts from code to the environment code is written in: documents, checks and feedback the agent can find on its own.

This lesson builds on Model Plus Harness, which showed the harness moves results; here the focus is the part of it that lives in your repository.

Full lesson text

All 10 steps on one page, for reading, reference, and search.

Show

1. If it is not in the repo, it does not exist

A coding agent starts each session knowing nothing about your team. It cannot read the chat thread where you agreed a naming rule, or the reviewer who always rejects a certain pattern. Whatever it needs has to be discoverable from the files and tools in front of it.

OpenAI's Ryan Lopopolo put it bluntly in Harness engineering: leveraging Codex in an agent-first world: "if it isn't discoverable in the repo, it effectively doesn't exist for the agent." His team built an internal product in which, by their account, agents wrote all of the code: roughly a million lines and about 1,500 merged pull requests over five months, with three engineers growing to seven, averaging 3.5 pull requests per engineer per day.

Key idea: In an agent-first codebase the engineer's main output shifts from code to the environment code is written in: documents, checks and feedback the agent can find on its own.

This lesson builds on Model Plus Harness, which showed the harness moves results; here the focus is the part of it that lives in your repository.

2. Guides and sensors

Böckeler's harness engineering essay on martinfowler.com sorts every control you add into two directions and two execution styles.

Computational (deterministic, fast)Inferential (LLM-judged, slower)
Guide (feedforward, before the agent acts)Project templates, code generators, type signaturesInstruction files, design docs, examples
Sensor (feedback, after it acts)Tests, linters, type checkers, structural checksAn LLM reviewer reading the diff

Guides steer the agent before it writes anything. Sensors observe what it did and let it self-correct. Computational controls give the same answer every time and cost almost nothing to run; inferential ones handle judgment calls but are slower and probabilistic.

In practice: Push each rule as far toward the computational column as it will go. A convention that lives only in an instruction file is a request; the same convention enforced by a linter is a fact the agent meets every time it runs the checks.

3. The instruction file is a map, not a manual

Most agents load a repository instruction file at the start of every session: AGENTS.md for Codex and many others, CLAUDE.md for Claude Code. The tempting move is to pour everything into it.

OpenAI's team tried the opposite. Their AGENTS.md is about 100 lines, written as a table of contents that points into a structured docs/ directory of design notes, plans and specs. Lopopolo lists why a monolithic file fails:

  • it crowds out the task: every session pays for every rule, relevant or not
  • the agent starts pattern-matching locally instead of navigating to the right document
  • it rots: nobody notices a stale rule in a 2,000-line file
  • it is hard to verify mechanically, unlike a lint rule

This is progressive disclosure: the always-loaded file says where things are, and the agent reads deeper documents only when a task touches them. The loading cost stays constant as the knowledge base grows.

4. Predict: do context files help?

Gloaguen, Mündler and colleagues at ETH Zurich tested this directly in Evaluating AGENTS.md (2026). They ran several coding agents and models on SWE-bench tasks with LLM-generated context files, and on a new set of issues from repositories whose developers had committed their own.

Predict first

Across agents and models, did adding a context file raise the task success rate?

5. What belongs in the file

Combining both findings gives a short file of things the agent cannot discover on its own, plus pointers:

# AGENTS.md

## Commands (run before declaring any task done)
- `make check` : types, lint, layer rules, unit tests (about 40 s)
- `make e2e`   : browser tests; needs `make dev` running

## Non-obvious rules
- Money is integer cents everywhere. Never use float for amounts.
- UI code never imports from `app/db`; go through `app/service`.
- Migrations are append-only. Never edit a merged migration file.

## Where to look
- Architecture and layers: docs/architecture.md
- How we test: docs/testing.md
- Current plan and open decisions: docs/plans/active.md

Notice what is absent: no directory tree, no description of the framework, no generic advice like "write clean code". Each line either tells the agent how to check its work or states a rule it would plausibly break without being told.

Gotcha: Instruction files shape what the agent tries; they enforce nothing. Claude Code's documentation says so explicitly: instructions in CLAUDE.md do not change what the tool allows. Enforcement needs a check or a permission rule.

6. Turn rules into checks with remediation

The rule "UI never imports the database layer" can be a sentence the agent may skip, or a check it cannot. OpenAI's team enforced such invariants with custom linters and structural tests, and wrote the error messages to include the fix, "turning every violation into a learning opportunity." A minimal version in Python:

import ast
from pathlib import Path

FORBIDDEN = {"app.ui": ["app.db"]}   # ui -> service -> db; no skipping
FIX = ("Move the query into app/service/ (see docs/architecture.md#layers) "
       "and call the service function from the UI instead.")

def check(root="app"):
    errors = []
    for f in Path(root).rglob("*.py"):
        module = ".".join(f.with_suffix("").parts)
        for node in ast.walk(ast.parse(f.read_text())):
            if isinstance(node, (ast.Import, ast.ImportFrom)):
                names = [node.module] if isinstance(node, ast.ImportFrom) else [a.name for a in node.names]
                for src, banned in FORBIDDEN.items():
                    for n in names:
                        if module.startswith(src) and n and any(n.startswith(b) for b in banned):
                            errors.append(f"{f}:{node.lineno}: {src} must not import {n}. {FIX}")
    return errors

Run against a UI file that imports app.db.session, it prints app/ui/orders.py:1: app.ui must not import app.db.session. Move the query into app/service/.... The message names the file, the line, the rule and the remedy, which is everything the agent needs for its next attempt.

7. Give the agent eyes

A sensor only helps if the agent can read it. The more of the running system it can observe, the more of its own mistakes it can catch before a human does.

SensorWhat the agent learnsExample from the sources
Syntax check on editThe edit broke the fileSWE-agent: worth 3 points on SWE-bench Lite
Unit and type checksBehaviour and contracts regressedStandard make check gate
Browser automationThe feature works as a user sees itAnthropic used a Puppeteer MCP server for end-to-end checks
Logs, metrics, tracesRuntime behaviour, latency, errorsOpenAI gave Codex LogQL, PromQL and TraceQL queries against a local stack per worktree
DOM snapshots and screenshotsUI state after an actionOpenAI wired in the Chrome DevTools Protocol

The last two rows show the direction of travel: each worktree boots its own copy of the app with its own observability, so an agent can reproduce a bug, fix it and verify the fix without asking anyone.

Key idea: An agent that can only read code will claim success when the code compiles. An agent that can run the app will notice when the button does nothing.

8. Harnessing work that spans many sessions

Anthropic's pattern from Effective harnesses for long-running agents: state lives in files, not in the context window.

sequenceDiagram
  participant H as Human
  participant I as Initializer agent
  participant R as Repository
  participant C as Coding agent
  H->>I: High-level spec
  I->>R: feature_list.json, all passes false
  I->>R: init.sh, claude-progress.txt, first commit
  C->>R: Read progress file and git log
  C->>R: Run init.sh and a smoke test
  C->>R: Implement one feature, test end to end
  C->>R: Set passes true, commit, update progress

9. Why each artifact exists

Each file in Anthropic's long-running pattern answers a failure they observed when an agent worked across many context windows:

Failure observedHarness fix
Declaring victory early: the agent sees progress and calls the project doneA feature list (over 200 entries for their chat-app clone), all initially failing
One-shotting: trying to build everything in one window, leaving half-finished workAn instruction to work on one feature at a time and leave a clean, mergeable state
Marking features done without testing themEnd-to-end checks through browser automation before flipping passes
Losing the thread between sessionsA progress file plus descriptive git commits, read at the start of each session

The feature list is JSON rather than Markdown for a specific reason given in the post: the model is less likely to inappropriately change or overwrite JSON files. Instructions also forbid removing or editing tests, only flipping the passes field.

Gotcha: A to-do list the agent can freely rewrite is not a guardrail. Pick a format and a rule that make the easy wrong move, deleting the hard item, conspicuous.

10. Entropy: agents copy what they see

Agents learn conventions from the surrounding code, including the bad ones. A shortcut taken once becomes a pattern the next session imitates. Lopopolo reports that before automating this, his team spent 20% of every week, each Friday, cleaning up drift by hand.

Their fix was to write down a small set of golden principles and run recurring cleanup agents that scan for deviations and open refactoring pull requests, a form of garbage collection for the codebase. The same idea works at smaller scale:

  1. When a reviewer rejects a pattern twice, turn the rejection into a lint rule with a remediation message.
  2. When a rule cannot be expressed mechanically, put it in a short principles document the instruction file points to.
  3. Schedule a periodic agent task that checks the codebase against those principles and proposes fixes, reviewed like any other change.

Try it: Take the last three review comments you left on agent-written code. For each, decide: lint rule, principles doc, or neither. The ones that become lint rules will never need to be said again.

Check your understanding

The lesson ends with a 5-question quiz. Take it in the player above to see your score.

  1. What did the ETH Zurich 'Evaluating AGENTS.md' study find about repository context files?
    • They raised success rates by about 20% on average
    • Agents ignored the instructions in them
    • They did not generally improve success rates and raised inference cost by over 20%
    • They only helped when written by an LLM
  2. In Böckeler's framing, a type checker that runs after the agent edits code is a:
    • Computational sensor
    • Inferential guide
    • Computational guide
    • Inferential sensor
  3. Why did OpenAI's team keep AGENTS.md to about 100 lines?
    • Codex rejects instruction files longer than 100 lines
    • A short map pointing into docs/ avoids crowding out the task and rotting, while deeper docs load on demand
    • Longer files cannot be committed to git
    • The model ignores anything after line 100
  4. In Anthropic's long-running agent pattern, why is the feature list stored as JSON?
    • JSON is faster for the model to parse
    • Puppeteer only reads JSON
    • Git cannot diff Markdown files
    • The model is less likely to inappropriately change or overwrite JSON than Markdown
  5. What makes a custom lint error useful to an agent rather than just a human?
    • It is printed in colour
    • It names the file, line, violated rule and the concrete remedy
    • It links to a style guide homepage
    • It blocks the commit without explanation

Related lessons