Loop engineering: verification, orchestration, and anti-patterns

A loop finishes when the model says "done." But "done" doesn't mean "right." In production the model is wrong about being done somewhere between 5% and 40% of the time — confidently. Verification is the practice of checking the agent's claim before treating it as the answer.

Three shapes show up most often:

• Mechanical check — does the code compile, does the test pass, does the output match the schema? Cheap, deterministic, no LLM.
• Judge panel — one or more separate LLM calls prompted to refute the claim. Catches plausible-but-wrong answers a mechanical check can't.
• Human check — for high-stakes claims. Slow, expensive, and the only one with no ceiling.

Rule of thumb: never ship an autonomous loop without at least one of the first two.

Adversarial verification spawns N independent verifier calls, each prompted to refute the claim. Accept the claim only if a majority cannot refute it.

def verify(claim, n=3):
    votes = [judge_refute(claim) for _ in range(n)]
    refuted = sum(1 for v in votes if v.refuted)
    return refuted < n / 2

Two design choices matter:

• Refute, don't confirm. "Try to find what's wrong" produces stronger checks than "is this correct?" — models are biased toward agreement.
• Independent prompts. N copies of the same prompt is redundant. Better: each verifier gets a different angle (correctness, security, reproducibility). Diversity beats quantity.

This pattern is what turns a 60% solo-pass rate into 85% verified output on real coding-agent tasks.

A claim isn't accepted until a majority of independent judges fail to refute it.

A sub-agent is a loop that another loop spawns. The orchestrator hands off a well-scoped sub-task with its own prompt, tool list, and budget; gets back one result; appends it to its own state.

Why it works:

• Context isolation — the orchestrator's prompt stays small even if the sub-task is verbose.
• Tool isolation — the sub-agent only sees the tools it needs (a code-runner doesn't see the email-sender).
• Budget isolation — sub-tasks have their own iteration caps; one runaway sub doesn't break the whole run.

When to reach for it: tasks that fan out ("verify each of these ten claims"), tasks that need a different model (a cheaper one for narrow sub-tasks), or tasks long enough that the orchestrator's context would otherwise blow up. When not to: anything you could solve in one well-prompted call. Sub-agents are a coordination tax.

For unbounded discovery tasks (find all bugs, list every edge case, gather every relevant source) a fixed iteration cap is wrong: you don't know how many real findings exist. The right shape is loop-until-dry:

findings, dry_streak = [], 0
while dry_streak < K:
    new = run_round(seen=findings)
    fresh = [x for x in new if x not in findings]
    if not fresh:
        dry_streak += 1
    else:
        dry_streak = 0
        findings += fresh

Keep iterating until K consecutive rounds produce nothing new (K is typically 2-3). The counter resets on any non-empty round. This catches long tails — round 7 produces nothing, round 8 produces three findings — that a fixed cap would silently miss.

Gotcha: dedup correctly. A finding judged "rejected" must stay in the seen-set, or it reappears every round and the loop never converges.

For tasks where depth scales with what the user paid for, use loop-until-budget: keep iterating until cumulative cost is within a stop-margin of the cap.

results = []
while budget.remaining() > 50_000:  # tokens
    results += run_round()

This is the right shape when:

• The user signed a token target ("+500k") and expects depth to scale.
• The work is monotonically additive (more iterations = more / better findings).
• A premature stop is worse than the marginal cost.

Two guards you need: a floor (budget.total must be set; without it remaining() is infinity and the loop runs to the iteration cap), and a stop margin (don't iterate down to zero; leave enough room for the final write-up turn).

When one search angle won't find everything, run several in parallel with different prompts. For a code review: one finder looks for correctness bugs, one for security holes, one for performance regressions, one for missing edge cases. Each is blind to what the others see.

Why diverse beats redundant:

• A correctness-focused prompt doesn't notice a SQL injection. A security-focused prompt doesn't notice an off-by-one.
• Three identical prompts produce three correlated answers (~one independent draw). Three different prompts produce three independent ones.

DIMENSIONS = ["correctness", "security", "perf", "edge_cases"]
findings = parallel(run_finder(d) for d in DIMENSIONS)

Dedup across dimensions on file + line before downstream verification. Combined recall is dramatically higher than any single finder.

You don't pick a loop's caps from intuition; you measure. For your task, run the loop at N=1, 3, 5, 10, 20 iterations and plot success rate.

The curve almost always plateaus. The cheap, surprising lesson: most agent loops you see in the wild are running 10× past their plateau, paying for nothing.

N  | success | cost
---|---------|------
1  |   42%   | 0.04
3  |   71%   | 0.12
5  |   78%   | 0.22
10 |   79%   | 0.48
20 |   79%   | 1.10

This loop's right cap is N=5. Past that you're burning tokens for noise. Re-run the table when the model changes, when the toolset changes, or when the task distribution shifts. "We've always set it to 20" is not eval-driven; it's superstition.

Loop-shaped failure modes that recur across teams:

• Silent caps — the loop truncates findings at N without logging it. Reads as "we covered everything"; actually skipped a tail. Always log when a cap fires.
• Infinite plan loops — the model replans every turn but never acts. Detect: an iteration produced zero tool calls and the plan changed.
• Accumulator drift — history grows turn after turn but the model's actions converge on a single repeated state. Compare the last K plans; if they're effectively identical, exit.
• Premature termination — the model says "done" with the wrong answer because the success signal is too easy to fake. Fix with verification, not with more prompting.
• Tool-call thrashing — the same call with the same args, every iteration. Hash recent calls; on collision, exit and surface the trace.

Most "the agent failed" incidents trace back to one of these. Each is also cheap to detect; the bug is usually that nobody added the check.