Subagents and Multi-Agent Orchestration Patterns

A single LLM call has a context window ceiling, a fixed inference budget, and no memory across turns. Multi-agent orchestration sidesteps all three by decomposing a large task into units that fit comfortably inside one model call each, then stitching results together.

The mental model: think of it like a process supervisor (supervisord, Kubernetes, a build graph). An orchestrator issues tasks; subagents execute them. The orchestrator can be another LLM, a deterministic scheduler, or plain application code. Subagents are usually stateless LLM calls — sometimes tool-using, sometimes pure text.

This decomposition unlocks parallelism, specialization, and iterative refinement. But every subagent invocation costs tokens, latency, and coordination complexity. The central engineering question is: does the output quality gain exceed that cost?

Each topology suits a different failure mode. Fan-out beats latency when subtasks are independent. Pipelines handle progressive enrichment. Judge panels reduce variance. Adversarial pairs catch hallucinations. Loop-until-dry drains a changing environment until no work remains.

Fan-out means dispatching $N$ independent subtasks simultaneously and merging the results. The canonical use case: summarize 50 documents for a synthesis report. With a single agent you'd either blow the context window or serialize calls (slow). With fan-out you fire all 50 in parallel and hit the wall-clock time of the slowest one.

In Python with the Anthropic SDK and asyncio:

import asyncio, anthropic

client = anthropic.AsyncAnthropic()

async def summarize(doc: str) -> str:
    msg = await client.messages.create(
        model="claude-opus-4-5",
        max_tokens=512,
        messages=[{"role": "user", "content": f"Summarize:\n{doc}"}],
    )
    return msg.content[0].text

async def fan_out(docs: list[str]) -> list[str]:
    return await asyncio.gather(*[summarize(d) for d in docs])

Heads up: asyncio.gather propagates the first exception by default. Wrap each call in a try/except and return a sentinel if you need fault-tolerant fan-out.

The reduce step is often a second LLM call that receives all summaries and produces a final synthesis — keeping any single call well inside context limits.

A pipeline threads the output of one stage into the input of the next. Unlike fan-out, stages are sequential by design because each one depends on what came before. Common examples:

1. Extract structured data from raw HTML
2. Validate extracted fields against a schema
3. Enrich with external lookups
4. Generate a final report

async def pipeline(raw_html: str) -> str:
    # Stage 1 — extraction
    extracted = await call_model(EXTRACT_PROMPT, raw_html)
    # Stage 2 — validation (deterministic or LLM)
    validated = validate_json(extracted)          # raises on bad output
    # Stage 3 — enrichment
    enriched = await call_model(ENRICH_PROMPT, validated)
    # Stage 4 — report
    return await call_model(REPORT_PROMPT, enriched)

Each stage has a narrow interface: a well-defined input schema and a well-defined output schema. This is the discipline that makes pipelines debuggable. When stage 3 fails you know exactly what it received — unlike a monolithic prompt where failure is diffuse.

Pipeline cost is additive: 4 stages = 4× the calls. Only add a stage when its transformation is genuinely hard to fold into an adjacent one.

A judge panel runs the same task through $N$ independent subagents and aggregates their answers. The goal is variance reduction: any one model call can hallucinate or choose arbitrarily among equally valid options; a panel makes systematic errors much less likely.

Two aggregation strategies:

• Majority vote — pick the answer that appears most often (works for classification, multiple-choice, numeric extraction).
• Synthesis judge — a meta-agent reads all $N$ drafts and produces a final answer, explicitly noting disagreements.

async def judge_panel(question: str, n: int = 3) -> str:
    drafts = await asyncio.gather(
        *[call_model(question) for _ in range(n)]
    )
    synthesis_prompt = (
        "You are a judge. Here are "
        + str(n)
        + " independent answers:\n"
        + "\n---\n".join(drafts)
        + "\nProduce the single best answer, noting any disagreements."
    )
    return await call_model(synthesis_prompt)

Panels multiply token cost by $N$ (plus the synthesis call). They earn their cost when wrong answers are expensive — legal analysis, medical triage, financial modelling — and when you can't use external ground truth to verify.

The adversarial pair is a two-agent loop: a generator produces an answer; a critic tries to break it. If the critic finds a flaw, the generator revises. Repeat until the critic passes or a max-iterations guard fires.

async def adversarial_loop(
    task: str, max_rounds: int = 4
) -> str:
    answer = await call_model(f"Complete this task: {task}")
    for _ in range(max_rounds):
        critique = await call_model(
            f"Task: {task}\nDraft: {answer}\n"
            "List specific errors or output PASS if correct."
        )
        if critique.strip().upper().startswith("PASS"):
            return answer
        answer = await call_model(
            f"Task: {task}\nDraft: {answer}\n"
            f"Critique: {critique}\nRevise the draft."
        )
    return answer  # best effort after max rounds

This pattern shines for verifiable outputs — code that must compile, SQL that must parse, JSON that must validate. When correctness is easy to check programmatically, replace the LLM critic with a hard assertion; LLM-as-critic is only necessary when correctness is semantic.

Common gotcha: the critic and generator sharing the same model weights are likely to share the same blind spots. Using a different model family for the critic is meaningfully better.

Loop-until-dry is an agent that repeatedly polls an environment (inbox, queue, feed, filesystem) and processes items until none remain. The loop condition is external state, not a counter.

async def drain_inbox(inbox_url: str) -> None:
    while True:
        items = await fetch_new_items(inbox_url)
        if not items:
            break
        await asyncio.gather(*[process(item) for item in items])
        await asyncio.sleep(POLL_INTERVAL_S)

This is the backbone of autonomous email assistants, CI triage bots, and data-pipeline catch-up jobs. Key concerns:

• Idempotency — processing the same item twice must be safe. Mark items as processed before starting work, or use an at-least-once queue with deduplication.
• Poison pills — one malformed item must not block the rest. Wrap process(item) in a try/except and dead-letter bad items.
• Cost runaway — an infinite stream means infinite spend. Add a per-run token budget or item cap.

The max-rounds guard is non-optional. Without it, a generator-critic pair that disagrees indefinitely will run until your API quota is exhausted. Four rounds is usually enough; diminishing returns set in fast because both agents share similar priors.

Subagents justify their cost when at least one of the following holds:

• Context overflow — the task literally cannot fit in one call (e.g., 200-page document analysis).
• Latency-sensitive parallelism — fan-out reduces wall-clock time by 10×+ and that matters to users.
• Variance is expensive — panel voting or adversarial review catches errors that would cost real money to fix (wrong drug dosage, wrong SQL on prod).
• Specialization improves quality — a dedicated extraction agent with a tight system prompt outperforms a generalist prompt by a measurable margin (run an eval to confirm).
• The environment keeps changing — loop-until-dry is the only sensible model for draining a live queue.

A rough heuristic: if a single expert human would read the whole task in one sitting and produce the answer, a single LLM call probably suffices. Subagents add overhead and coordination failure modes — you've now got N+1 things that can go wrong.

Architects reach for multi-agent patterns out of habit or hype more often than necessity. Red flags that you're over-engineering:

• The task fits in context — if everything fits in 4K tokens, a second call is pure waste.
• Stages share so much context they're redundant — if stage 2 just reformats stage 1's output, fold it into stage 1's prompt.
• The reducer is harder than the original task — synthesizing 50 poorly-scoped summaries is worse than one well-scoped prompt over the originals.
• You haven't measured quality improvement — running an eval is mandatory before committing to a panel. Anecdotal confidence is not evidence.
• Latency doesn't matter — async fan-out still has overhead (API round-trips, queuing). If you're running a nightly batch and nobody's waiting, serialize.

The honest default: start with a single, well-crafted prompt. Add orchestration only when a specific, measured deficiency demands it.

The weakest link in any multi-agent system is the inter-agent interface. Agents communicate via structured data; unstructured free text propagates ambiguity and model errors across stages.

Best practices:

from pydantic import BaseModel

class ExtractionResult(BaseModel):
    company_name: str
    revenue_usd: float | None
    fiscal_year: int

# Force structured output
response = client.messages.create(
    model="claude-opus-4-5",
    max_tokens=256,
    tools=[{
        "name": "submit_extraction",
        "description": "Submit the extracted fields.",
        "input_schema": ExtractionResult.model_json_schema(),
    }],
    tool_choice={"type": "any"},
    messages=[{"role": "user", "content": raw_text}],
)

With Pydantic + tool-use forced output, the next stage receives a typed Python object, not a string it has to parse. Schema validation at each boundary is what separates a multi-agent system that degrades gracefully from one that silently propagates corrupted state.

Store intermediate outputs in a content-addressed store (S3 key = sha256(input)) so you can replay any stage without re-running the whole pipeline.

A multi-agent system with no observability is a black box that burns money in the dark. Minimum viable telemetry:

• Trace ID — assign a UUID to the root orchestrator call and thread it through every subagent. Log {trace_id, stage, model, input_tokens, output_tokens, latency_ms, error}.
• Token budget guards — track cumulative token spend per trace. Abort if it exceeds a configured threshold.
• Stage-level evals — don't only eval the final output. Instrument each stage independently so you know which stage degrades quality.

import structlog
log = structlog.get_logger()

async def instrumented_call(stage: str, prompt: str, trace_id: str) -> str:
    resp = await client.messages.create(...)
    log.info("stage_complete",
        trace_id=trace_id, stage=stage,
        input_tokens=resp.usage.input_tokens,
        output_tokens=resp.usage.output_tokens,
    )
    return resp.content[0].text

For production systems, pipe logs to a time-series DB and alert on per-trace cost spikes — an orchestration bug that infinite-loops is much more expensive than a single-call bug.