InterviewsVector
Arc 8
Design reviewAdvanced100 min estimateOriginal publication

When Multiple Agents Are Actually Justified

Multiple prompts with job titles are not an architecture. Add agent boundaries only when independently owned work, isolated authority, parallel execution, or context partitioning repays coordination cost.

Authorship
InterviewsVector
Published / updated
2026-09-21 / 2026-09-21
Review status
Artifact tests passing · primary sources recorded

Original InterviewsVector teaching. Executable artifacts are deterministic illustrative audits with focused tests and recorded primary sources; they do not claim production durability, performance, penetration testing, or security certification.

The decision in one pass

Start with the simplest evaluated baseline: deterministic code, one model call, a fixed workflow, or one bounded agent. Introduce multiple agents only when the task contains at least two genuinely independent work fronts and a concrete boundary supplies value: different owners or credentials must be isolated, branches can run concurrently, or evidence does not fit one usable context and can be compressed behind typed outputs. Draw a dependency DAG, give every unit one objective, input contract, tool scope, budget, output schema, stop condition, and owner, then make one deterministic aggregation boundary responsible for conflicts and completion. Estimate critical-path latency, total tokens, duplicated context, coordination overhead, and failure amplification. Compare the design with the single-agent baseline on the same held-out tasks and budgets. A persona, a different system prompt, or a desire for an impressive diagram is not sufficient justification.

Why this matters

Every additional agent adds another probabilistic loop, context boundary, queue, credential surface, retry domain, and opportunity for inconsistent partial progress. Parallel branches can reduce wall-clock time or expand effective working context, but dependent chatty agents can increase both latency and cost while hiding ownership. The design decision should be reversible and evidence-bound before coordination becomes the product's accidental core.

You will be able to

  • Distinguish deterministic workflows, single agents, parallel model calls, and multi-agent systems by control flow rather than naming.
  • Use dependency structure, ownership, credential isolation, and context pressure to justify boundaries.
  • Estimate critical path, total work, duplicated context, aggregation cost, and failure amplification.
  • Define typed handoffs, conflict policy, verification, retry ownership, and terminal conditions.
  • Evaluate a multi-agent candidate against simpler baselines without generalizing from vendor-specific demonstrations.

Your Vector Loop for this lab

  1. 01

    Model

    Map the task DAG, evidence volume, mutable resources, security scopes, owners, deadlines, and acceptance test before naming agents.

  2. 02

    Derive

    Derive independent fronts, context budgets, isolation requirements, aggregation work, critical path, total work, and failure ownership.

  3. 03

    Build

    Build a content-bound fork/join decision record with typed units and a single deterministic read-only merge.

  4. 04

    Stress

    Collapse branches, add dependencies, duplicate evidence, poison estimates, violate credential isolation, overload aggregation, and bypass constructors.

  5. 05

    Operate

    Compare against one call, one bounded agent, and fixed-workflow baselines; observe queueing, tokens, conflicts, retries, abandoned workers, and merge defects.

  6. 06

    Defend

    Defend every agent boundary with measured local value and preserve a rollback to the simplest design that meets the release bar.

Climb a complexity ladder, not a persona ladder

Agent count is an implementation choice, not a capability metric. A deterministic function is strongest when the path is known. A single model call can classify or transform bounded input. A fixed workflow can route, fan out, vote, or evaluate without giving a model control of the graph. One agent fits open-ended work inside one authority and state boundary. Multiple agents become a candidate only after the task exposes boundaries that one loop cannot handle cleanly.

DesignUse whenStop signal
deterministic coderules and transformations are specifiablemodel adds variance without needed judgment
one model callbounded context and one typed output are enoughno iterative environmental feedback is needed
fixed workflowstages or parallel branches are known in advancea model need not invent the topology
single bounded agentsteps are unknown but authority and context fit one ownerextra workers would share the same state and tools
multiple agentsindependent work benefits from isolation, parallelism, or separate contextcoordination consumes the expected gain

Demand a concrete reason for decomposition

SignalEvidence to collectCounterexample
parallelismindependent branches and measured critical-path reductionworkers serialize on one dependency or rate limit
context partitioneach branch fits; typed summaries preserve required evidenceevery worker needs the full shared context
authority isolationseparate credentials, sandboxes, tenants, or write scopesonly the prompt says not to use another tool
ownershipdifferent services or teams own stable contracts and incidentsone team owns a chatty internal loop

These signals can combine, but none is automatic proof. Parallel calls may hit the same external bottleneck. Separate contexts may lose cross-branch facts. Credential isolation helps only when runtimes issue attenuated tokens and prevent lateral access. Organizational ownership can warrant a service boundary without warranting an autonomous agent on both sides.

  • Require at least two work units that can begin without consuming one another's generated output.
  • Make dependencies explicit in a DAG; a long serial chain is prompt chaining, even if every node is called an agent.
  • Assign each mutable resource one writer or an explicit concurrency protocol. Natural-language consensus is not a lock.
  • Let workers return bounded typed evidence; the aggregator owns deduplication, disagreement, missing branches, and final acceptance.

Estimate both elapsed time and total work

Tcritical ≈ max(Tbranch₁ … Tbranchₙ) + Tmerge; Wtotal = Σ(tokensᵢ + toolᵢ) + coordination

Ideal fork/join latency follows the slowest branch plus aggregation, but total work sums all branches. Queueing, shared rate limits, retries, and stragglers make the latency expression optimistic.

Context pressure is a capacity argument only when the task can be partitioned. Three workers each receiving the same giant dossier triple input work without expanding useful context. A better design gives each worker a bounded evidence shard and requires citations or content identities in its output so the merge can recover provenance rather than trusting a lossy narrative.

First-party engineering reports can motivate a hypothesis, not set your release bar. Anthropic describes a research product where parallel subagents explore independent directions and also reports materially higher token use; that product-specific observation does not establish that a multi-agent topology improves coding, support, or your task distribution. Reproduce the comparison locally with the same models, tools, data, timeouts, and stopping rules.

Turn the topology argument into a reviewable record

multi_agent_decision.py
1def audit_decomposition(
2 contract: DecisionContract, record: DecompositionRecord
3) -> DecisionReport:
4 """Decide whether a bounded fork/join plan justifies separate agent contexts."""
5 contract = validate_record(contract, DecisionContract)
6 record = validate_record(record, DecompositionRecord)
7 if record.scope != contract.scope or record.contract_content_id != contract.content_id:
8 raise ValueError("decomposition belongs to another decision contract")

Expected output

example=illustrative_only
decision=MULTI_AGENT_JUSTIFIED
branches=3;critical_path_units=2
signals=parallel-frontier,context-pressure,credential-isolation,bounded-coordination
estimated_tokens=27000;coordination_ratio=0.111
claim=LOCAL_DECISION_RECORD_NOT_MULTI_AGENT_BENCHMARK

Verify: python3 -m unittest discover courses/ai-engineering/reference-impl/multi_agent_justification

The invented supplier-risk fixture has three independent read-only branches for policy, finance, and security evidence. Each has a distinct owner, tool domain, credential isolation key, context estimate, and typed finding schema. A fourth read-only unit depends on all three and emits the decision record. Their declared branch context exceeds one local context budget while each branch fits; the merge is below the contract's coordination-ratio threshold.

The audit accepts only this bounded fork/join shape. It reconstructs frozen records, checks exact contract scope, unique contiguous positions, backward dependencies, shared mutable isolation, a complete read-only merge, worker and context limits, finite exact-float ratios, exact booleans and strings, constructor bypass, and collection copying. Branches may share an isolation key only when every branch sharing it is read-only; mutable collisions require an explicitly serialized design outside this flat fork. `MULTI_AGENT_JUSTIFIED` is the result of an invented heuristic over estimates. The artifact runs no agent, measures no latency or quality, prices no tokens, and makes no benchmark claim.

Make handoffs smaller than the work they coordinate

  1. 01Contract the branchDeclare objective, admissible evidence, context cap, tools, credentials, side effects, deadline, output schema, and completion test.
  2. 02Bind the taskGive each worker a task and evidence digest so stale or cross-run output cannot enter the merge.
  3. 03Collect partial outcomesDistinguish success, abstention, timeout, policy denial, and malformed output; do not let silence masquerade as no finding.
  4. 04Resolve conflicts deterministicallyDefine evidence priority, duplicate identity, disagreement escalation, quorum if justified, and the sole owner of final acceptance.
  5. 05Stop the graphCap worker count, depth, retries, tokens, tool calls, and elapsed time; cancellation must reach children and their external effects.

Retries need one owner. If both orchestrator and worker retry the same branch, multiplicative attempts can exceed budgets and repeat effects. A child that can write needs its own idempotency and recovery contract; a parent retry does not erase an ambiguous child outcome. Persist lineage from final claims to branch evidence and tool receipts so a bad synthesis can be separated from a bad source.

Release the topology only if it beats a simpler baseline

MeasureCompareSlice
task success and verificationone call, fixed workflow, one agent, multi-agentdependency depth and evidence breadth
critical-path latencyend-to-end plus branch and merge spansnormal, straggler, timeout, degraded mode
total costinput/output tokens, tool calls, computesuccess, retry, abandoned work
coordination defectsmissing, duplicate, stale, conflicting handoffsbranch and aggregator version
security surfacecredentials, writable resources, cross-tenant denialstool domain and worker identity

Use paired tasks and identical acceptance tests. Hold the model family, tool versions, corpus snapshot, budgets, and timeout policy constant enough to attribute the effect of topology. Report confidence intervals or repeated-run distributions when sampling variance matters. A win on one curated task is not a universal multi-agent advantage, and an internal vendor evaluation is not a transferable baseline.

Operate at three altitudes

Production lens

  • — Trace branch lineage, queue time, model/tool versions, retries, tokens, typed outcome, cancellation, and merge decision under one task identity.
  • — Alert on orphan workers, fan-out growth, repeated or stale handoffs, conflict rate, missing branches, cross-scope tool denials, and merge bottlenecks.
  • — Evaluate degraded modes explicitly: fail closed, return partial evidence with disclosure, or escalate; never silently synthesize a complete answer from missing required branches.

Staff lens

  • — Treat agent topology as an owned, versioned resource and security design with capacity limits, incident boundaries, and an evidence-backed simplification path.
  • — Require a local baseline and costed decision record before creating a platform whose primary output is more agents, queues, and coordination protocols.

Interview defense

A team proposes five specialist agents to improve a research workflow. How would you decide whether the design is justified?

I would first benchmark a fixed workflow and one bounded agent. Then I would draw the dependency DAG and identify genuinely independent work, evidence that exceeds one usable context, different credential or sandbox boundaries, and stable ownership seams. For each branch I would specify inputs, tools, budgets, output schema, termination, and failure owner, with one deterministic aggregator. I would estimate slowest-branch critical path and total tokens/tool work, including duplicated context and retries. On paired held-out tasks I would compare quality, verified completion, latency distribution, cost, handoff defects, and security scope. I would keep multiple agents only for measured local value, not because the prompts have different personas.

Expect the interviewer to press on

  • — When is parallel model calling a workflow rather than a multi-agent system?
  • — How can credential isolation be real rather than prompt-level advice?
  • — What happens when one required branch times out after others have completed?

Misconceptions to remove

“Different personas provide independent agents.”

Independence comes from task, state, context, tools, credentials, ownership, and failure boundaries. Labels alone enforce none of them.

“Parallel agents always reduce cost because they finish sooner.”

Parallelism can reduce elapsed time while increasing total token, tool, and coordination work; shared bottlenecks can erase even the latency gain.

“A majority vote verifies a result.”

Votes can share the same model, context, and systematic error. Verification needs task-grounded evidence, calibrated independence assumptions, and an explicit decision rule.

Check your model

1. Why is a serial chain of three model calls not evidence for three agents?

Its topology is a known dependency chain that a fixed workflow can own; separate autonomy adds no parallel front or isolation by itself.

2. What two cost views should a fork/join estimate include?

Critical-path elapsed work—roughly the slowest branch plus merge—and total work across every branch, tool, retry, and coordination step.

3. When does context partitioning fail as a justification?

When every worker needs the same full context or the typed summaries discard cross-branch evidence required for correct aggregation.

Prove the mechanism

Create a decision record for an incident investigation with logs, deployment history, and customer reports. Compare a fixed parallel workflow, one agent, and isolated agents; define branch evidence schemas, a missing-branch policy, and the exact measurement that would collapse the topology.

Add a production constraint

Add a dependent second wave to the decision artifact without calling it a flat fork/join. Compute critical-path units over the DAG, constrain fan-out and credentials per wave, detect cycles, and preserve deterministic aggregation and rollback to a fixed workflow.

Artifact: Multi-agent decomposition decision record

courses/ai-engineering/reference-impl/multi_agent_justification/multi_agent_decision.py

Download reference implementation

Primary references and next links

References

  1. 1. Building Effective AI Agents

    Anthropic. First-party engineering guidance distinguishing workflows and agents and describing parallel and orchestrator-worker patterns.

  2. 2. How we built our multi-agent research system

    Anthropic. A product-specific first-party report on parallel context, token cost, orchestration, and unsuitable tightly coupled tasks; its internal results are not generalized here.

  3. 3. Why Do Multi-Agent LLM Systems Fail?

    Cemri et al.. Primary empirical study organizing observed multi-agent failures around system design, inter-agent alignment, and task verification or termination.

Continue through the graph

Glossary: multi-agent system · fork/join · dependency DAG · critical path · coordination overhead · context partitioning · credential isolation · typed handoff