When Multiple Agents Are Actually Justified
Multiple prompts with job titles are not an architecture. Add agent boundaries only when independently owned work, isolated authority, parallel execution, or context partitioning repays coordination cost.
- Authorship
- InterviewsVector
- Published / updated
- 2026-09-21 / 2026-09-21
- Review status
- Artifact tests passing · primary sources recorded
Original InterviewsVector teaching. Executable artifacts are deterministic illustrative audits with focused tests and recorded primary sources; they do not claim production durability, performance, penetration testing, or security certification.
The decision in one pass
Start with the simplest evaluated baseline: deterministic code, one model call, a fixed workflow, or one bounded agent. Introduce multiple agents only when the task contains at least two genuinely independent work fronts and a concrete boundary supplies value: different owners or credentials must be isolated, branches can run concurrently, or evidence does not fit one usable context and can be compressed behind typed outputs. Draw a dependency DAG, give every unit one objective, input contract, tool scope, budget, output schema, stop condition, and owner, then make one deterministic aggregation boundary responsible for conflicts and completion. Estimate critical-path latency, total tokens, duplicated context, coordination overhead, and failure amplification. Compare the design with the single-agent baseline on the same held-out tasks and budgets. A persona, a different system prompt, or a desire for an impressive diagram is not sufficient justification.
Why this matters
Every additional agent adds another probabilistic loop, context boundary, queue, credential surface, retry domain, and opportunity for inconsistent partial progress. Parallel branches can reduce wall-clock time or expand effective working context, but dependent chatty agents can increase both latency and cost while hiding ownership. The design decision should be reversible and evidence-bound before coordination becomes the product's accidental core.
You will be able to
- Distinguish deterministic workflows, single agents, parallel model calls, and multi-agent systems by control flow rather than naming.
- Use dependency structure, ownership, credential isolation, and context pressure to justify boundaries.
- Estimate critical path, total work, duplicated context, aggregation cost, and failure amplification.
- Define typed handoffs, conflict policy, verification, retry ownership, and terminal conditions.
- Evaluate a multi-agent candidate against simpler baselines without generalizing from vendor-specific demonstrations.
Your Vector Loop for this lab
- 01
Model
Map the task DAG, evidence volume, mutable resources, security scopes, owners, deadlines, and acceptance test before naming agents.
- 02
Derive
Derive independent fronts, context budgets, isolation requirements, aggregation work, critical path, total work, and failure ownership.
- 03
Build
Build a content-bound fork/join decision record with typed units and a single deterministic read-only merge.
- 04
Stress
Collapse branches, add dependencies, duplicate evidence, poison estimates, violate credential isolation, overload aggregation, and bypass constructors.
- 05
Operate
Compare against one call, one bounded agent, and fixed-workflow baselines; observe queueing, tokens, conflicts, retries, abandoned workers, and merge defects.
- 06
Defend
Defend every agent boundary with measured local value and preserve a rollback to the simplest design that meets the release bar.
Climb a complexity ladder, not a persona ladder
Agent count is an implementation choice, not a capability metric. A deterministic function is strongest when the path is known. A single model call can classify or transform bounded input. A fixed workflow can route, fan out, vote, or evaluate without giving a model control of the graph. One agent fits open-ended work inside one authority and state boundary. Multiple agents become a candidate only after the task exposes boundaries that one loop cannot handle cleanly.
| Design | Use when | Stop signal |
|---|---|---|
| deterministic code | rules and transformations are specifiable | model adds variance without needed judgment |
| one model call | bounded context and one typed output are enough | no iterative environmental feedback is needed |
| fixed workflow | stages or parallel branches are known in advance | a model need not invent the topology |
| single bounded agent | steps are unknown but authority and context fit one owner | extra workers would share the same state and tools |
| multiple agents | independent work benefits from isolation, parallelism, or separate context | coordination consumes the expected gain |
Demand a concrete reason for decomposition
| Signal | Evidence to collect | Counterexample |
|---|---|---|
| parallelism | independent branches and measured critical-path reduction | workers serialize on one dependency or rate limit |
| context partition | each branch fits; typed summaries preserve required evidence | every worker needs the full shared context |
| authority isolation | separate credentials, sandboxes, tenants, or write scopes | only the prompt says not to use another tool |
| ownership | different services or teams own stable contracts and incidents | one team owns a chatty internal loop |
These signals can combine, but none is automatic proof. Parallel calls may hit the same external bottleneck. Separate contexts may lose cross-branch facts. Credential isolation helps only when runtimes issue attenuated tokens and prevent lateral access. Organizational ownership can warrant a service boundary without warranting an autonomous agent on both sides.
- Require at least two work units that can begin without consuming one another's generated output.
- Make dependencies explicit in a DAG; a long serial chain is prompt chaining, even if every node is called an agent.
- Assign each mutable resource one writer or an explicit concurrency protocol. Natural-language consensus is not a lock.
- Let workers return bounded typed evidence; the aggregator owns deduplication, disagreement, missing branches, and final acceptance.
Estimate both elapsed time and total work
Tcritical ≈ max(Tbranch₁ … Tbranchₙ) + Tmerge; Wtotal = Σ(tokensᵢ + toolᵢ) + coordination
Ideal fork/join latency follows the slowest branch plus aggregation, but total work sums all branches. Queueing, shared rate limits, retries, and stragglers make the latency expression optimistic.
Context pressure is a capacity argument only when the task can be partitioned. Three workers each receiving the same giant dossier triple input work without expanding useful context. A better design gives each worker a bounded evidence shard and requires citations or content identities in its output so the merge can recover provenance rather than trusting a lossy narrative.
First-party engineering reports can motivate a hypothesis, not set your release bar. Anthropic describes a research product where parallel subagents explore independent directions and also reports materially higher token use; that product-specific observation does not establish that a multi-agent topology improves coding, support, or your task distribution. Reproduce the comparison locally with the same models, tools, data, timeouts, and stopping rules.
Turn the topology argument into a reviewable record
1def audit_decomposition(2 contract: DecisionContract, record: DecompositionRecord3) -> DecisionReport:4 """Decide whether a bounded fork/join plan justifies separate agent contexts."""5 contract = validate_record(contract, DecisionContract)6 record = validate_record(record, DecompositionRecord)7 if record.scope != contract.scope or record.contract_content_id != contract.content_id:8 raise ValueError("decomposition belongs to another decision contract")Expected output
example=illustrative_only
decision=MULTI_AGENT_JUSTIFIED
branches=3;critical_path_units=2
signals=parallel-frontier,context-pressure,credential-isolation,bounded-coordination
estimated_tokens=27000;coordination_ratio=0.111
claim=LOCAL_DECISION_RECORD_NOT_MULTI_AGENT_BENCHMARKVerify: python3 -m unittest discover courses/ai-engineering/reference-impl/multi_agent_justification
The invented supplier-risk fixture has three independent read-only branches for policy, finance, and security evidence. Each has a distinct owner, tool domain, credential isolation key, context estimate, and typed finding schema. A fourth read-only unit depends on all three and emits the decision record. Their declared branch context exceeds one local context budget while each branch fits; the merge is below the contract's coordination-ratio threshold.
The audit accepts only this bounded fork/join shape. It reconstructs frozen records, checks exact contract scope, unique contiguous positions, backward dependencies, shared mutable isolation, a complete read-only merge, worker and context limits, finite exact-float ratios, exact booleans and strings, constructor bypass, and collection copying. Branches may share an isolation key only when every branch sharing it is read-only; mutable collisions require an explicitly serialized design outside this flat fork. `MULTI_AGENT_JUSTIFIED` is the result of an invented heuristic over estimates. The artifact runs no agent, measures no latency or quality, prices no tokens, and makes no benchmark claim.
Make handoffs smaller than the work they coordinate
- 01Contract the branchDeclare objective, admissible evidence, context cap, tools, credentials, side effects, deadline, output schema, and completion test.
- 02Bind the taskGive each worker a task and evidence digest so stale or cross-run output cannot enter the merge.
- 03Collect partial outcomesDistinguish success, abstention, timeout, policy denial, and malformed output; do not let silence masquerade as no finding.
- 04Resolve conflicts deterministicallyDefine evidence priority, duplicate identity, disagreement escalation, quorum if justified, and the sole owner of final acceptance.
- 05Stop the graphCap worker count, depth, retries, tokens, tool calls, and elapsed time; cancellation must reach children and their external effects.
Retries need one owner. If both orchestrator and worker retry the same branch, multiplicative attempts can exceed budgets and repeat effects. A child that can write needs its own idempotency and recovery contract; a parent retry does not erase an ambiguous child outcome. Persist lineage from final claims to branch evidence and tool receipts so a bad synthesis can be separated from a bad source.
Release the topology only if it beats a simpler baseline
| Measure | Compare | Slice |
|---|---|---|
| task success and verification | one call, fixed workflow, one agent, multi-agent | dependency depth and evidence breadth |
| critical-path latency | end-to-end plus branch and merge spans | normal, straggler, timeout, degraded mode |
| total cost | input/output tokens, tool calls, compute | success, retry, abandoned work |
| coordination defects | missing, duplicate, stale, conflicting handoffs | branch and aggregator version |
| security surface | credentials, writable resources, cross-tenant denials | tool domain and worker identity |
Use paired tasks and identical acceptance tests. Hold the model family, tool versions, corpus snapshot, budgets, and timeout policy constant enough to attribute the effect of topology. Report confidence intervals or repeated-run distributions when sampling variance matters. A win on one curated task is not a universal multi-agent advantage, and an internal vendor evaluation is not a transferable baseline.
Operate at three altitudes
Production lens
- — Trace branch lineage, queue time, model/tool versions, retries, tokens, typed outcome, cancellation, and merge decision under one task identity.
- — Alert on orphan workers, fan-out growth, repeated or stale handoffs, conflict rate, missing branches, cross-scope tool denials, and merge bottlenecks.
- — Evaluate degraded modes explicitly: fail closed, return partial evidence with disclosure, or escalate; never silently synthesize a complete answer from missing required branches.
Staff lens
- — Treat agent topology as an owned, versioned resource and security design with capacity limits, incident boundaries, and an evidence-backed simplification path.
- — Require a local baseline and costed decision record before creating a platform whose primary output is more agents, queues, and coordination protocols.
Interview defense
A team proposes five specialist agents to improve a research workflow. How would you decide whether the design is justified?
I would first benchmark a fixed workflow and one bounded agent. Then I would draw the dependency DAG and identify genuinely independent work, evidence that exceeds one usable context, different credential or sandbox boundaries, and stable ownership seams. For each branch I would specify inputs, tools, budgets, output schema, termination, and failure owner, with one deterministic aggregator. I would estimate slowest-branch critical path and total tokens/tool work, including duplicated context and retries. On paired held-out tasks I would compare quality, verified completion, latency distribution, cost, handoff defects, and security scope. I would keep multiple agents only for measured local value, not because the prompts have different personas.
Expect the interviewer to press on
- — When is parallel model calling a workflow rather than a multi-agent system?
- — How can credential isolation be real rather than prompt-level advice?
- — What happens when one required branch times out after others have completed?
Misconceptions to remove
“Different personas provide independent agents.”
Independence comes from task, state, context, tools, credentials, ownership, and failure boundaries. Labels alone enforce none of them.
“Parallel agents always reduce cost because they finish sooner.”
Parallelism can reduce elapsed time while increasing total token, tool, and coordination work; shared bottlenecks can erase even the latency gain.
“A majority vote verifies a result.”
Votes can share the same model, context, and systematic error. Verification needs task-grounded evidence, calibrated independence assumptions, and an explicit decision rule.
Check your model
1. Why is a serial chain of three model calls not evidence for three agents?
Its topology is a known dependency chain that a fixed workflow can own; separate autonomy adds no parallel front or isolation by itself.
2. What two cost views should a fork/join estimate include?
Critical-path elapsed work—roughly the slowest branch plus merge—and total work across every branch, tool, retry, and coordination step.
3. When does context partitioning fail as a justification?
When every worker needs the same full context or the typed summaries discard cross-branch evidence required for correct aggregation.
Prove the mechanism
Create a decision record for an incident investigation with logs, deployment history, and customer reports. Compare a fixed parallel workflow, one agent, and isolated agents; define branch evidence schemas, a missing-branch policy, and the exact measurement that would collapse the topology.
Add a production constraint
Add a dependent second wave to the decision artifact without calling it a flat fork/join. Compute critical-path units over the DAG, constrain fan-out and credentials per wave, detect cycles, and preserve deterministic aggregation and rollback to a fixed workflow.
Artifact: Multi-agent decomposition decision record
courses/ai-engineering/reference-impl/multi_agent_justification/multi_agent_decision.py
Download reference implementationPrimary references and next links
References
- 1. Building Effective AI Agents
Anthropic. First-party engineering guidance distinguishing workflows and agents and describing parallel and orchestrator-worker patterns.
- 2. How we built our multi-agent research system
Anthropic. A product-specific first-party report on parallel context, token cost, orchestration, and unsuitable tightly coupled tasks; its internal results are not generalized here.
- 3. Why Do Multi-Agent LLM Systems Fail?
Cemri et al.. Primary empirical study organizing observed multi-agent failures around system design, inter-agent alignment, and task verification or termination.
Continue through the graph
- Durable Agent Workflows →
Give parent and child work explicit retry, cancellation, reconciliation, and terminal-state ownership.
- Agent Red-Team: Untrusted Instructions Meet Real Tools →
Test peer output, shared credentials, delegation, and aggregation as additional attack surfaces.
Glossary: multi-agent system · fork/join · dependency DAG · critical path · coordination overhead · context partitioning · credential isolation · typed handoff