A Tool-Using Agent Is a Bounded State Machine
The model may propose the next action. The runtime—not the model—owns authority, limits, validation, side effects, and the definition of done.
- Authorship
- InterviewsVector
- Published / updated
- 2026-08-11 / 2026-08-11
- Review status
- Artifact tests passing · primary sources recorded
Original InterviewsVector material. Code examples are covered by repository tests and primary references are recorded. No named human reviewer is claimed.
The decision in one pass
Implement an agent as a deterministic runtime around a probabilistic planner. The runtime holds typed state, validates every proposed action, enforces step, time, token, and cost budgets, checks authorization and approval, executes idempotent tools, records observations, and ends in explicit succeeded, failed, cancelled, or budget-exhausted states.
Why this matters
A while-loop that asks a model what to do next is a demo. In production, repeated retries, ambiguous tool results, partial side effects, prompt injection, cancellation, and human approval turn the loop into a workflow engine with security consequences.
You will be able to
- Separate planner, policy, executor, and state persistence.
- Model agent execution with explicit transitions and terminal states.
- Enforce budgets and sensitive-action approval outside the model.
- Design idempotent tool calls and recover from partial success.
- Map MCP tool schemas into—but not beyond—the runtime authority boundary.
Prerequisite contract
- — State machines and retry semantics
- — Structured outputs or tool calling
- Agent loop and tool boundary →
Your Vector Loop for this lab
- 01
Model
Treat model output as an untrusted transition proposal.
- 02
Derive
Define state, invariants, guards, effects, and terminal outcomes.
- 03
Build
Implement a bounded TypeScript runtime with fake tools.
- 04
Stress
Inject loops, invalid actions, timeouts, partial success, and cancellation.
- 05
Operate
Add authorization, approvals, durable logs, idempotency, and SLOs.
- 06
Defend
Justify every piece of autonomy with a bounded business capability.
The planner proposes; the runtime disposes
| Component | Owns | Must not own |
|---|---|---|
| Planner model | proposed action and arguments | permissions or budget truth |
| Policy | allowed tools, tenants, data, approvals | natural-language interpretation alone |
| Executor | validated side effect and timeout | open-ended planning |
| State store | history, status, idempotency, checkpoints | secret material in model context |
| Human gate | high-impact authorization | routine low-risk retries |
Give every run a finite set of states
Bounded agent simulator
Step through valid, repeated, denied, and budget-exhausted proposals. Observe which layer rejects the transition and what remains recoverable.
01
Observe
Read the user goal and current state.
02
Propose
Choose one bounded next action.
03
Authorize
Apply tool, tenant, budget, and human-approval policy.
04
Act
Execute one idempotent tool call.
05
Verify
Inspect the result and decide whether the goal is satisfied.
Next boundary: Observe.
- 01PlanningThe planner receives bounded context and proposes a typed action or completion.
- 02ValidatingSchema, policy, budget, preconditions, and duplication are checked.
- 03Awaiting approvalA sensitive action pauses with exact proposed inputs visible.
- 04ExecutingThe tool runs with timeout, scoped credentials, and an idempotency key.
- 05ObservingA typed result or actionable error is appended to state.
- 06TerminalSucceeded, failed, cancelled, or exhausted is persisted and cannot silently resume.
Build a loop with guards outside the model
1type Status = "planning" | "executing" | "succeeded" | "failed" | "exhausted"2 3type Proposal =4 | { kind: "tool"; name: string; args: Record<string, unknown> }5 | { kind: "finish"; answer: string }6 7type State = { status: Status; steps: number; maxSteps: number; seen: Set<string> }8 9export function validate(state: State, proposal: Proposal, allowed: Set<string>) {10 if (state.steps >= state.maxSteps) return { ok: false, reason: "step_budget" } as const11 if (proposal.kind === "finish") return { ok: true } as const12 if (!allowed.has(proposal.name)) return { ok: false, reason: "tool_denied" } as const13 const fingerprint = proposal.name + ":" + JSON.stringify(proposal.args)14 if (state.seen.has(fingerprint)) return { ok: false, reason: "repeated_action" } as const15 return { ok: true, fingerprint } as const16}17 18const state = { status: "planning", steps: 0, maxSteps: 2, seen: new Set<string>() } satisfies State19const denied = validate(state, { kind: "tool", name: "delete", args: {} }, new Set(["search"]))20if (!denied.ok) console.log(denied.reason)21const exhausted = validate({ ...state, steps: 2 }, { kind: "tool", name: "search", args: {} }, new Set(["search"]))22if (!exhausted.ok) console.log(exhausted.reason)Expected output
tool_denied
step_budgetVerify: Run node --experimental-strip-types --test courses/ai-engineering/reference-impl/agent/bounded-agent.test.ts.
A real fingerprint must canonicalize arguments and include tenant, principal, tool version, and resource scope. The example keeps the mechanism visible; the production contract must prevent two users from sharing an idempotency namespace.
Budgets are multidimensional
| Budget | Protects | Terminal behavior |
|---|---|---|
| steps | loops and compounding error | exhausted with trace |
| wall time | stale work and resource capture | cancel in-flight tools |
| tokens | context growth and model cost | summarize or stop by policy |
| money | provider and tool spend | deny before side effect |
| side effects | blast radius | require approval or stop |
| retries | duplicate actions and downstream pressure | fail with actionable error |
A step limit alone is insufficient: one tool call can run for an hour, spend money, mutate hundreds of records, or return enough data to overflow context. Each budget needs a counter, a guard before commitment, and a terminal reason visible to users and operators.
Test the trajectory, not only the final answer
- 01Repeated actionThe planner proposes the same mutation after an ambiguous timeout.
- 02Partial successThe external API commits but the response is lost.
- 03Indirect prompt injectionA retrieved document asks the agent to export secrets.
- 04Tool schema driftA once-optional argument becomes required.
- 05Human pauseApproval arrives after relevant external state has changed.
- 06CancellationThe user stops the run while a tool is in flight.
Protocols standardize transport, not trust
MCP standardizes how hosts, clients, and servers discover resources, prompts, and tools using JSON-RPC and negotiated capabilities. Tool definitions include schemas and metadata, but the current specification explicitly treats tool annotations as untrusted unless they come from a trusted server. The host still owns consent, policy, access control, timeouts, result validation, and audit.
Operate at three altitudes
Production lens
- — Persist transitions before and after side effects so recovery can distinguish not-started, in-flight, and committed work.
- — Use scoped short-lived credentials per tool and tenant.
- — Record model, prompt, tool schema, arguments, result status, approvals, budgets, and trace IDs with redaction.
- — Define cancellation and compensation semantics for every mutating tool.
Staff lens
- — Reject agent architecture when a deterministic workflow meets the requirement with less uncertainty.
- — Bound autonomy by capability and blast radius, not by a marketing label such as assistant or copilot.
- — Make the tool platform enforce policy uniformly across models and agent frameworks.
Interview defense
How would you make an LLM agent safe enough to call production tools?
I would put the model behind a deterministic state-machine runtime. Model output is an untrusted proposal validated against schemas, authenticated policy, budgets, preconditions, duplication, and approval requirements. Tools use scoped credentials, timeouts, idempotency keys, typed results, and durable transition logs. Runs end explicitly and can be cancelled or recovered without replaying side effects.
Expect the interviewer to press on
- — How do you recover after a tool times out after committing?
- — What belongs in durable state versus model context?
- — What security does MCP provide and what remains the host's responsibility?
Misconceptions to remove
“A stronger model makes the agent safe.”
Model quality may reduce proposal errors; it does not replace authorization, validation, idempotency, or blast-radius controls.
“Human in the loop solves tool safety.”
Approvals help for selected high-impact actions but can become rubber stamps and do not replace deterministic policy.
“MCP secures tool execution.”
MCP standardizes protocol interactions; implementations still enforce trust, consent, access control, and validation.
Check your model
1. Why is a tool timeout not proof that no side effect happened?
The external system may have committed before the response was lost. Recovery needs idempotency or a status lookup, not blind retry.
2. What makes a terminal state useful?
It makes completion reason explicit, prevents accidental resume, supports user communication, and enables reliable metrics and recovery.
Prove the mechanism
Add cost, time, and side-effect budgets plus a human approval state to the runtime. Test every transition and forbidden transition.
Add a production constraint
Persist state to an append-only event log and recover safely from a crash at every line around a mutating tool call.
Artifact: Tested bounded agent runtime
courses/ai-engineering/reference-impl/agent/bounded-agent.ts
Download reference implementationPrimary references and next links
References
- 1. ReAct: Synergizing Reasoning and Acting in Language Models
Yao et al.. Primary paper on interleaved reasoning and actions.
- 2. Model Context Protocol specification
Model Context Protocol. Current authoritative protocol specification at publication.
- 3. LLM06:2025 Excessive Agency
OWASP GenAI Security Project. Current risk description and mitigation guidance.
Continue through the graph
- Secure agent tool boundaries →
Rehearse the security design.
- Bounded autonomy →
Continue into staff-level architecture judgment.
- MCP protocol comparison →
Place MCP alongside adjacent protocols.
Glossary: agent · tool · state machine · idempotency · approval gate · MCP · excessive agency