State, Memory, and Context Are Different Things
State decides what the workflow may do next, memory supplies revisable evidence, and context is the bounded view assembled for one model call.
- Authorship
- InterviewsVector
- Published / updated
- 2026-09-21 / 2026-09-21
- Review status
- Artifact tests passing · primary sources recorded
Original InterviewsVector teaching. Executable artifacts are deterministic illustrative boundary checks covered by focused tests; they do not claim production authorization, protocol conformance, durability, memory quality, or safety certification.
The decision in one pass
Keep three contracts separate. Durable workflow state is the authoritative, versioned record used to resume and guard transitions. Memory is retrieved, revisable evidence with provenance, tenant, retention, and confidence; it must not silently become permission or completion truth. Context is an ephemeral, token-bounded projection assembled for one model call from selected state, instructions, tool results, and memory. Bind the state projection to an immutable state revision, label memory as evidence, enforce tenant and access filters before retrieval, and persist important outcomes through explicit state transitions rather than assuming conversation history is durable.
Why this matters
When one transcript serves as database, memory index, policy, and prompt, restarts lose progress, stale recollections override current facts, private records cross tenants, and context truncation changes control flow. Separating ownership lets each layer have the right consistency, retention, access, observability, and failure semantics.
You will be able to
- Distinguish authoritative workflow state, retrievable memory, and ephemeral model context by ownership and lifecycle.
- Bind model-visible state to an immutable durable revision without granting memory control authority.
- Design memory provenance, tenant, confidence, retention, correction, and deletion boundaries.
- Assemble a bounded context view and treat declared token counts as tokenizer-specific evidence.
- Recover workflow progress from durable events rather than replaying a conversation as truth.
Your Vector Loop for this lab
- 01
Model
Name every datum, its authority, owner, source, tenant, lifetime, and consumer.
- 02
Derive
Derive state-transition, memory-provenance, retrieval, and context-budget invariants separately.
- 03
Build
Build immutable state records, evidence-labelled memories, and a one-call context projection bound to state identity.
- 04
Stress
Inject stale revisions, cross-tenant memory, authority relabelling, token overflow, deletion, restart, and constructor bypass.
- 05
Operate
Monitor state lag, replay failures, memory freshness, retrieval access denials, context composition, and truncation by cohort.
- 06
Defend
Say which records are authoritative, which are evidence, how corrections propagate, and what durability or quality remains unproved.
Give each layer one job
| Layer | Primary job | Typical lifetime | Must not imply |
|---|---|---|---|
| workflow state | authoritative progress, guards, budgets, terminal status | run or business retention window | that a model explanation is true |
| memory | retrieve potentially useful prior evidence | policy-defined and correctable | permission, approval, or completed effect |
| context | present a bounded view for one inference | one call or short turn | durable persistence or complete history |
| trace | explain proposals, decisions, attempts, and outcomes | observability retention window | source-of-truth business state |
The same fact can appear in more than one layer, but it needs lineage. A durable approval record may be rendered into context; the rendering does not become a second authority. A retrieved customer preference may influence wording; it does not authorize contact. A tool result may propose a state transition; the transition commits only after the workflow store accepts it under the expected revision.
State is versioned control truth
A state record should contain only what the deterministic runtime needs to resume: run identity, revision, status, budgets, pending operation, approval reference, last applied event, and immutable external operation identities. Update it with compare-and-set or append-only event rules so two workers cannot both consume the same transition. Terminal states are explicit; silence or an absent model response is not success.
apply(event, stateᵣ) → stateᵣ₊₁ only if expected_revision = r
The exact persistence mechanism can vary, but the transition must reject a stale writer. An LLM-generated statement about the revision is not the revision check.
- 01LoadRead the authoritative revision and current policy-relevant fields.
- 02ProposeLet deterministic code or a planner propose one typed transition.
- 03ValidateCheck state, budget, approval, idempotency, and external preconditions.
- 04CommitPersist the event and new revision atomically or fail on a stale expectation.
- 05RenderCreate a bounded context projection from the committed revision.
Memory is evidence with provenance and deletion semantics
Agent memory may hold episodes, extracted facts, preferences, summaries, or reflections. Each record needs source identity, observation time, tenant and subject, access policy, extraction version, confidence or verification state, retention class, and a correction path. Retrieval score is not truth probability, and a generated summary can introduce claims absent from its source.
| Memory risk | Required control | Operational evidence |
|---|---|---|
| stale fact | validity interval and source freshness | age and supersession rate by memory kind |
| cross-tenant recall | filter before candidate scoring | denied and impossible cross-tenant fixtures |
| incorrect extraction | source link, confidence, correction workflow | sampled claim-to-source review |
| deletion drift | lineage fan-out and tombstone propagation | source-to-derived deletion completion |
| authority escalation | fixed evidence label at context assembly | no memory item rendered as control state |
Context is a bounded projection, not storage
Build context from typed sources under a declared order and budget: system policy, current state projection, current user input, selected memory, and tool evidence. Label origin and authority so untrusted content cannot masquerade as instruction. Token counts depend on the exact tokenizer and framing; a caller-declared count is useful for a fixture but production assembly must measure the real serialized input.
context = project(policy, state_revision, input, selected_evidence, budget)
Projection is deterministic for declared inputs and policy, while model output remains probabilistic. Changing the state revision or selection policy creates a different context identity.
Audit a frozen state, memory, and context snapshot
The artifact content-addresses a frozen workflow state, copies caller-owned collections, reconstructs nested records at its public boundary, enforces exact scalar types and finite confidence, rejects cross-tenant memory, requires one current state projection with control authority, keeps memory as evidence, and checks a declared context budget. It does not run storage, retrieval, a tokenizer, or a model, so its pass is neither durability nor memory-quality evidence.
1def main() -> None:2 print(format_example())Expected output
example=illustrative_only
state_id=workflow-state@sha256:c48042d6f171c4860d95ef2eb3795fff63f4dacbec8dbf93e6c997cc47207d36
state=AUTHORITATIVE_DURABLE_RECORD
memory=RETRIEVABLE_NON_AUTHORITY
context=BOUNDED_MODEL_VIEW
declared_context_tokens=29/64
selected_memories=1
decision=PASS_SEPARATION_INVARIANTS
claim=LOCAL_FIXTURE_ONLY
certification=NOT_A_DURABILITY_OR_MEMORY_QUALITY_CERTIFICATIONVerify: python3 -m unittest discover courses/ai-engineering/reference-impl/agent_state_memory
Recover from events; evaluate memory separately
A restart should load or replay authoritative state, reconcile any in-flight external operation, and then assemble fresh context. It should not regenerate state by asking the model what probably happened. Observe workflow recovery latency, stale-write rejection, stuck states, approval age, and unknown external outcomes separately from memory precision, freshness, correction time, deletion lag, retrieval latency, and context truncation.
Operate at three altitudes
Production lens
- — Use optimistic concurrency or an event log for authoritative state and bind external operation identities before dispatch.
- — Enforce tenant, subject, purpose, retention, and deletion policy before memory candidates reach semantic ranking.
- — Measure the real serialized context with the deployed tokenizer and trace source, authority, revision, and truncation decisions.
Staff lens
- — Assign distinct owners and SLOs to workflow persistence, memory governance, retrieval, and context assembly even when one platform implements them.
- — Define which facts are authoritative by domain; do not let a convenient vector store become an accidental system of record.
- — Fund correction and deletion fan-out before launching durable personalization or cross-session memory.
Interview defense
An agent must resume a week-long support workflow and remember customer preferences. How do you separate state, memory, and context?
I persist workflow status, revision, budgets, approvals, pending operation IDs, and terminal outcome in an authoritative store with stale-write protection. Preferences live as tenant-scoped memory evidence with source, observation time, confidence, retention, correction, and deletion lineage; retrieval never grants permission. For each model call I assemble a bounded context from policy, a projection bound to the current state revision, current input, and selected evidence labelled by authority. After restart I replay or load state and reconcile in-flight effects, then build fresh context rather than trusting the transcript. I evaluate durability and memory quality separately and do not claim that local structural checks prove either.
Expect the interviewer to press on
- — Can a retrieved memory change workflow status directly?
- — What must happen when a source record is deleted?
- — Why is a transcript insufficient for crash recovery?
Misconceptions to remove
“The conversation transcript is the agent's durable state.”
It is model-visible context that may be truncated or summarized. Durable state needs explicit revisions, transitions, terminal outcomes, and recovery semantics.
“A high retrieval score makes a memory authoritative.”
The score ranks candidates under one retriever. Authority comes from the source-of-truth domain and policy, not semantic similarity.
“Longer context eliminates the need for memory design.”
Context is still bounded, access-controlled, costly, and ephemeral; stale evidence and deletion obligations remain even when the window grows.
Check your model
1. Which layer determines whether a workflow may execute the next tool?
Authoritative workflow state plus current deterministic policy; retrieved memory and model context may supply evidence but cannot grant the transition.
2. Why bind a state rendering to a content identity or revision?
So the runtime can detect a context assembled from stale control state rather than silently acting on it.
3. What does a memory confidence value not prove?
It does not prove truth, freshness, authorization, retrieval relevance, or calibrated probability unless those properties have separate evidence.
Prove the mechanism
Design records for a support agent that can pause for approval, resume after restart, and remember a communication preference. Mark authority, tenant, provenance, retention, revision, correction, deletion, and context-budget fields, then specify five fail-closed fixtures.
Add a production constraint
Plan a memory-policy migration that changes extraction and retention while workflows remain active. Define dual reads, state and memory identities, backfill, correction precedence, deletion fan-out, context comparability, rollback, and contraction.
Artifact: Agent state and memory separation audit
courses/ai-engineering/reference-impl/agent_state_memory/state_memory_audit.py
Download reference implementationPrimary references and next links
References
- 1. ReAct: Synergizing Reasoning and Acting in Language Models
Yao et al.. Primary research on interleaving reasoning traces and task-specific actions; a trace is not thereby durable state.
- 2. Generative Agents: Interactive Simulacra of Human Behavior
Park et al.. Primary research architecture using stored experiences, reflection, retrieval, and planning.
- 3. MemGPT: Towards LLMs as Operating Systems
Packer et al.. Primary work on tiered memory and virtual context management beyond a bounded model window.
- 4. Temporal Workflow Execution overview
Temporal. Official documentation on durable workflow state, replay, event history, memo, and worker-local cache distinctions.
Continue through the graph
- Durable Agent Workflows →
Extend authoritative state into retry, restart, pause, and compensation behavior.
- The Embedding Contract →
Version the representation used to retrieve memory evidence.
Glossary: workflow state · event history · optimistic concurrency · episodic memory · semantic memory · context projection · provenance · retention