Context Engineering Interview Guide: Prompts, Memory, Retrieval, Compression, and Context Windows

Most teams reach for a bigger model or a longer context window when an LLM feature starts returning vague, stale, or contradictory answers. That rarely fixes it. The usual cause is that the wrong tokens are reaching the model: too much irrelevant history, retrieved passages buried in the middle, no durable memory, and a prompt that competes with untrusted text for the model's attention. Context engineering is the discipline that fixes this by deciding, on every call, exactly what goes into the window and what gets left out.

This guide is written for the interview and for production. It defines context engineering precisely, breaks it into its five levers, gives runnable TypeScript for the part interviewers actually probe, the assembly budget, and closes with trade-offs, common mistakes, and how to explain it out loud.

Quick Answer

Context engineering is deciding what information enters a model's limited context window on each request, and how it is formatted, ordered, and trimmed. You treat the window as a token budget and fill it from four sources: instructions, memory, retrieved evidence, and tools. Everything that does not fit is compressed or evicted, so the model sees the most relevant tokens without overflowing or losing focus.

TL;DR

• Context engineering ≠ prompt engineering. Prompt engineering tunes wording; context engineering tunes the whole payload and what gets dropped.
• The window is a budget. Allocate tokens deliberately; do not concatenate until it overflows.
• Selection beats capacity. Models attend unevenly across long inputs, so relevance and position matter more than raw window size.
• Memory is tiered: short-term turns, a working scratchpad, and a durable long-term store you retrieve from.
• Compression is lossy. Summarize with structure, version it, and keep raw sources retrievable.
• Isolate trust. Everything in context is treated as input the model may follow, so keep instructions separate from untrusted retrieved or user content.

What Is Context Engineering?

A language model is a stateless function. It has no memory of your last request and no access to your database. Each call, it sees exactly one thing, the context, a sequence of tokens you assemble, and it produces a continuation. Everything the model "knows" for that call either came from its training weights or from the context you built.

Context engineering is the practice of building that context well. Formally, it is the set of decisions about:

• What information to include (which documents, which past turns, which facts).
• How much of each to include, given a fixed token budget.
• In what order and format to place it, because position and structure change how the model uses it.
• What to drop when everything relevant does not fit.

Prompt engineering is a subset of this: it optimizes the wording of the instruction. Context engineering optimizes the information environment the instruction lives in. For a deeper interview-focused treatment of how instructions, user input, and retrieved evidence should be layered, see how system, user, and context should be separated.

The Five Levers of Context Engineering

Every context-engineering decision falls under one of five levers. Interviewers often name one ("how would you handle memory?") but are really testing whether you see how all five share a single budget.

The rest of this guide takes each lever in turn, then shows the assembly step that combines them.

Lever 1: Prompts and instruction layering

The instruction layer is the smallest part of the budget and the most load-bearing. Three practices matter for interviews:

1. Layer by trust and stability. Put stable, trusted rules in the system message; put the volatile task in the user message; and clearly delimit any retrieved or user-supplied content as data, not instructions.
2. Make output contracts explicit. Ask for structured output (JSON, a schema) when a downstream system will parse it, and describe the schema in the prompt.
3. Keep it stable for caching. Providers cache repeated prefixes. A system prompt that changes every call defeats prompt caching and raises cost and latency.

// A message is more than a string: it carries a trust level that decides
// where it goes and whether the model should treat it as an instruction.
type Trust = "system" | "user" | "retrieved" | "tool"
 
interface ContextBlock {
  trust: Trust
  content: string
  /** Estimated token cost. Use a real tokenizer in production (see below). */
  tokens: number
  /** Higher = more important; used when the budget forces eviction. */
  priority: number
}
 
// Retrieved and tool content is wrapped so the model sees it as quoted DATA.
// This is a defense-in-depth boundary, not a guarantee, so still validate
// tool calls and outputs downstream.
function asUntrustedData(source: string, body: string): string {
  return [
    `<data source="${source}">`,
    body.replaceAll("</data>", "&lt;/data&gt;"), // prevent tag breakout
    `</data>`,
  ].join("\n")
}

Placing untrusted content in context is exactly where prompt injection lives: the model cannot inherently tell your instructions from an attacker's text pasted into a retrieved document. Wrapping and delimiting help, but they are not a full defense. For the threat model and mitigations, see the AI agent security interview guide.

Lever 2: Retrieval

Retrieval decides which external facts enter the window. The context-engineering point is not how to build a retriever, it is that retrieval output is untrusted, ranked, and budgeted: you fetch broadly, rerank to a small high-precision set, and include only what fits after reserving tokens for instructions and output.

Two rules that separate a senior answer from a junior one:

• Rank, then truncate by tokens, not by document count. "Top 5 chunks" is a bad budget because chunks vary in size. Fill a token allowance in rank order instead.
• Prefer relevance over recency-only or similarity-only. Combine lexical and semantic signals and rerank; raw vector similarity alone buries exact-match terms like error codes and identifiers.

Retrieval is a deep topic in its own right. This post treats it as one lever; for the full pipeline, hybrid search, reranking, evaluation, and cache safety, read Design a Production RAG System.

Lever 3: Memory

Memory is what lets a stateless model feel continuous. The mistake is treating memory as "append every turn to a list." That grows without bound and pushes relevant older facts out of the window. Senior systems tier memory:

• Short-term memory: the last N turns, kept verbatim. Cheap, high fidelity, small.
• Working memory: a compact, structured scratchpad for the current task, such as the plan, decisions so far, and open questions. Rewritten, not appended.
• Long-term memory: durable facts (user preferences, prior outcomes) persisted to a store and retrieved on demand, the same way you retrieve documents.

interface MemoryState {
  shortTerm: ContextBlock[] // recent turns, verbatim
  working: string // rewritten each step: plan + decisions + open items
  // long-term lives in a store; you retrieve relevant items, you do not inline all
}
 
// When short-term history exceeds its budget, summarize the OLDEST turns into a
// compact recap and drop the raw turns. Keep the most recent turns verbatim,
// because the current exchange usually matters most.
function evictShortTerm(
  turns: ContextBlock[],
  budgetTokens: number,
  summarize: (blocks: ContextBlock[]) => ContextBlock
): ContextBlock[] {
  let total = turns.reduce((sum, t) => sum + t.tokens, 0)
  if (total <= budgetTokens) return turns
 
  const kept: ContextBlock[] = []
  const toSummarize: ContextBlock[] = []
  // Walk newest → oldest, keeping verbatim turns until the budget is spent.
  for (const turn of [...turns].reverse()) {
    if (total <= budgetTokens) {
      toSummarize.unshift(turn)
    } else {
      kept.unshift(turn)
      total -= turn.tokens
    }
  }
  if (toSummarize.length === 0) return kept
  return [summarize(toSummarize), ...kept]
}

Deciding what to promote from working to long-term memory, and how an agent reads it back, is one of the harder parts of agent design. The interview-level breakdown is in how an agent should manage memory, state, and context, and the system-design view is in How to Design a Multi-Agent System.

Lever 4: Compression

Compression buys budget by reducing tokens: summarizing history, pruning low-value passages, deduplicating boilerplate, or extracting structure from raw dumps. It is always potentially lossy, so treat it like a cache with correctness risk:

• Summarize with structure, not prose. A fixed schema (decisions, facts, open_questions) is reproducible and easy to validate; a free-form paragraph drifts.
• Compress the old, keep the recent raw. Recency correlates with relevance in most conversations and agent loops.
• Never discard the source. Keep raw content retrievable so a later step can re-expand what a summary dropped.
• Version and measure. Record which summarizer produced a recap, and track answer quality with and without compression so you can catch silent regressions.

// A structured recap is deterministic to parse and cheap to validate,
// unlike a free-form summary. The token cost is bounded by schema, not input.
interface Recap {
  decisions: string[]
  facts: string[]
  openQuestions: string[]
  summarizerVersion: string // version so regressions are attributable
}
 
function recapToBlock(recap: Recap, estimate: (s: string) => number): ContextBlock {
  const content = [
    `Decisions: ${recap.decisions.join("; ")}`,
    `Facts: ${recap.facts.join("; ")}`,
    `Open questions: ${recap.openQuestions.join("; ")}`,
  ].join("\n")
  return { trust: "system", content, tokens: estimate(content), priority: 80 }
}

Lever 5: The context window

The window is the hard constraint every other lever spends against. Two facts drive the design:

1. Attention is not uniform. Models weight the beginning and end of a long input more than the middle, a pattern documented as the Lost in the Middle effect. Practically: put the most important evidence and the instruction near the edges, not buried in a wall of context.
2. Cost and latency scale with tokens. Longer inputs cost more and are slower, and the per-token attention work in the prefill stage grows with sequence length. Filling a huge window "just in case" is a direct cost and latency regression.

This is why a larger window is not a free upgrade. It raises the ceiling on how much you can include; it does not decide what you should. The interview framing of tokenization and window limits is covered in why tokenization and context-window limits matter in production.

On numbers and freshness: context-window sizes, per-token prices, and tokenizer behavior change with every model release and differ by provider. Treat any specific number you memorize as perishable, and read the current values from the provider's own documentation and token counter rather than a blog post (this one included).

Putting It Together: the Context Assembly Budget

This is the piece interviewers most want to see: given all five levers, how do you actually fill one window? The algorithm is a priority budget. Reserve output space, spend the rest in priority order, and compress or evict when you run out.

interface AssemblyConfig {
  windowTokens: number // total model context window
  reservedOutputTokens: number // never allocate this to input
  safetyMarginTokens: number // headroom for tokenizer drift
}
 
interface AssemblyResult {
  included: ContextBlock[]
  droppedBlocks: ContextBlock[]
  usedTokens: number
  inputBudget: number
}
 
/**
 * Assemble a context window from prioritized blocks.
 *
 * Blocks are included highest-priority first until the input budget is spent.
 * System instructions should carry the highest priority so they are never the
 * thing that gets dropped. Anything that does not fit is returned in
 * `droppedBlocks` so the caller can summarize or re-rank instead of silently
 * truncating mid-sentence.
 */
function assembleContext(
  blocks: ContextBlock[],
  config: AssemblyConfig
): AssemblyResult {
  const inputBudget =
    config.windowTokens -
    config.reservedOutputTokens -
    config.safetyMarginTokens
 
  if (inputBudget <= 0) {
    throw new Error(
      `No input budget: window ${config.windowTokens} is smaller than reserved output + margin`
    )
  }
 
  // Stable sort by priority (desc). Preserve input order within a priority tier
  // so retrieval rank and turn order are respected.
  const ordered = blocks
    .map((block, index) => ({ block, index }))
    .sort((a, b) => b.block.priority - a.block.priority || a.index - b.index)
    .map(({ block }) => block)
 
  const included: ContextBlock[] = []
  const droppedBlocks: ContextBlock[] = []
  let usedTokens = 0
 
  for (const block of ordered) {
    if (usedTokens + block.tokens <= inputBudget) {
      included.push(block)
      usedTokens += block.tokens
    } else {
      droppedBlocks.push(block) // caller decides: compress, re-rank, or drop
    }
  }
 
  return { included, droppedBlocks, usedTokens, inputBudget }
}

Running it with a small worked example:

// A rough char/4 heuristic ONLY for the example. In production use the model's
// real tokenizer (for example tiktoken or the provider's token-count endpoint),
// because token counts, not characters, are what the window measures.
const estimate = (s: string) => Math.ceil(s.length / 4)
const mk = (
  trust: Trust,
  content: string,
  priority: number
): ContextBlock => ({ trust, content, tokens: estimate(content), priority })
 
const blocks: ContextBlock[] = [
  mk("system", "You are a support assistant. Cite sources. Answer only from context.", 100),
  mk("user", "Why was my invoice #4471 charged twice?", 95),
  mk("retrieved", asUntrustedData("billing-kb", "Duplicate charges occur when a retry ..."), 70),
  mk("retrieved", asUntrustedData("billing-kb", "Refunds post within 5-10 business days ..."), 65),
  mk("retrieved", asUntrustedData("marketing", "Upgrade to Pro for priority support ..."), 20), // low value
]
 
const result = assembleContext(blocks, {
  windowTokens: 200, // deliberately tiny to force a drop
  reservedOutputTokens: 60,
  safetyMarginTokens: 10,
})
 
console.log(result.included.map((b) => b.trust)) // system, user, retrieved, retrieved
console.log(result.droppedBlocks.map((b) => b.trust)) // ["retrieved"]  ← the low-priority marketing block
console.log(`${result.usedTokens}/${result.inputBudget} input tokens used`)

Expected behavior: the system instruction and user question are kept because they carry the highest priority; the two relevant billing passages fit; the low-value marketing passage is dropped rather than allowed to push out something important. Because the low-priority block is returned, not silently cut, the caller can choose to summarize it or leave it out entirely, never truncate a block mid-sentence.

Which Lever Should You Reach For?

When an LLM feature misbehaves, the levers are not interchangeable. This decision tree maps the symptom to the lever that actually fixes it, which is the reasoning an interviewer wants to hear before you touch the window size.

Common Mistakes

• Concatenate-until-it-fits. Appending history and top-k chunks until the API errors, then blindly truncating from the top, which usually deletes the system prompt. Budget explicitly and drop by priority.
• Counting documents instead of tokens. "Top 5 results" ignores that chunks differ in size. Fill a token allowance.
• Burying the instruction. Putting the task in the middle of a huge context where the model attends to it least.
• Treating a bigger window as a strategy. More capacity without better selection just costs more and answers no better.
• Summarizing everything eagerly. Compressing recent, high-value turns to save tokens you did not need to save, and losing detail the next step required.
• Trusting retrieved text as instructions. Not delimiting untrusted content, which opens the door to prompt injection.
• Unversioned compression. Silent summarizer changes make quality regressions impossible to attribute.

Trade-offs a Senior Engineer Should Name

The senior move is to state the constraint that decides the trade-off, latency budget, cost ceiling, freshness requirement, or accuracy target, rather than declaring one side universally correct.

How to Explain This in an Interview

Keep it to a crisp mental model and then go one level deep on whichever lever they push on.

"I treat the context window as a fixed token budget. On each call I reserve tokens for the system prompt and the model's output, then spend the rest in priority order: instructions first, then the current user turn, then reranked retrieved evidence, then tiered memory. Anything that doesn't fit gets compressed, summarized older turns into a structured recap, or dropped from the low-priority end, and stays retrievable. I keep untrusted retrieved content clearly separated from instructions, and I measure relevance and answer quality per stage so I can tell which lever caused a regression."

Likely follow-up questions

• "The window doubled. What changes?" Fewer forced evictions and less aggressive compression, but the same selection pipeline; you do not suddenly stop retrieving, because cost, latency, and mid-context attention loss still apply.
• "Answers went stale after a doc update. Where do you look?" Retrieval freshness and any answer/summary cache keyed without a corpus version, not the prompt.
• "How do you stop the agent from following instructions inside a retrieved page?" Trust separation, delimiting untrusted data, and validating tool calls downstream.
• "History keeps growing. How do you bound it?" Tiered memory with summarize-and-evict of the oldest turns, keeping recent turns verbatim.
• "How do you know compression didn't hurt quality?" Versioned summarizers plus an eval set compared with and without compression.

Production Checklist

• The context window is filled from an explicit token budget, not concatenation.
• Output tokens and a safety margin are reserved before any input is added.
• Blocks carry a priority; eviction drops the lowest first and never truncates mid-block.
• Token counts use the model's real tokenizer, not a character heuristic.
• Retrieved and user content is delimited as untrusted data, separate from instructions.
• Memory is tiered (short-term, working, long-term) with summarize-and-evict.
• Compression is structured, versioned, and the raw source stays retrievable.
• The system prompt prefix is stable enough to cache where the provider supports prefix/prompt caching.
• Quality is measured per lever (retrieval relevance, post-compression answer quality) so regressions are attributable.
• Cache keys for answers include corpus and model versions so stale content is invalidated.

When This Model Does Not Apply

Context engineering is overhead you should skip when the task does not need it:

• Single-shot, self-contained prompts (classify this text, format this JSON) need no memory, retrieval, or budgeting, just a good prompt.
• Deterministic lookups are better served by SQL or an API than by putting data in context and hoping the model reads it correctly.
• Small, stable corpora that fit comfortably in the window may not need retrieval at all; include them directly and skip the retriever's operational cost.
• Fixed behavior/style is often better addressed by fine-tuning than by re-sending the same long instructions every call.

Reach for the full pipeline when context is large, changing, per-user, or multi-turn, which is exactly when the budget starts to bind.