Production AI SystemsThe Staff Engineer's Playbook

The official rubric is generic. The unspoken rubric is what gets you hired. This lesson names the 7 specific signals interviewers grade in any AI design round, and the moves that signal each one. Every later lesson in this course is one or two of these seven signals expanded into a technique.

Most candidates open AI design questions by drawing boxes. Staff candidates spend the first 5–7 minutes defining the problem so precisely that the architecture becomes mechanical. CLARO is the 5-step opening you run before any line is drawn.

Staff candidates name the dimensions of their trade-offs out loud. Senior candidates describe options. This lesson builds the precise vocabulary so you stop saying 'it depends' and start saying 'it depends on whether [X], and here is the rule.' TRACK is the five dimensions every AI system design trade-off lives on.

Every LLM latency conversation gets stuck on quantization and batching because candidates don't have a structural way to decompose where time is actually spent. The 5-Phase Latency Anatomy gives you a vocabulary for diagnosing any LLM serving system in under 60 seconds.

Every production RAG system fails for one of four reasons, and most candidates conflate them. The Retrieval Quality Loop is the four-stage diagnostic that decomposes 'RAG accuracy is bad' into a fixable problem instead of a category.

Agentic systems fail in the boundary, not in the model. The Bounded Autonomy Pattern names the five dimensions where autonomy lives and lets you design the boundary deliberately. Without the pattern, every agent design becomes a debate about ReAct vs Reflexion; with it, the architecture follows from the bounds.

A production prompt is not a string. It is a versioned, evaluated, cached, observably-degraded distributed system in disguise. The Prompt Lifecycle Stack names the five layers; once you see prompts that way, the right interview answers become obvious — and so do the failure modes most production teams hit.

Every feature-store debate is the same triangle: freshness, consistency, cost. Pick two. This lesson builds the vocabulary to name which two you picked, why, and what corner case the interviewer is trying to push you into.

Reproducibility isn't a checkbox — it's a system property that requires versioning data, code, environment, randomness, and intent. This lesson covers the 5-Axis Model and how to design for it without paying 10× in operational overhead.

Blue/green is a deploy pattern for code, where the failure mode is crashes. Models need a different one because their failure mode is silent quality drift. This lesson covers the shadow → canary → interleaved/A/B → ramped pattern and when each step is the right call.

ML systems fail silently. The interview question is: how would you know? This lesson builds the Silent Failure Detection Stack — the four observability layers a Staff candidate names and the three layers a Senior candidate forgets exist.

A 45-minute simulated Staff interview on a 200M-user video platform serving personalized recommendations at p99 100ms. Walked phase by phase: the CLARO opening, the five clarifying questions with dependency mapping, the architecture, five deep-dive calibration ladders, the latency Gantt, the scale math, the interviewer's follow-up probes, and the cross-company scoring lens. Two anonymized post-mortems and two downloadable artifacts. The lesson assumes you know what a two-tower model is — and tells you what most candidates miss when designing one under SLA.

LLM-powered search systems all converge to the same four-stage architecture: Retrieve, Ground, Stream, Audit. This walkthrough applies RGSA to a Perplexity-class system at 10M monthly users with the 1.5-second TTFT bar, and shows where the structural decisions live.

Inline risk scoring at the payment-auth latency budget. Feature freshness, model size constraints, and the rules-vs-ML composition pattern that always comes up. The Inline Risk Budget is the framework that converts 'design fraud detection' from a tour of ML techniques into a budget-driven architectural decomposition.

Meta-scale content moderation is two systems with two SLAs and a human-review feedback loop, not one pipeline that does everything. The Cascaded Sync + Streaming Async framework names the decomposition and forces the candidate to address the structural fact that policy categories have different latency budgets.

Copilot-class coding assistants serve completions inline in IDE keystrokes with sub-100 ms perceived latency. The Ghost-Text Completion Stack names the five layers — anticipatory fetch, FIM prompting, speculative cancellation, privacy-preserving tenancy, acceptance feedback — and shows where the architectural decisions live.

The first 5 minutes set the floor for your final score. The Opening Triplet is the three-move choreography Staff candidates execute by reflex: name a framework, surface the highest-leverage constraint, commit under uncertainty. Done in 5 minutes, those three moves score three of the seven signals from Lesson 1.1 before the design even begins.

Interviewers push back for four distinct reasons: stress-testing, ceiling-finding, debugging the candidate, or offering a hint. Each requires a different response. The 4-Probe Decoder names the four types and the first-sentence cues that distinguish them — so you respond appropriately instead of defending against a stress test when you were actually being taught.

Everyone makes a visible mistake. The interview is decided by what you do in the next 90 seconds. The 3-Move Recovery — acknowledge, walk back, commit — turns a stumble into a Staff signal. Done well, it is the highest-leverage moment in the entire interview because it demonstrates the metacognitive ability the rest of the loop only tests indirectly.