Interview prompt
Problem context
Skills being evaluated
Use the sequence below to surface constraints, choose boundaries, test failure behavior, and defend trade-offs. Concrete numbers are interview assumptions, not claims about a real production system.
Clarify the decision
- Define the customer journey and valid success, then map serial, parallel, optional, and cached dependencies. Ask which teams control fallbacks and which failure modes are shared or correlated.
Establish scale assumptions
- Model observed conditional failure and latency contribution rather than multiplying nominal availability blindly. Correlated regional, identity, and database failures dominate independent-probability math.
Functional and non-functional requirements
- The journey owns a customer SLO; dependencies publish usable service-level objectives and degradation contracts. Budget policy should trigger joint action, not merely transfer blame.
High-level architecture
- Instrument journey traces and dependency outcomes, classify criticality, and create interface SLOs for latency, correctness, availability, and freshness. Optional calls receive strict deadlines and fallback expectations.
Data model and flow
- Carry journey and dependency classifications in telemetry, with outcome attribution that distinguishes caller cancellation, provider failure, and intentional shed. Aggregate at the customer journey first.
Consistency and transaction boundaries
- Correctness and freshness failures consume budget alongside HTTP errors when customers receive the wrong result. Fallback success counts only if it meets the declared degraded contract.
Failure modes and recovery
- Use burn-rate alerts and shared incident review for fast consumption. If one dependency exhausts budget, options include reliability work, caller isolation, reduced rollout, or renegotiated product objectives.
Security and privacy
- Security and compliance controls are mandatory constraints, not optional dependencies to bypass for availability. Their latency and availability still require investment and explicit design.
Observability and SLOs
- Provide journey burn, per-edge conditional contribution, correlated failure clusters, fallback-quality SLI, and change correlation. Avoid league tables that reward teams for narrow local metrics.
Capacity and cost
- Allocate reliability investment where marginal journey improvement is largest. Sometimes a caller-side cache or fallback is cheaper and more effective than demanding another nine from a dependency.
Alternatives and trade-offs
- Hard contractual dependency SLOs clarify expectations but can cause defensive behavior. Joint journey reviews keep incentives aligned while still assigning owners to measurable interface gaps.
Evolution and migration
- Start with one revenue journey, fix telemetry semantics, and run a quarterly budget review. Expand only after teams trust attribution and see that policy changes priorities rather than producing punishment.
What Staff and Principal candidates should emphasize
- Principal candidates reason about composition, correlation, and incentives. They make the customer journey the unit of truth while still producing actionable contracts at team boundaries.
Decision trade-offs
Budget allocation
Option A
Assign equal reliability targets to every dependency
Option B
Allocate by criticality and observed journey contribution
Recommendation:Allocate by criticality, fallback capability, and marginal customer impact; equal targets waste investment and hide dominant risks.
Governance
Option A
Provider team alone owns misses
Option B
Caller and provider jointly own the interface outcome
Recommendation:Use joint ownership because callers control deadlines, retries, fallbacks, and fan-out while providers control service behavior.
Follow-up interview questions
- 01How do twelve 99.9% services compose for one customer request?
- 02Should a successful stale fallback consume error budget?
- 03How do you account for shared regional or identity failures?
- 04What action follows a dependency budget exhaustion?
Common weak answers and mistakes
- 01Multiplying or adding advertised SLOs without observed topology and correlation.
- 02Measuring only HTTP availability and ignoring correctness or freshness.
- 03Using error budgets as a punitive ranking system between teams.
- 04Demanding every dependency reach the same number regardless of fallback value.
Interviewer evaluation rubric
Assigns higher SLOs to every service but cannot explain composition, correlation, or customer outcome.
Defines journey SLIs, dependency criticality, burn alerts, and shared review with actionable contracts.
Uses conditional contribution, fallback quality, correctness, marginal investment, and balanced caller-provider ownership.
Designs a trusted incentive system that changes portfolio priorities while improving end-to-end outcomes across organizational boundaries.