Interview prompt
Problem context
Skills being evaluated
Use the sequence below to surface constraints, choose boundaries, test failure behavior, and defend trade-offs. Concrete numbers are interview assumptions, not claims about a real production system.
Clarify the decision
- Separate operational history, legal audit evidence, and debugging replay because their retention and fidelity differ. Define clock semantics, acceptable reconstruction time, redaction obligations, and which actors or automated jobs must be attributable.
Establish scale assumptions
- Estimate events per mutation, seven-year storage, index growth, and reconstruction frequency. Use periodic snapshots so answering one old query does not replay millions of events from origin.
Functional and non-functional requirements
- Capture all mutation paths, preserve causal actor and reason, detect tampering, reconstruct state under the schema valid at the time, and support policy-compliant redaction without falsifying that an event occurred.
High-level architecture
- Write current state and an immutable audit envelope in one transaction or outbox. Store append-only events in partitioned durable storage, chain integrity hashes per partition, build searchable metadata indexes, and create versioned snapshots for reconstruction.
Data model and flow
- An envelope includes entity, tenant, actor, effective and recorded times, request and causation IDs, schema version, before/after patch or command, and policy labels. Reconstruct from the latest preceding snapshot, then apply verified events in order.
Consistency and transaction boundaries
- The business transaction and audit intent must commit atomically. Recorded time orders evidence within an authority; effective time supports late business corrections without rewriting history.
Failure modes and recovery
- A broken audit publisher blocks or quarantines high-risk mutations according to policy rather than silently dropping evidence. Gap detectors compare source transaction sequences to audit offsets and backfill from durable outbox history.
Security and privacy
- Encrypt by tenant and field class, restrict audit access by purpose, and record audit reads. Keep personal values in separately erasable encrypted payloads while retaining a tombstoned event envelope and integrity chain.
Observability and SLOs
- Measure audit gap, publication lag, integrity verification, reconstruction failures, snapshot age, and unauthorized access attempts. Periodically verify chains and restore historical partitions from backup.
Capacity and cost
- Tier old payloads to low-cost immutable storage, retain metadata indexes for likely queries, and snapshot based on replay cost. Compression and patches reduce storage but must remain decodable across schema evolution.
Alternatives and trade-offs
- Full before-and-after images simplify reconstruction but multiply sensitive storage; commands or diffs are compact but rely on versioned interpreters. Use envelopes plus periodic materialized snapshots and choose payload fidelity by regulatory need.
Evolution and migration
- Instrument every write path and reconcile coverage before claiming audit completeness. Introduce new envelopes alongside legacy logs, verify equivalent capture, then backfill only the history whose provenance can be defended.
What Staff and Principal candidates should emphasize
- Strong answers distinguish immutable evidence from immutable personal data. They cover atomic capture, two notions of time, schema evolution, gap detection, integrity proof, and the right to erase payloads.
Decision trade-offs
Event payload
Option A
Full before-and-after entity images
Option B
Versioned commands or field patches
Recommendation:Use compact patches with periodic snapshots for high-volume entities, but retain full evidence where regulators require exact rendered state.
Integrity
Option A
Database access controls only
Option B
Append-only storage plus cryptographic hash chains
Recommendation:Add tamper-evident chains for defensible evidence; controls prevent many changes but do not independently reveal privileged mutation.
Follow-up interview questions
- 01How do you erase personal data without breaking the integrity chain?
- 02How do you reconstruct state across five schema versions?
- 03What happens when the audit publisher is unavailable during a critical mutation?
- 04How do you prove a bulk migration emitted complete audit evidence?
Common weak answers and mistakes
- 01Equating event sourcing with compliant audit without actor, reason, integrity, or access controls.
- 02Using wall-clock timestamps alone as a reliable total order.
- 03Keeping every personal value forever because the audit log is called immutable.
- 04Claiming historical completeness without a source-to-audit gap detector.
Interviewer evaluation rubric
Adds an append-only table but cannot prove atomic capture, ordering, integrity, privacy, or schema-aware reconstruction.
Defines audit envelopes, atomic outbox capture, durable retention, snapshots, and controlled search access.
Handles effective versus recorded time, hash verification, erasable payloads, gap detection, versioned replay, and restore tests.
Separates multiple audit purposes, calibrates evidence fidelity and cost, and refuses to backfill history whose provenance cannot be defended.