Financial
Idempotent Payment Processing API
Charge customers exactly once even when networks, clients, and servers retry aggressively.
Scale to anchor on
Millions of transactions per day, sub-second p99, zero tolerance for duplicate charges, full audit trail.
Requirements
Functional
- Authorize and capture payments idempotently.
- Support retries from any layer (mobile, server, queue).
- Surface stable status on subsequent calls.
Non-functional
- Exactly-once user-perceived semantics.
- Compliance and audit-ready logs.
- Reconcilable with upstream banks daily.
High-level architecture
Client supplies an idempotency key. The server records (key, request fingerprint, response) in the same transaction as the payment side effect. Duplicate keys return the stored response. Reconciliation runs against bank settlements daily.
Components
Idempotency store
Persistent (key → outcome) with bounded TTL.
Payment orchestrator
Wraps the call to the processor and persists outcome in one transaction.
Processor adapters
Network calls to card networks / banks with retries.
Reconciliation worker
Matches local ledger to bank settlement; flags drift.
Key decisions
Persist idempotency in same DB transaction as the effect.
Otherwise a crash between the two reintroduces double execution — the defining failure mode.
Fingerprint of request to detect key reuse with different body.
Catches client bugs where the same key is reused with mutated parameters.
Bounded TTL on idempotency rows.
Unbounded growth; choose the TTL longer than the longest realistic retry window.
Reconciliation as a first-class system.
Even with idempotency, settlement drift happens; reconciliation is the safety net.
Pitfalls
- Idempotency record stored outside the payment transaction.
- Hashing the request body as the key — small format changes break dedup.
- Ignoring processor-side retries that bypass your idempotency layer.
- No reconciliation, so silent drift becomes a regulatory event.
Follow-up questions
- How does the system behave on a network failure between you and the processor?
- How is the idempotency TTL chosen?
- How do you handle a partial settlement mismatch?
- How do you migrate the idempotency table without downtime?