Payments / Design note (conceptual)
A timeout is not a failed payment.
Designing for uncertain outcomes in an asynchronous payment flow.
- 01Client + key
- 02Payment record
- 03Provider adapter
- 04Reconciliation
01 Problem: the response can disappear
A network timeout cannot tell us whether a provider accepted a payment. Retrying immediately can move money twice. The system needs to distinguish a rejected request from an unknown outcome.
02 Make the request durable
Scope an idempotency key to the caller and operation, enforce uniqueness in the database, and store a fingerprint of the request. Reusing a key with different input should fail. Persist a pending payment before calling a provider; return the existing result for a repeat request.
03 Handle callbacks as untrusted, repeatable events
Authenticate the provider callback, validate amount and currency, and deduplicate its event identifier. Apply allowed state transitions in a transaction using a lock or conditional update. A late pending event must not overwrite a confirmed success.
04 Reconcile before retrying
For an unknown outcome, query the provider or reconcile against settlement records. Use bounded retries with backoff for safe status checks. If the provider cannot confirm an outcome, route it to manual review instead of guessing. A provider idempotency key helps only if that provider actually guarantees its behavior.
05 Keep ledger updates atomic
Where a ledger is required, balanced debit and credit entries should commit together with the local payment transition. An outbox can publish follow-up events after that commit; consumers still deduplicate. This provides atomic local writes, not exactly-once delivery across services.
06 Trade-off: more states, more operational work
Asynchronous processing improves recovery but introduces pending states and support workflows. Track pending age, callback failures, reconciliation mismatches, and correlation identifiers. Test duplicate callbacks, concurrent requests, lost responses, and out-of-order events before treating the flow as reliable.