**What idempotency means (brief)**Idempotency for HTTP APIs means repeated identical requests have the same effect as a single request — no duplicate side-effects. For payments this prevents double-charges when clients retry due to timeouts or network errors.**Why it matters for reliability**- Prevents duplicate transactions and financial loss.- Makes clients safe to retry; improves user experience.- Simplifies error handling and improves system correctness.**Strategy 1 — Redis idempotency key (cache + TTL + lock)**- Flow: - Client sends an Idempotency-Key header. - Server does SET idempotency:<key> <placeholder> NX PX <processing-ttl> to claim processing. - If SET returns OK, proceed; if not, read stored result or return "in-flight". - On success store final response (or pointer) in Redis with a longer TTL.- TTL / cleanup: - Processing TTL short (e.g., 30s–2min). Final result TTL longer (e.g., 24–72h) to allow retries. - If processing TTL expires, another worker can retry; ensure eventual consistency.- Concurrency: - SET NX prevents two workers from processing simultaneously. Use atomic GET/SET or Lua scripts to ensure atomic replace of placeholder with result.Example (Node.js Redis):javascript
// assume redis client supports SET with NX and PX
const lock = await redis.set(`idem:${key}`, 'PROCESSING', { NX: true, PX: 120000 });
if (!lock) {
const cached = await redis.get(`idem:${key}:result`);
if (cached) return JSON.parse(cached);
// respond with 409 or wait/poll
}
try {
// process payment
await redis.set(`idem:${key}:result`, JSON.stringify(response), { PX: 3*24*60*60*1000 });
} finally {
await redis.del(`idem:${key}`); // release lock
}
**Strategy 2 — Durable DB idempotency (unique constraint + stored response)**- Schema: payments(id, idempotency_key UNIQUE, status, amount, provider_id, created_at, response_json)- Flow: - Begin transaction. INSERT INTO payments (idempotency_key, ...) VALUES (...) RETURNING *. - If insert succeeds, process payment and UPDATE row with provider details and response. - If insert fails with unique_violation, SELECT the existing row and return stored outcome.- TTL / cleanup: - Keep idempotency entries for a retention window (e.g., 90 days) for dispute handling, then archive/delete. - Use background jobs to purge or move old rows.- Concurrency: - UNIQUE constraint + transactional insert is the strongest guarantee. Two concurrent inserts: one succeeds, other gets unique_violation — safe.Example (Node.js + pg):javascript
await client.query('BEGIN');
try {
const res = await client.query(
`INSERT INTO payments (idempotency_key, amount, status)
VALUES ($1,$2,'processing') RETURNING *`,
[key, amount]
);
// do payment provider call, then UPDATE payments SET status='succeeded', response_json=$1 WHERE id = $2
await client.query('COMMIT');
} catch (err) {
await client.query('ROLLBACK');
if (err.code === '23505') {
// unique violation: fetch existing payment and return stored result
} else throw err;
}
**Choosing between strategies / trade-offs**- Redis: low-latency, good for high throughput and short-term storage; requires reliable Redis and careful TTL to avoid orphaned locks.- DB unique constraint: strongest durability and simple semantics; slightly higher latency and more DB storage; recommended for payments where durability matters.**Handling concurrent duplicate requests (practical advice)**- Prefer DB-unique + transactional insert for payments to ensure exactly-one creation.- Use Redis locks only as a fast guard; always fall back to DB atomic semantics.- Always return the original response (status and body) for duplicate keys instead of reprocessing.- Log and monitor idempotency metrics (hits, collisions, expirations) and surface failed renewals for manual review.