hardSystem Design

How do you guarantee idempotency in a queue consumer?

Q: How do you guarantee idempotency in a queue consumer?

**TL;DR.** Message queues usually deliver **at-least-once**, so the same job can arrive **twice**. Make consumers **idempotent**: derive a stable **idempotency key**, **dedupe** processed ids, and use **upserts / conditional writes** so reprocessing produces **no extra side effects**. job #42 job #42 (dup) seen #42? <text

364 views

Understand the problem

Making at-least-once delivery safe to retry.

nodejsqueueidempotencyreliability

Attempt it yourself

Sketch your approach before reading the solution — that's what interviews test.

Stuck? AI Nudge Available

Get a conceptual hint to guide your logic without spoiling the final implementation.

Study the solution

The solution is waiting

Give it an honest attempt first — then compare your thinking with the full walkthrough.

Read the code

Dedupe with a unique key

async function handle(job) {
  try {
    await db.tx(async (t) => {
      await t.insert('processed_jobs', { id: job.id });  // unique PK
      await applyEffect(t, job);                          // same tx
    });
  } catch (e) {
    if (e.code === 'UNIQUE_VIOLATION') return;  // already handled → skip
    throw e;
  }
}

Join the discussion

Discussion (0)

No responses yet. Be the first to share what you think.

How do you build a worker-thread pool (e.g. Piscina) for CPU-bound work?

What is AsyncLocalStorage and what problem does it solve?