Stop Processing the Same Webhook Twice

Webhook delivery is "at least once." That's not a bug in any specific provider. It's the nature of the protocol: when a provider fires a webhook at your endpoint, it waits for a 2xx response. If that response doesn't arrive in time, the provider has no way to know whether your server received the event or not, so it retries.

Most of the time this is fine. Your endpoint responds quickly, the provider logs a success, nothing is sent twice. But networks are unreliable and deploys cause brief outages. In those cases, your server may have already processed the event when the retry arrives.

What goes wrong

Consider a charge.succeeded event from Stripe. Your handler:

1. Verifies the signature
2. Looks up the order
3. Marks it as paid
4. Sends a confirmation email to the customer

If Stripe retries that event because your first response was slow, step 4 fires again. The customer gets two confirmation emails. Depending on what else your handler does (provisioning accounts, updating inventory, triggering fulfilment), the consequences can be worse.

This isn't a Stripe-specific problem. GitHub does not automatically redeliver failures; manual redelivery or a recovery script can produce duplicate deliveries. Shopify will retry across multiple attempts. Any provider that cares about reliable delivery does this, because the alternative is silently dropping events.

The standard advice: make your handlers idempotent

The usual recommendation is to make your webhook handler idempotent. Check whether you've already processed this event before doing anything. Most providers give you a stable event identifier you can use as a deduplication key.

// Stripe
const eventId = event.id; // evt_1234...

// GitHub
const deliveryId = req.headers['x-github-delivery']; // UUID

// Shopify
const deliveryId = req.headers['x-shopify-webhook-id']; // UUID

Before processing, check your database for this ID. If it's already there, return 200 and skip the handler logic. Simple in theory.

In practice, most teams only add this after they've been bitten. The check requires a database round-trip before every handler invocation, which is easy to forget to add and easy to implement incorrectly under concurrent load.

The race condition

Even with an idempotency check, there's a window between "check if processed" and "mark as processed" where two concurrent deliveries of the same event can both pass the check. Closing this window properly requires either a database unique constraint on the event ID column (which turns duplicate inserts into errors you can catch) or a distributed lock. Neither is trivial to get right.

Detecting duplicates at the ingest layer

A cleaner approach is to detect duplicates before the event reaches your handler at all. If the ingest layer knows that event evt_1234 has already been received and processed, it can return 200 to the provider without forwarding the event downstream.

This is what Webhook Watchtower does. When an event arrives, it checks whether a matching event has already been received for that source. For Stripe, GitHub, and Shopify, the check uses the provider's own event ID: exact match, unlimited lookback. Custom sources fall back to a SHA-256 hash of the raw request body within a 24-hour window.

Duplicates are stored with a Duplicate label linking back to the original, but they're not forwarded and don't count toward your monthly usage.

Why provider retries don't cause duplicates in practice

When Watchtower receives an event, it returns 200 to the provider immediately. From the provider's perspective, delivery succeeded. Retries are only triggered when the provider doesn't get a 2xx back.

This means that in normal operation, provider-side retries are rare. The duplicate detection layer handles the edge cases: network hiccups during the first delivery, manual resends during testing, and the occasional provider bug.

Your handler still needs idempotency

Deduplication at the ingest layer handles provider retries. It doesn't handle the case where your own code triggers the same action twice, or where you replay an event intentionally and your handler needs to be safe to run again.

Idempotency in your application code is still worth building correctly. The benefit of catching duplicates early is that you stop paying for them (compute, database writes, outbound notifications) before they ever reach your system.