Webhook Rate Limiting: Protecting Your Endpoint from Bursts

A payment provider goes down for twenty minutes. When it recovers, every failed delivery in its retry queue fires at once. Your endpoint, which was handling a comfortable trickle of events all morning, suddenly gets a week's worth of traffic in ninety seconds.

This isn't a hypothetical. It's the single most common cause of "the webhook integration that worked fine for months suddenly fell over."

Where bursts actually come from

Provider-side outage recovery. When a provider's own infrastructure has a bad day, failed deliveries queue up on their end. The moment they recover, everything queued gets sent, often within the same short window, to every affected customer's endpoint simultaneously.

Bulk operations you didn't trigger. A merchant re-imports their entire product catalog into Shopify. A customer bulk-updates ten thousand records through an admin panel. Neither event was caused by you, but both can generate thousands of webhook events aimed at your endpoint in a burst.

Your own downtime. If your endpoint was down for maintenance and the provider retries failed deliveries, recovery looks identical to an external outage from your side: a wall of retries arriving the moment you come back up.

Why rejecting requests makes it worse

The instinct when traffic spikes is to add a rate limiter that returns 429 past a threshold. For inbound webhooks, this usually backfires.

A rejected webhook doesn't go away. Most providers treat a non-2xx response as a delivery failure and retry it, often on a schedule you don't control. Reject during a burst and you're not reducing load, you're converting each rejected request into a future retry, stacking more load onto whatever window you retry into next.

Burst arrives → you reject with 429 → provider retries later
             → burst arrives again → you reject again → ...

The volume doesn't shrink. It just gets deferred and, if your rejection threshold stays the same, deferred again.

Accept fast, process at your own pace

The pattern that actually works: accept every request quickly regardless of volume, and let a queue control the rate at which your system does real work on them.

app.post('/webhooks/incoming', async (req, res) => {
  const event = verifySignature(req);

  // Accept immediately, no matter how many arrived this second
  await queue.add('process-event', event);
  res.status(200).send('ok');

  // Downstream workers pull from the queue at whatever pace
  // your database, APIs, and other dependencies can sustain
});

This decouples "how fast can we accept requests" (very fast, since enqueueing is cheap) from "how fast can we process them" (bounded by your slowest downstream dependency). A burst turns into a temporarily deep queue instead of a wave of timeouts and failed deliveries.

Sizing your workers to the real bottleneck

Once processing is decoupled from ingestion, the question becomes how many workers you run and how fast they should go. This usually comes down to whatever's slowest in your actual processing path: a database write, a third-party API with its own rate limit, or a downstream service that can't be scaled as easily as your worker count.

const worker = new Worker(
  'process-event',
  async (job) => processEvent(job.data),
  {
    connection: redis,
    concurrency: 10, // tune to what your DB and downstream APIs can absorb
  }
);

Ten concurrent workers processing a ten-thousand-event burst clears it in a predictable, bounded time. The same burst hitting a synchronous handler with no queue in front of it produces exactly the timeout cascade this whole approach is meant to avoid.

Monitoring queue depth, not just error rates

A queue absorbing a burst gracefully looks, from an error-rate dashboard, identical to a queue that's fine. The signal you actually want is queue depth and processing lag: how far behind is the queue right now, and is that number growing or shrinking?

A queue depth that climbs steadily during a burst and drains back to zero within the expected window is working as designed. One that keeps climbing after the burst has clearly ended means your worker throughput can't keep up with sustained load, not just spikes, and needs more capacity rather than just patience.

---

Bursts are a capacity problem, not a correctness problem. Accept fast, queue, process at a rate your downstream systems can actually sustain, and watch queue depth rather than just error counts.

Webhook Watchtower puts a queue in front of your endpoint by default, so a burst from your provider lands there first instead of hitting your infrastructure directly. See the retries and delivery docs for how delivery and backoff work underneath it.