DebuggingApril 18, 20267 min read

Debugging Failed Webhooks in Production Without Losing Your Weekend

A practical framework for debugging webhook delivery issues: what to look for in headers, status codes, timing, and payloads when your integration stops working.

WHAT THE PROVIDER DASHBOARD RECORDEDproviderdelivery log2xxarrived, then dropped after the response4xxyou rejected it: signature, WAF, wrong URL5xxyour handler threw at that exact timestamptimeouthandler too slow; move the work async
Start at the provider's own delivery log. The status it recorded tells you which layer to open next, and skips about thirty minutes of guessing.

A customer emails: "my Stripe webhook is broken." You check the Stripe dashboard and the events are firing. You check your server logs and there's nothing. Sentry, nothing. The events are going somewhere, just not where you expect.

This post is the step-by-step framework for debugging webhook delivery issues in production, based on the failure modes that come up most often.

TL;DR

  • Start with the provider dashboard: is the event firing, and what status is it getting back?
  • 4xx means you're rejecting it (signature, middleware, wrong URL)
  • 5xx means your handler crashed. Check error monitoring at that exact timestamp
  • A timeout means your handler is too slow. Move heavy work async and return 200 fast
  • The fix for "took 2 hours to debug" is one place that shows inbound, outbound, retries, and replay together, which usually means a webhook relay

The decision tree

Start here. It'll save you 30 minutes of guessing.

Q1: Is the provider actually sending the event?

Check the provider's dashboard (Stripe: Developers → Events; GitHub: Settings → Webhooks → Recent Deliveries; Shopify: Notifications → Webhooks → View Recent).

If the event isn't there, the trigger isn't firing, so go debug the upstream state that's supposed to generate it. If it is there, go to Q2.

Q2: Is the provider getting a 2xx back?

In the same dashboard, look at the response status from your endpoint.

  • 2xx: the provider thinks it succeeded. If your app never saw it, the event arrived and got dropped after the response. Check your async queue, your DB write, your handler logic.
  • 4xx: you're rejecting it. Most often a signature verification failure, a missing header, or middleware (WAF, rate limiter, auth layer) blocking the request.
  • 5xx: your endpoint crashed. Check error monitoring, and not just "the last hour" but the exact timestamp of this delivery.
  • Timeout: your endpoint is too slow. See Q4.
  • Connection refused or DNS: your endpoint is unreachable, which is a different problem living somewhere in network, DNS, or firewall.

Q3: If 4xx, which one?

  • 400, bad request. Usually a header mismatch, or your signature check threw before it could return a clear error. Check what your handler returns when the body is malformed or the signature is missing.
  • 401 / 403, you're rejecting the request as unauthorized. Signature verification failure? Middleware checking a user session on an endpoint that shouldn't need one?
  • 404, the URL is wrong. Did you deploy a route change that moved the webhook endpoint?
  • 413, payload too large. Shopify's bulk operations can send bodies over 1MB. Raise your body-parser limit.
  • 429, rate limiter tripped. Whatever is rate-limiting your webhook route shouldn't be. Most webhook providers come from a finite set of IPs, and your limiter may be treating them all as one attacker.

Q4: If timeout, what's slow?

GitHub gives you 10 seconds. Shopify gives you 5. Slack and Discord give you 3. Stripe never published a number; treat it as about 20. The full table, with a source per row, is the webhook provider reference.

waitUntil · THE 200 IS A PROMISE YOU CANNOT KEEPverifyreturn 200waitUntil(work)in memory onlyinstance recycled → gone, and the provider will never resendPERSIST FIRST · THE 200 IS TRUEverifydurable writereturn 200processretryable
The shape that keeps you inside every provider's timeout, regardless of how slow the real work is.

Most timeouts come from doing too much work in the handler. The usual offenders are synchronous database writes into a saturated pool, a call out to another service (Stripe, Salesforce, Notion, your own API) from inside the handler, sending an email or a Slack message synchronously, and logging the whole body to stdout on a log ingester that's backpressuring.

The fix pattern: verify the signature, enqueue the event, return 200. All business logic happens downstream, asynchronously.

What's usually actually broken

After a few years of watching this, here's the 80/20:

  1. Signature verification is re-serializing the body. See the signature verification guide
  2. A proxy or WAF is eating the webhook before it reaches your handler. Check Cloudflare, Vercel, or whatever your platform uses for request logs
  3. Rate limiting is tripping. Cloudflare's Bot Management can false-positive on legitimate webhooks
  4. The endpoint URL changed in a deploy and nobody updated the provider
  5. An env var is missing on production (signing secret, DB URL) and the handler silently 500s
  6. A background job is crashing after the webhook returns 200. The provider thinks it's fine, but nothing is getting processed

Number 6 is endemic on serverless, where work scheduled after the response has no durability guarantee at all: Receiving Webhooks on Vercel Without Losing Them. And if the symptom is a duplicate rather than a miss, that's usually working as designed: You Received the Same Webhook Twice.

Tools that actually help

Provider dashboards are good for one event at a time and bad for patterns. If you want "all failed events in the last 24 hours," you're scrolling manually.

ngrok and webhook.site are for local dev. They don't help you in production.

Your own logs help if you logged the right thing. If you logged "webhook received for event evt_xxx" before the signature check, you at least know it arrived. Most people log after, which makes signature-failure events invisible.

A webhook relay like AnyHook sits in front of your handler and logs headers, body, timing, response status, and attempt count. The debugging loop becomes: look at the event stream, click the failing one, inspect the raw payload, replay it.

The AnyHook debugging loop

When a user emails AnyHook support with "my webhook isn't working," the dashboard usually makes the cause obvious in under a minute:

  1. Open the event stream, filter to status=failed
  2. Click the failing event
  3. Look at inbound headers (was the signature there?), inbound body (was it the shape you expected?), outbound response (what did your server say?), and the timing breakdown (edge → queue → delivery)
  4. If it's a one-off, a bad deploy or a flaky network, click Replay and watch it succeed
  5. If every event is failing, you now know exactly what to fix, and your events sit safely stored until you do

What turns a 2-hour debugging session into a 2-minute one is having the request your provider actually sent, the response your server actually returned, and the timestamps between them all in one place. Without that you're reconstructing it from three dashboards and your own logs.

Takeaway

If this is your Sunday afternoon more often than you'd like, AnyHook turns most of it into a dashboard click.

All postsApril 18, 2026 · 7 min

More guides

Stop losing webhooks.

Change one URL. Get retries, event log, and one-click replay.

Start freeMore posts