Skip to main content

Webhook Dispatcher — Application Logic

Status: populated Owner: Platform Engineering Last updated: 2026-04-18 Companion: DOMAIN_MODEL · DATA_MODEL · FAILURE_MODES

1. Dispatch Pipeline (NATS Consumer Path)​

[NATS webhook.dispatch]
        │
        ▼
  1. Deserialise + validate (Zod)
        │
        ▼
  2. Lookup active WebhookConfigs for accountId + event filter match
        │
   ┌────┴────┐
 none      one or more
   │            │
  ACK      3. For each WebhookConfig:
               Generate deliveryId (UUID, stable per eventId+webhookId)
               Insert delivery_attempts row (status=PENDING, attempt_number=1, scheduled_at=now())
               ACK NATS message
                    │
                    ▼
              4. Execute first attempt immediately (async post-ack)
                    │
              ┌─────┴─────┐
            2xx         non-2xx / timeout
              │                │
         5a. Mark SUCCESS  5b. Schedule retry
                               Update delivery_attempts:
                               status=FAILED_RETRY, next_retry_at=now()+delay

2. Retry Worker (Scheduled Poller)​

Runs every 10 seconds in-process:

SELECT da.*, wc.url, wc.secret_enc
FROM hook.delivery_attempts da
JOIN hook.webhook_configs wc ON wc.webhook_id = da.webhook_id
WHERE da.status IN ('PENDING','FAILED_RETRY')
  AND da.next_retry_at <= now()
ORDER BY da.next_retry_at ASC
LIMIT 50
FOR UPDATE SKIP LOCKED;

For each row returned:

  1. Decrypt wc.secret_enc → plaintext secret
  2. Reconstruct payload from da.payload_snapshot
  3. Execute HTTP delivery (see §3)
  4. Update delivery_attempts based on result

3. HTTP Delivery​

async function deliver(url: string, secret: string, payload: object, deliveryId: string): Promise<DeliveryResult> {
  const body = JSON.stringify(payload);
  const timestamp = Math.floor(Date.now() / 1000);
  const signature = crypto.createHmac('sha256', secret).update(body, 'utf8').digest('hex');

  const response = await fetch(url, {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      'X-Ghasi-Signature': `sha256=${signature}`,
      'X-Ghasi-Event': payload.event,
      'X-Ghasi-Delivery-Id': deliveryId,
      'X-Ghasi-Timestamp': String(timestamp),
    },
    body,
    signal: AbortSignal.timeout(5_000),  // 5 second timeout
    redirect: 'manual',                   // do not follow redirects
  });

  return {
    success: response.status >= 200 && response.status < 300,
    httpStatusCode: response.status,
    responseBody: (await response.text()).slice(0, 512),
  };
}

Redirect handling: HTTP 3xx responses are treated as failure; redirect: 'manual' prevents automatic following.

4. Retry Backoff Schedule​

attemptNumbernext_retry_at delay
1 (first)Immediate
2+ 30 seconds
3+ 5 minutes
4+ 30 minutes
5+ 2 hours
5 (exhausted)Dead-letter; next_retry_at += 8 hours (deadline for alerting reference only)
const RETRY_DELAYS_MS = [0, 30_000, 300_000, 1_800_000, 7_200_000];

function nextRetryAt(attemptNumber: number): Date | null {
  if (attemptNumber >= 5) return null; // dead-letter
  const delayMs = RETRY_DELAYS_MS[attemptNumber]; // 0-indexed: attempt 1 → index 0
  return new Date(Date.now() + delayMs);
}

5. Dead-Letter Path​

When attempt 5 fails:

  1. Update delivery_attempts: status = DEAD_LETTER, next_retry_at = NULL.
  2. Publish webhook.dispatch.deadletter to NATS (direct publish, not via outbox).
  3. Increment hook_deliveries_dead_lettered_total counter.
  4. Log WARN hook.dead_lettered with deliveryId, webhookId, accountId, lastHttpStatus.

6. Event Filter Matching​

When processing a webhook.dispatch event with dlrStatus = 'FAILED':

const eventType = dlrStatusToWebhookEventType(dlrStatus); // 'DLR_FAILED'
const matchingWebhooks = webhooks.filter(w =>
  w.isActive && w.events.includes(eventType)
);

A webhook with events: ['DLR_DELIVERED'] does NOT receive DLR_FAILED dispatches.

7. NATS Consumer Configuration​

const consumerConfig = {
  name:            'webhook-dispatcher',
  durable_name:    'webhook-dispatcher',
  ack_policy:      AckPolicy.Explicit,
  ack_wait:        15_000_000_000,  // 15s (short: Ack before HTTP attempt)
  max_ack_pending: 20,
  deliver_policy:  DeliverPolicy.All,
  filter_subject:  'webhook.dispatch',
};

Note: NATS message is Acked after DB write (before HTTP attempt). Retry is DB-driven, not NATS-driven.

8. REST API Logic​

POST /v1/webhooks​

  1. Validate request body with Zod schema.
  2. Check account webhook count ≤ 10.
  3. Encrypt secret with AES-256-GCM using KMS-derived key.
  4. Insert into hook.webhook_configs.
  5. Return created record (secret not returned).

PUT /v1/webhooks/:id​

  1. Verify webhook belongs to requesting accountId.
  2. If secret updated: re-encrypt before persisting.
  3. Update updated_at = now().

DELETE /v1/webhooks/:id​

  1. Verify ownership.
  2. Hard delete from webhook_configs.
  3. Pending delivery_attempts for this webhook retain their rows but will not be retried (webhook config gone — retry worker skips missing webhook).