All posts Notifications

How we sign webhooks, and why we refuse redirects

An outgoing webhook lets anyone point our server at any URL. Here is how Butters signs JSON deliveries so your receiver can trust them, how to verify one in Node or Python, and why the delivery code refuses private addresses and never follows a redirect.

7 min read
Node.js Security Webhooks
A white envelope with a blue wax seal showing a key, beside a blue shield with a coral check mark

An outgoing webhook is a strange kind of feature. You give strangers a text box, they type a URL into it, and from then on your server makes HTTP requests to wherever that URL points. Two problems come with that.

The first is on the receiving end. Anyone who learns your endpoint's address can post to it, so your receiver needs a way to tell a real Butters delivery from a forged one.

The second is on our end. If the URL box accepts anything, it becomes a way to make our server send requests into places it should never reach.

This post covers how Butters handles both: how JSON deliveries are signed and how to check them, and why the delivery code refuses private addresses and never follows a redirect.

What gets signed

Outgoing webhooks come in three formats: JSON, Slack and Discord. Only JSON deliveries are signed. Slack and Discord incoming-webhook URLs carry their own secret in the URL itself, and those services don't check signatures anyway.

Each JSON destination gets its own signing secret when you create it. It looks like whsec_ followed by 32 random bytes in base64. You'll find it on the destination's row in project settings, which is the only place it's shown. At rest it's encrypted with AES-256-GCM.

Every signed request carries three headers:

  • webhook-id: evt_<event id> for a real delivery. It stays the same across retries of that delivery. A Test send from settings uses evt_test_<unix ms> instead, because there's no real event behind it.
  • webhook-timestamp: the Unix time in seconds when this attempt was made. Each retry gets a new one.
  • webhook-signature: v1, followed by a base64 HMAC-SHA256.

The HMAC key is the part of your secret after whsec_, base64-decoded. The signed message is the id, the timestamp and the raw request body, joined with dots: 

<webhook-id>.<webhook-timestamp>.<raw body>

That's the Standard Webhooks scheme, unchanged. If your language has a Standard Webhooks library, use it and skip the rest of this section.

Checking a signature in Node

Two details matter more than the rest.

Use the raw body. The signature covers the exact bytes Butters sent. If your framework parses the JSON and you stringify it again, the key order or whitespace can change and every check fails. In Express that means express.raw({ type: 'application/json' }) on this route, not express.json().

Compare in constant time. A plain === stops at the first byte that differs, and the time that takes can leak how much of a forged signature was right. timingSafeEqual doesn't. 

import { createHmac, timingSafeEqual } from 'node:crypto'

const TOLERANCE_SECONDS = 5 * 60

export function verifyWebhook(secret, headers, rawBody) {
  const id = headers['webhook-id']
  const timestamp = headers['webhook-timestamp']
  const signature = headers['webhook-signature']
  if (!id || !timestamp || !signature) return false

  const age = Math.abs(Date.now() / 1000 - Number(timestamp))
  if (!Number.isFinite(age) || age > TOLERANCE_SECONDS) return false

  const key = Buffer.from(secret.replace(/^whsec_/, ''), 'base64')
  const expected = createHmac('sha256', key)
    .update(`${id}.${timestamp}.${rawBody}`)
    .digest()

  // Standard Webhooks allows several space-separated signatures.
  return signature.split(' ').some((part) => {
    const [version, value] = part.split(',')
    if (version !== 'v1' || !value) return false
    const given = Buffer.from(value, 'base64')
    return given.length === expected.length && timingSafeEqual(given, expected)
  })
}

Butters only ever sends one signature. The loop is there because the Standard Webhooks format allows several, and it costs nothing to accept them.

The length check before timingSafeEqual isn't optional. That function throws if the two buffers differ in length, and a forged header can be any length.

The same check in Python: 

import base64, hashlib, hmac, time

def verify_webhook(secret, headers, raw_body: bytes) -> bool:
    msg_id = headers.get("webhook-id")
    timestamp = headers.get("webhook-timestamp")
    signature = headers.get("webhook-signature")
    if not (msg_id and timestamp and signature):
        return False
    if abs(time.time() - int(timestamp)) > 5 * 60:
        return False

    key = base64.b64decode(secret.removeprefix("whsec_"))
    signed = f"{msg_id}.{timestamp}.".encode() + raw_body
    expected = base64.b64encode(hmac.new(key, signed, hashlib.sha256).digest()).decode()

    for part in signature.split(" "):
        version, _, value = part.partition(",")
        if version == "v1" and hmac.compare_digest(value, expected):
            return True
    return False

Replays are yours to stop

The timestamp is signed along with the body, so nobody can change it without breaking the signature. The five-minute window in the examples above means a captured request can't be replayed next week.

That window is enforced by your receiver, not by us. Butters stamps the time and signs it, and whether an old delivery gets rejected is up to the code above. Five minutes is the common choice. Keep your server's clock synced if you go much tighter.

Inside the window, dedupe on webhook-id. A delivery that timed out on our side may still have reached you, and the retry carries the same id so you can drop the repeat. Store the ids you've processed for longer than the window and you're covered against both retries and replays.

Rotating a secret

If a secret leaks, click Rotate secret on the destination. The old secret stops working immediately. There's no overlap period where both are accepted, so deliveries will fail verification until your receiver has the new one. Rotate when you can update the receiver right away, and expect a few rejected deliveries in between.

The other direction: where we'll send

Now the second problem. Signup is open, so anyone can create a project and type any URL into the destination form. Without a check, that URL can point at things only our server can reach: a database on the private network, an admin port on localhost, or the cloud metadata service at 169.254.169.254 that hands out credentials to whatever asks. The server makes the request, and the error message on the settings page shows whoever typed the URL a piece of the response.

This is server-side request forgery, or SSRF. Butters guards against it in three ways.

Private addresses are refused. Before a URL is saved, and again before every delivery attempt, the host is resolved and checked. Anything in these ranges is rejected:

  • 0.0.0.0/8, loopback 127.0.0.0/8, and the private ranges 10/8, 172.16/12 and 192.168/16
  • link-local 169.254/16, which includes cloud metadata
  • carrier-grade NAT 100.64.0.0/10
  • multicast and reserved space, 224.0.0.0 and up
  • in IPv6: unique-local fc00::/7, link-local fe80::/10 and multicast ff00::/8

A hostname is blocked if any of the addresses it resolves to is private, not just the first one.

Other spellings of the same address are caught. An IP address has more than one way to be written. 127.0.0.1 can also appear as a single decimal number, in hex, with parts left out, or wrapped inside an IPv6 address. The URL parser turns the IPv4 forms back into the usual dotted form before the guard sees them. The IPv6 forms needed their own handling: IPv4-mapped (::ffff:…), IPv4-compatible (::…) and the NAT64 prefix 64:ff9b::/96 are unwrapped and the IPv4 address inside is checked against the list above. Anything the guard can't parse is refused rather than guessed at.

The check runs again on every attempt. DNS isn't fixed. A name that pointed at a public address when you saved it can point at 127.0.0.1 a minute later, and the URL can be edited after it was saved. So the guard runs at save time to give you a clear error in the form, and again right before every attempt, including each retry.

Why redirects are refused

A redirect undoes all of that. The guard approves https://some-public-host.example, and that host answers with a 302 to a private address. If the HTTP client follows it, the request lands somewhere the guard never saw.

So every delivery is sent with redirects turned off. Any 3xx response counts as a failure, isn't retried, and records this error on the destination: Endpoint redirected (302); webhook URLs must be final. The ntfy delivery code does the same.

In practice this bites in one common case: a URL typed as http:// for a site that redirects to https://, or one missing a trailing slash the server insists on. Paste the final URL and it works.

Self-hosting on a private network

If you run Butters yourself and your receiver really is on a private network, like a Docker network, a VPN or a self-hosted ntfy server, set: 

ALLOW_PRIVATE_HOSTS=true

This turns the guard off for every destination on the instance. Only do it when you trust every account on it.

What this doesn't cover

Two limits worth knowing:

  • The guard resolves the hostname, then the HTTP request resolves it again. Those are separate lookups. A hostname with a very short DNS lifetime could in principle answer differently between them. Closing that fully means connecting to the exact address the guard checked, which the delivery code doesn't do today.
  • Delivery isn't durable. Retries happen in memory: three attempts, one second and then five seconds apart. If the server restarts mid-retry, that delivery is lost. The last error and last-sent time on the settings row are the only record.

The short version

For receivers: verify webhook-signature against the raw body using a constant-time compare, reject timestamps more than five minutes old, and dedupe on webhook-id.

For destination URLs: give Butters the final, public URL. Anything private, anything that redirects, and anything the guard can't read will be refused, and the settings row will tell you why.

Continue reading