This refactors our inbound pipeline into a more future-proof, extensible system.
31 KiB
Outbound Webhooks
For every inbound message Messages can POST a notification to an HTTP
endpoint of your choosing. This page documents the on-the-wire format so
receivers can be implemented against a stable contract.
Webhooks are outbound — Messages calls out to your endpoint. Anything
inbound (third parties calling into Messages) goes through other
channel types (api_key, widget, mta, etc.) and is not the subject
of this document.
When does it fire?
A webhook channel has a single trigger: the point in a message's
lifecycle that fires it. The event name says both when it fires and
whether it can influence delivery — there's no separate blocking flag,
so invalid combinations can't be expressed:
trigger |
Fires | Blocking? | is_spam |
|---|---|---|---|
message.inbound |
The message just arrived, before the spam check | yes (sync) | pending |
message.delivering |
After the spam verdict, while delivery is in flight | yes (sync) | known |
message.delivered |
After the message has landed in the mailbox | no (async) | final |
Future lifecycle events (e.g. message.sent) are added as new
trigger values.
message.inbound and message.delivering are synchronous: they
run inline on the pipeline worker and get to shape delivery — drop the
message, or return a small JSON body that overrides the spam verdict
and/or attaches labels to the resulting thread (see
Response contract). They can't ask to be retried
through the body: a webhook that needs the message redelivered returns a
non-2xx status, which is treated as a transient failure → RETRY.
message.delivered is asynchronous — fire-and-forget; failures are
logged and the pipeline continues unchanged. Because it can't influence
delivery, it doesn't run on the pipeline worker at all: the channel is
recorded during the pipeline (capturing the final is_spam) and the
actual POST is handed to a background task after the Message is
persisted. The task renders the body from the stored message, so
nothing is copied or sent through the broker. Two consequences:
- It fires only for messages that become a
Message— not for one a blocking webhook later drops. (Spam is not a drop: it still lands in the spam folder, so it still fires, withX-StMsg-Is-Spam: true.) - It always runs after the spam step, so its
X-StMsg-Is-Spamis the final verdict.
Channel scopes
A webhook channel can be configured at three scopes:
scope_level |
Fires on | How to create |
|---|---|---|
mailbox |
Messages delivered to that mailbox | Mailbox admin via the Integrations modal |
maildomain |
Messages delivered to any mailbox of the domain | Maildomain admin via API / admin |
global |
Every message on the instance | Superuser via the Django admin or CLI |
A given inbound message fans out to every matching channel.
global is intentionally not creatable through the public REST API —
it's a sensitive instance-wide hook.
Configuration
A webhook channel stores its configuration in Channel.settings
(a JSON dict):
{
"url": "https://example.com/inbox-hook",
"trigger": "message.delivered",
"format": "eml",
"auth_method": "jwt"
}
| Key | Type | Default | Description |
|---|---|---|---|
url |
string | required | https:// endpoint. Rejected at create/update if it resolves to an internal address or doesn't resolve, and re-validated by the SSRF guard (with IP pinning) at each call. http:// is accepted only when Django DEBUG is on (the local-dev escape hatch). |
trigger |
string | required | message.inbound, message.delivering, or message.delivered (see When does it fire?). |
format |
string | eml |
eml, jmap, or jmap_metadata (see Payload formats). |
auth_method |
string | required | jwt or api_key (see Authentication). |
The serializer validates every change to settings, on create and
on settings-only PATCH — there is no path that lets a malformed value
slip onto an existing channel.
HTTP request shape
Every call is:
POSTtosettings.url.User-Agent: Messages-Webhook/1.0.- 30-second timeout.
- HTTP
3xxis followed (small hop limit), but every hop is re-validated and re-pinned by the SSRF guard and thePOSTis re-issued (method + body preserved), so a receiver behind a load balancer or URL canonicaliser still gets the signed payload — and a redirect can't point the delivery at an internal target. - The destination hostname/IP must pass the shared SSRF check (no loopback, link-local, private, multicast, reserved, or cloud metadata addresses; no IP literals).
Authentication
Every webhook channel has one root secret, minted server-side,
returned exactly once at create time and rotatable by POSTing to the
channel's regenerate-secret/ action. That action's path prefix
depends on the channel's scope: a mailbox-scoped channel is reached via
the mailbox route as POST /mailboxes/{mailbox_id}/channels/{id}/regenerate-secret/, while a
caller's own channels are at POST /users/me/channels/{id}/regenerate-secret/. The auth_method
setting picks how that root is presented on each POST. The root itself
never travels on the wire.
auth_method |
Headers sent | Wire value | Receiver verifies |
|---|---|---|---|
jwt |
Authorization: Bearer <HS256 JWT> |
HS256 JWT keyed by the root | Verify the JWT with the root using any JWT library; it binds the exact body via the body_sha256 claim and expires (exp, 5 min) with a jti nonce. |
api_key |
Authorization: Bearer <whk_…> |
whk_ + HMAC-SHA256(root, "messages.webhook.api_key.v1").hex() |
Constant-time compare of the Bearer token against the receiver's stored copy (an opaque static key, not a JWT). |
A channel sends only the headers for its configured method — the unused presentation never rides on the wire, so it can't leak through receiver-side proxies or debug panes. The API-key value is a one-way derivation of the root, so a receiver-side leak of the API key reveals nothing about the root: JWT verification (and the API-key derivation) on other receivers stays unforgeable.
Picking a method
jwt— best when the receiver controls a server (n8n, your own Lambda, a Flask/Express app, Cloudflare Worker). Verify with stock JWT libraries; the token binds the exact body (body_sha256) and expires.api_key— for low-code receivers that can only do a static header-equals-value check (Zapier, IFTTT, a Zap webhook step): they just compareAuthorizationagainst the storedBearer <whk_…>value.
Switching methods on an existing channel
PATCH the channel's settings.auth_method. The root secret is not
rotated — only the wire presentation changes — but the receiver was
given the old method's credential at creation. To get the new method's
credential, call the channel's regenerate-secret/ action (scoped path
as above): the
response returns either secret (jwt) or api_key (api_key),
matching the channel's current method. Rotation invalidates
the previous credential, so update the receiver before the next inbound
message lands.
Envelope headers (always set, regardless of format)
| Header | Value |
|---|---|
Content-Type |
message/rfc822 for eml, application/json for both JMAP variants |
X-StMsg-Trigger |
The lifecycle event that fired (message.inbound / message.delivering / message.delivered). Route on this — it says what happened and, implicitly, whether the webhook blocked. |
X-StMsg-Instance |
Public base URL of the originating instance (e.g. https://messages-public-url.example.com). Only present when the instance sets INSTANCE_URL. Combine with the *-Id headers to build callback API URLs. |
X-StMsg-Channel-Id |
UUID of the firing webhook Channel |
X-StMsg-Mailbox |
Destination mailbox address |
X-StMsg-Mailbox-Id |
UUID of the destination Mailbox (the API is keyed by this, not the address) |
X-StMsg-Recipient |
Envelope RCPT TO (usually the same as X-StMsg-Mailbox) |
X-StMsg-Is-Spam |
true, false, or pending (pending for message.inbound, which fires before the spam check) |
X-StMsg-Message-Id |
UUID of the stored Message — non-blocking only (see note) |
X-StMsg-Thread-Id |
UUID of the Message's Thread — non-blocking only |
The MIME message-id is not sent as a header — every body format
already carries it (messageId in the JMAP variants, the raw
Message-ID: header in eml).
X-StMsg-Mailbox-Id, X-StMsg-Message-Id and X-StMsg-Thread-Id are the
platform's own ids (the API is keyed by UUID, not by address). To call back
into the API, join them to the instance base URL — sent as X-StMsg-Instance
when configured — e.g.
GET {X-StMsg-Instance}/api/v1.0/mailboxes/{X-StMsg-Mailbox-Id}/….
X-StMsg-Message-Id / X-StMsg-Thread-Id are only present on non-blocking
webhooks, which fire after the Message is persisted; blocking webhooks
run before it exists, so they can't carry them.
Response contract
The classification below applies to blocking webhooks. Non-blocking webhooks treat every outcome as success — their bodies are ignored.
The three possible decisions are about the inbound email itself:
- CONTINUE — deliver it: the
Messageand its thread are created normally. - DROP — discard it: no
Messageand no thread are created, so the recipient never sees the email. The original sender is not notified (the inbound SMTP transaction was already accepted). The short-lived internalInboundMessageprocessing row is also removed — but that happens on every terminal outcome, including normal delivery, so "theInboundMessageis deleted" is not what makes DROP special; the email never landing is. - RETRY — keep the email queued and re-fire the webhook on the next 5-minute sweep (bounded by the deferral window below).
A webhook error never drops the email. The only way a blocking
webhook discards a message is by explicitly returning
{"action": "drop"} with an HTTP 2xx (see
JSON action body). Other failures split two ways:
- A transient failure — a
4xx/5xx, a timeout, a connection error, a receiver that is temporarily down — is held for RETRY and re-fired by the sweep until it recovers (bounded by the deferral window). - A config failure that waiting cannot fix — the URL is refused by the
SSRF guard (resolves to an internal address, or won't
resolve), or the channel is missing its secret / url / auth_method —
delivers the mail past the broken webhook (CONTINUE) and logs an
ERRORso an admin fixes or disables the channel. We never POST the internal target (the guard already blocked it); we just don't stall a whole scope's inbound — instance-wide for a GLOBAL blocking webhook — for 48h on a config that has never worked. Create/update validation rejects internal/unresolvable URLs up front, so at dispatch this is only a DNS rebinding or a hand-edited row.
Either way the user never loses mail. Note the config path is fail-open
for that one webhook: a hard spam/security gate is bypassed during the
failure (the rest of the pipeline, incl. rspamd, still runs) — a conscious
trade against stalling all inbound; the ERROR log is your signal to fix it.
| Outcome | Decision | What happens |
|---|---|---|
HTTP 2xx, empty / non-JSON body |
CONTINUE | Email delivered normally. |
HTTP 2xx + {"action": "drop"} |
DROP | The only path to DROP — the receiver deliberately discards the email. |
HTTP 2xx + other JSON action body |
see below | Body parsed for action / is_spam / add_labels; default is CONTINUE. |
Any non-2xx (4xx/5xx/3xx), timeout, conn. |
RETRY | Transient — held, re-fired by the sweep, bounded by the deferral window. |
| SSRF rejection (internal / unresolvable URL) | CONTINUE + alert | Config error retry can't fix; deliver past it, log.error. URL never dialed. |
| Missing secret / url / auth_method (misconfig) | CONTINUE + alert | Non-DRF misconfig retry can't fix; deliver past it, log.error. |
RETRY is bounded by a deferral window so a persistently-failing
processing step can neither pin a row forever nor lose mail. If a step
is still failing 48 hours after the message arrived, we stop holding
and deliver the message anyway — landed in the inbox (is_spam=False
so it isn't buried) and stamped with an X-StMsg-Processing-Failed
marker. The web UI reads that marker and shows a prominent warning banner
(the same surface as the unverified-sender warning), so the recipient
knows the message bypassed a processing step and can review it with
caution. Nothing is ever silently dropped; if the step recovers within
the window, the next sweep delivers normally with no marker. (That
X-StMsg-Processing-Failed marker rides in the stored MIME as an
X-StMsg-* header; sender-supplied X-StMsg-* headers are stripped at
ingest, so a malicious sender can't forge it to fake a bypassed check.)
The mechanism is generic: a blocking webhook is the trigger today, but
any step that returns RETRY (e.g. a persistently-unreachable spam
checker) is deferred the same way.
Delivery is at-least-once — make your receiver idempotent. A
RETRYre-fires the webhook on the next sweep, and a worker crash after we POST but before we record success can re-deliver the same message. TheMessageitself is created exactly once (deduplicated byMessage-ID), but your endpoint may legitimately see the same message more than once. Key on theMessage-ID(or theX-StMsg-*envelope headers) and treat repeats as no-ops.
JSON action body
When a blocking webhook returns HTTP 2xx with Content-Type: application/json, the body MAY contain the following keys. All are
optional; unknown keys are ignored.
{
"action": "drop",
"is_spam": true,
"add_labels": ["b3c9c1c3-1f4a-4d4a-9b2d-9c5a2a7c0a01"],
"assign_to": ["alice@example.org"],
"mark_starred": true,
"mark_read": true,
"mark_trashed": false,
"mark_archived": true,
"skip_autoreply": true,
"add_event": [
{"type": "im", "content": "AI summary: budget Q4 update"}
],
"reply_draft": {"template": "b3c9c1c3-1f4a-4d4a-9b2d-9c5a2a7c0a01"}
}
| Key | Type | Meaning |
|---|---|---|
action |
"drop" |
"drop" drops the message at this phase. Any other value (or omission) is treated as accept. Case-insensitive. There is no body-driven "retry": a 2xx is a successful response. If you need the message redelivered later, return a non-2xx status (e.g. 429/503) — it is held for retry, bounded by the 48h deferral window. |
is_spam |
bool | Override the spam verdict. Acts as a full antispam: for a message.inbound webhook this skips rspamd. |
add_labels |
string[] | UUIDs of Label rows in the destination mailbox to attach to the thread once it is created. |
assign_to |
string[] | OIDC emails of users to assign to the resulting thread (one ThreadEvent ASSIGN per webhook, channel-attributed). |
mark_starred |
bool (true only) | Star the resulting thread for the destination mailbox. |
mark_read |
bool (true only) | Mark the resulting thread as read for the destination mailbox. |
mark_trashed |
bool (true only) | Land the message with is_trashed=true. (Distinct from action: "drop" — the row stays, just hidden.) |
mark_archived |
bool (true only) | Land the message with is_archived=true. |
skip_autoreply |
bool (true only) | Suppress the standard autoreply for this message (in addition to the is_spam=true suppression). |
add_event |
object[] | Persist one ThreadEvent per entry, attributed to this webhook channel. See Events. |
reply_draft |
object | {template: "<MessageTemplate UUID>"} — materialise a draft reply for the user to refine + send. See Reply drafts. |
Notes:
action: "drop"always wins. Settingaction: "drop"together withadd_labelsorassign_tostill drops — the thread is never created, so neither side effect is applied.is_spamdiscriminates between explicit false (ham) and no opinion: returning{}leaves the dispatcher's verdict (typically rspamd) untouched, while returning{"is_spam": false}forces ham.add_labelsonly makes sense for mailbox-scoped channels: labels are per-mailbox. For domain- or global-scoped channels the UUIDs are validated against the receiving mailbox; unknown UUIDs are logged and skipped, not raised — a misbehaving webhook must not stall delivery.assign_toresolves each email to a User row withemail__iexact. The resolution is strict but quiet: emails that resolve to zero users, to multiple users (ambiguous —User.emailisn't unique, seeMAILBOX_ROLES_CAN_BE_ASSIGNED), or to a user whose mailbox role isn't one ofEDITOR/SENDER/ADMINare logged and skipped. No auto-create: a webhook receiver cannot mint a User row. Each blocking webhook that contributes assignees produces its ownThreadEventwithchannelset to that webhook's channel, so the audit timeline keeps per-receiver attribution. The resultingThreadEvent.authorisnull(the receiver is not a user); the existing partial UniqueConstraint onUserEvent(user, thread) WHERE type=assignmakes duplicate asks idempotent.- Bool flags (
mark_starred/mark_read/mark_trashed/mark_archived/skip_autoreply) usetrue-only semantics: a receiver opting in withtrueflips the flag;false, missing, or non-bool values are "no opinion". The multi-webhook merge is therefore a simple OR — a later receiver can't silently veto an earlier receiver's directive.mark_trashed/mark_archivedset the corresponding field on theMessagerow at creation time;mark_starred/mark_readsetstarred_at/read_aton the destinationThreadAccess(no- op when already set, so re-firing doesn't reset them).
add_event events
add_event is a list of structured events to persist on the resulting
thread. Each entry becomes one ThreadEvent, attributed to the
firing webhook via the channel FK; author is null.
Supported types:
type |
Required fields | Effect |
|---|---|---|
"im" |
content (string) |
Persists as an internal-message ThreadEvent — the same surface humans post into. |
The im content is stored on every inbound, so it is capped at
32 KiB (UTF-8) — longer content is truncated. At most 20
add_event entries are processed per response; extras are dropped.
Unknown types are silently skipped at the classifier — the contract
stays forward-compatible so receivers can begin emitting new types
(e.g. "iframe") before the server learns them, with no churn for
the receivers that already work.
reply_draft drafts
reply_draft: {"template": "<UUID>"} materialises a draft reply
to the incoming message, pre-filled from a MessageTemplate. The
draft is threaded under the inbound message, Re:-prefixed, and
addressed to the original sender — the user reviews and refines it
in the UI, then sends with a click. We do not auto-send.
Implementation reuses the autoreply pipeline (sender contact, subject
prefix, message + recipient creation, signature resolution); the only
difference is the body lands in draft_blob (the rich-text editor's
JSON shape, from the template's raw_body), not in blob. The
editor round-trip is therefore identical to a hand-composed draft.
Validation:
- The template must be
type=messageandis_active=true, scoped to the destination mailbox or its maildomain. Templates from other mailboxes / domains are silently skipped — a webhook receiver cannot draft from arbitrary templates. - Templates from outside the destination scope are silently skipped (logged, not raised).
- If the inbound message has no sender we can reply to, the draft is skipped (same rule the autoreply path uses).
Each blocking webhook that asks produces one draft attributed to
its own channel (Message.channel FK preserved for audit). If two
webhooks each ask, the user sees two drafts — they pick which one to
send, or delete both.
Multi-webhook merge
When several blocking webhooks fire on the same phase, their outcomes merge deterministically:
- decision: most severe wins (
DROP>RETRY>CONTINUE). The dispatcher short-circuits the fan-out as soon as any webhook drops. - is_spam: last decisive value wins (DB iteration order).
- add_labels: set union across all webhooks.
- assign_to: each webhook's list lands as its own ThreadEvent (channel attribution preserved). A user assigned by an earlier webhook is absorbed by the partial UniqueConstraint when a later webhook re-asks — no duplicate UserEvent, the first ask is the canonical attribution.
- mark_starred / mark_read / mark_trashed / mark_archived / skip_autoreply:
OR-merged — any
truewins. - add_event: each entry lands as its own ThreadEvent, in the order webhooks fired. No deduplication.
- reply_draft: each blocking webhook that asks produces one draft Message, attributed to its own channel. No deduplication — multiple receivers each asking yield multiple drafts.
Payload formats
The three formats are mutually exclusive — pick one per channel. The envelope headers above are identical across formats.
eml (default)
The request body is the raw RFC-822 message bytes, exactly as the MTA received them.
POST /inbox-hook HTTP/1.1
Content-Type: message/rfc822
X-StMsg-Trigger: message.delivered
X-StMsg-Instance: https://messages-public-url.example.com
X-StMsg-Channel-Id: 05f1f991-c2e9-4fa7-8a78-98c3aa904c7c
X-StMsg-Mailbox: alice@example.com
X-StMsg-Mailbox-Id: 3c2e0b1a-9d4f-4e8c-bf2a-1a2b3c4d5e6f
X-StMsg-Recipient: alice@example.com
X-StMsg-Is-Spam: false
From: Bob <bob@example.org>
To: alice@example.com
Subject: Hi
Message-ID: <abc123@example.org>
Content-Type: text/plain; charset=utf-8
Hello, Alice!
This is the simplest format. Any email library can parse it
(email.message_from_bytes in Python, JavaMail's MimeMessage,
mailparser in Node, etc.).
jmap
The request body is a strictly JMAP-compliant Email object per
RFC 8621 §4.1 serialised as JSON. The body is the object
itself — there is no surrounding envelope in the JSON; envelope
metadata lives in the headers above.
{
"messageId": ["abc123@example.org"],
"inReplyTo": [],
"references": [],
"from": [{"email": "bob@example.org", "name": "Bob"}],
"to": [{"email": "alice@example.com", "name": ""}],
"cc": null,
"bcc": null,
"sender": null,
"replyTo": null,
"subject": "Hi",
"sentAt": "2026-01-01T12:00:00Z",
"receivedAt": "2026-06-01T08:43:21Z",
"headers": [
{"name": "from", "value": "Bob <bob@example.org>"},
{"name": "to", "value": "alice@example.com"},
{"name": "subject", "value": "Hi"}
],
"bodyValues": {
"1": {"value": "Hello, Alice!", "isEncodingProblem": false, "isTruncated": false}
},
"textBody": [
{"partId": "1", "blobId": null, "size": 13, "name": null, "type": "text/plain", "charset": "utf-8", "disposition": null, "cid": null, "language": null, "location": null}
],
"htmlBody": [],
"attachments": [],
"hasAttachment": false,
"preview": null
}
Fields omitted on purpose
JMAP defines a few Email properties that only make sense once the
message is stored:
idandthreadIdare included only onmessage.delivered— it fires after theMessagerow exists, so we stamp them (the same values also ride in theX-StMsg-Message-Id/X-StMsg-Thread-Idheaders). The blocking triggers (message.inbound/message.delivering) fire before the row exists — there is no id yet, so both are absent.blobId,mailboxIds,keywordsare absent on every trigger:blobIdwould imply a JMAP blob-download endpoint we don't expose, andmailboxIds/keywordsneed a folder/flag mapping we haven't designed.
Attachment bytes are also intentionally omitted: JMAP keeps
attachment content behind a blobId and a separate fetch, which has no
analogue in a fire-and-forget webhook. The attachments[] entries
still describe each attachment's type, size, name, disposition
and cid. If you need the raw bytes pick format: "eml" instead.
Date formatting
sentAt and receivedAt are JMAP UTCDate strings: ISO-8601 in UTC
with an explicit Z suffix, e.g. 2026-01-01T12:00:00Z (not
+00:00). This matches RFC 8621 §1.4.
jmap_metadata
Same JMAP Email shape as jmap, but the body content and attachments
are dropped:
textBody,htmlBody,bodyValues,attachmentsare omitted.hasAttachmentis preserved as a single boolean so receivers can still tell whether the original message had attachments.- All envelope fields (
from,to,subject,messageId,headers,sentAt,receivedAt, …) are included.
Use this format when you only need the "a message arrived" signal plus addressing metadata — for example to forward to a chat channel — and don't want the body content to leave the instance over the wire.
Example receiver
A minimal Python receiver that accepts both formats:
import email
import json
from flask import Flask, request
app = Flask(__name__)
@app.post("/inbox-hook")
def inbox_hook():
content_type = request.headers.get("Content-Type", "")
if content_type.startswith("message/rfc822"):
msg = email.message_from_bytes(request.get_data())
print("EML subject:", msg["subject"])
elif content_type.startswith("application/json"):
body = request.get_json()
print("JMAP subject:", body["subject"])
# Body content may not be there in jmap_metadata mode.
body_values = body.get("bodyValues") or {}
for part_id, value in body_values.items():
print(f" part {part_id}: {value['value'][:80]}")
else:
return "unsupported", 415
# Echo envelope metadata for logging.
print("trigger:", request.headers["X-StMsg-Trigger"])
print("is_spam:", request.headers["X-StMsg-Is-Spam"])
print("mailbox:", request.headers["X-StMsg-Mailbox"])
return "", 200
Security notes
- The endpoint URL is caller-controlled (a mailbox admin sets it),
so every call goes through the shared
SSRFSafeSession:- Only
http://andhttps://URLs are accepted. - IP literals are rejected — a domain name is required.
- Hostnames resolving to loopback, link-local, private, multicast, reserved, or cloud-metadata addresses are rejected.
- The validated IP is pinned for the actual connection, defeating DNS-rebinding (TOCTOU). For HTTPS the TLS certificate is verified against the original hostname.
- Redirects are followed (up to a small hop limit) but each hop is
re-validated and re-pinned, so an endpoint can't 3xx-redirect the
delivery to an internal target. The
POSTis re-issued on each hop (method + body preserved), so a receiver behind a load balancer or URL canonicaliser still gets the signed payload.
- Only
- Blocking webhooks are silent for the original sender — the inbound
SMTP transaction has already been accepted. A blocking-drop is
visible only through logs and the pipeline's
dropped_by_webhookreturn value.
Performance notes & future work
- Per-webhook blob re-fetch (non-blocking). Each non-blocking
webhook for a message runs in its own task that independently
re-fetches
Message.blob(object-storage download + decrypt + decompress) and re-parses the MIME. With K non-blocking webhooks on the same message that's K downloads and K parses of identical bytes — wasteful for large messages. This is an accepted trade-off for now (each task stays self-contained and the payload never rides the broker). A future optimization is a short-lived blob/parse cache (keyed byblob_id, scoped to a single message's fan-out) so the bytes are fetched and parsed once and reused across that message's webhook tasks.