lasuite-messages/docs/spam.md

Spam & sender-authentication processing

This document describes how Messages classifies inbound mail as spam or forged, how that classification is configured (globally or per mail domain), how it can be extended (native rspamd, header rules, custom webhooks, or an upstream MX gateway), and how the results are surfaced to users.

Overview

Every externally-received message runs through an inbound pipeline before it is delivered. The pipeline is assembled per recipient mailbox and, for external mail, runs these steps in order:

1. Before-spam webhooks (message.inbound) — user/integration webhooks that may drop, defer, or pre-decide the spam verdict.
2. Hardcoded header rules — deterministic header_match rules from config.
3. rspamd — native /checkv2 scan.
4. Inbound authentication — DKIM/DMARC verdict (SPF indirectly).
5. After-spam webhooks (message.delivering, then message.delivered).

Each step returns a Decision (CONTINUE / RETRY / DROP) and may set the spam verdict. The verdict is a tri-state ctx.is_spam: None (undecided) until a step decides it; the last decisive step wins, and the hardcoded-rules and rspamd steps are skipped entirely once a verdict already exists. This lets a before-spam webhook, an internal-origin short-circuit, or a header rule pre-empt rspamd.

Two distinct outcomes are produced:

• is_spam (boolean) — routes the message to the Junk view and suppresses auto-reply and push notifications. Set only by high-confidence signals (rspamd quarantine/reject, a matching spam rule, or a webhook override).
• Graded UI markers (postmark["spam"] = "possible" / "likely") — the message still lands in the inbox but shows a "may be / likely spam" banner.

Internal mailbox-to-mailbox mail and selfcheck probes are trusted: the task pre-sets is_spam = False and the pipeline omits the spam and auth steps (webhooks still fire, so consumers can't tell the difference).

Implementation: core/mda/inbound_pipeline.py (pipeline), core/mda/spam.py (rspamd + rules), core/mda/inbound_auth.py (DKIM/DMARC), core/mda/inbound_tasks.py (task wrapper). Webhooks are documented in webhooks.md.

rspamd native integration

Messages talks to rspamd over HTTP. The reference deployment ships an MPA container (rspamd engine + nginx proxy, see src/mpa/) that exposes the /checkv2 endpoint.

On each external message, call_rspamd() POSTs the raw RFC-822 bytes to {rspamd_url}/checkv2 (Content-Type: message/rfc822, 10 s timeout), optionally with an Authorization header, and forwards the SMTP envelope as rspamd scan headers (From→mail_from, Rcpt→rcpt_to, IP→ip, Helo→helo, Hostname→hostname) so rspamd can evaluate SPF, reputation, etc.

Messages does not interpret the rspamd score. It reads back rspamd's action and maps it to a delivery outcome. This is the single source of truth for that mapping:

rspamd action	Outcome
no action	Deliver to inbox, not spam
greylist, soft reject	RETRY — held and re-tried (temporary defer, see below)
add header	Deliver to inbox, UI marker spam = "possible"
rewrite subject	Deliver to inbox, UI marker spam = "likely"
quarantine, reject	is_spam = True → Junk (reject can't be honored at SMTP time, so it lands in Junk)
discard	DROP — accepted and silently blackholed, no bounce
unknown / unmapped	Deliver to inbox

If rspamd is not configured (rspamd_url absent), the step is a no-op and the verdict is left to other steps. If rspamd is configured but errors or is unreachable, the message is held for RETRY (fail-closed, never fail-open) rather than delivered unchecked — see deferral below.

DKIM/DMARC symbols from the same /checkv2 response are reused by the inbound authentication step (below) when it runs in rspamd mode, so a message is only scanned once.

Configuration: global and per-domain

Spam behavior is driven by a single SPAM_CONFIG dictionary.

• Global — the SPAM_CONFIG Django setting (env var SPAM_CONFIG, a JSON/dict value, default {}).

• Per mail domain — a MailDomain may override any subset of keys via MailDomain.custom_settings["SPAM_CONFIG"]. Resolution is a shallow key-by-key merge over the global config:

  spam_config = settings.SPAM_CONFIG.copy()
  if maildomain.custom_settings and "SPAM_CONFIG" in maildomain.custom_settings:
      spam_config.update(maildomain.custom_settings["SPAM_CONFIG"])
  

  (MailDomain.get_spam_config(), resolved at delivery time from the recipient mailbox's domain.)

There is no per-mailbox spam configuration — the effective scope is global → mail domain. (A future enhancement could add a per-mailbox layer.)

SPAM_CONFIG keys

Key	Type	Meaning
rspamd_url	string	Base URL of the rspamd HTTP endpoint (/checkv2 is appended). Omit to disable rspamd.
rspamd_auth	string	Optional value for the Authorization header sent to rspamd.
inbound_auth	string	Sender-auth backend: native, rspamd, or authentication-results. Omit/empty to disable DKIM/DMARC checks.
trusted_relays	int	Number of sender-side Received/Authentication-Results blocks to trust, counting from the boundary our own MTA prepends. Default 0 (trust only our own hop). Raise this when a fixed upstream gateway sits in front.
rules	list	Ordered hardcoded header-match rules (see below).

Related settings

Setting	Default	Meaning
SPAM_CONFIG	{}	Global spam config dict (above).
MESSAGES_INBOUND_DEFERRAL_MAX_AGE	172800 (48h)	Max time a message may be held on RETRY before it is force-delivered flagged (see deferral).

Note: there are no dedicated RSPAMD_* env vars — the rspamd URL and auth live inside SPAM_CONFIG (rspamd_url, rspamd_auth).

Sender authentication (DKIM / DMARC)

The inbound-auth step produces a sender-auth verdict independent of the spam verdict, stored in postmark["auth"] and surfaced in the UI (below):

• fail → the message is a likely forgery (DMARC fail).
• none → the sender's identity could not be verified (DKIM not passing, no enforceable DMARC).
• verified → nothing recorded (no banner).

The backend is selected by SPAM_CONFIG["inbound_auth"]:

• native — verify the DKIM signature locally (crypto + DNS) and require strict alignment between the signing d= domain and the From: domain. Never returns fail (no DMARC policy lookup); worst case is none.
• rspamd — read DKIM/DMARC symbols from the rspamd /checkv2 result (reusing the spam-step scan). Verdict precedence: fail > pass > none.
• authentication-results — parse dkim=/dmarc= from the Authentication-Results header(s) added by trusted upstream relays (bounded by trusted_relays). Use this when an upstream MX gateway already does authentication.

rspamd symbol → outcome (mode rspamd)

Check	Symbols → pass	Symbols → fail	Symbols → none
DKIM	R_DKIM_ALLOW	R_DKIM_REJECT, R_DKIM_PERMFAIL, R_DKIM_TEMPFAIL, DKIM_INVALID	R_DKIM_NA, DKIM_NA
DMARC	DMARC_POLICY_ALLOW	DMARC_POLICY_REJECT, DMARC_POLICY_QUARANTINE, DMARC_BAD_POLICY	DMARC_NA

Final rule: a DMARC fail yields forged (auth = "fail"); otherwise if DKIM is not pass, unverified (auth = "none"); otherwise verified.

SPF is not surfaced as a standalone verdict. It only influences classification indirectly through rspamd scoring (the envelope is forwarded to rspamd). The user-facing auth verdict is DKIM + DMARC only.

Hardcoded header rules

SPAM_CONFIG["rules"] is an ordered list of deterministic header-match rules, evaluated before rspamd. The first matching rule decides the verdict. Each rule:

Field	Meaning
header_match	Literal Header-Name: value (case-insensitive). Must contain a colon.
header_match_regex	Regex alternative, full-match, case-insensitive.
action	spam / reject → mark spam; ham / no action → mark not-spam. Default spam.

Rules honor trusted_relays: only headers within the trusted window (the most recent trusted_relays + 1 header blocks, newest first) are considered, so a spammer can't forge a header that an upstream you trust would have stripped or overwritten. The Return-Path header is always ignored (spoofable envelope value).

This is the primary mechanism for honoring the verdict of an upstream filter (next section).

Upstream / edge MX filtering ("en amont")

Many deployments put a dedicated anti-spam gateway in front of Messages at the MX edge — it scans mail before it ever reaches a mailbox and typically stamps its verdict into a header (e.g. X-Spam-Flag: YES) and/or adds its own Authentication-Results. Messages accommodates this without any native scanning of its own:

1. Trust the gateway's position. Set trusted_relays to the number of hops the gateway adds, so its headers fall inside the trusted window and forged copies from further upstream are ignored.

2. Honor its spam verdict with a hardcoded rule, e.g.:

   {
     "SPAM_CONFIG": {
       "trusted_relays": 1,
       "rules": [
         { "header_match_regex": "X-Spam-Flag:\\s*YES", "action": "spam" }
       ]
     }
   }
   

   (Set globally or per mail domain, e.g. only for domains whose MX points at the gateway.)

3. Honor its authentication results by setting inbound_auth: "authentication-results" so the gateway's DKIM/DMARC checks drive the sender-auth verdict instead of re-checking locally.

You can run an upstream gateway and rspamd together (defense in depth): header rules run first and short-circuit rspamd when they match, so the gateway's decision takes precedence and rspamd only scores what the gateway passed through.

Custom spam processor via webhooks

Webhooks can fully replace or override the built-in spam decision — see webhooks.md for the general webhook contract; this section covers the spam-specific behavior.

A blocking webhook (message.inbound or message.delivering) may return a 2xx JSON body containing:

{ "is_spam": true }

which sets ctx.is_spam to that boolean. Only a real JSON boolean is honored; any other value means "no opinion".

Phase determines the semantics:

• message.inbound (before-spam) runs before the header-rules and rspamd steps. Because those steps skip once a verdict exists, a before-spam webhook that sets is_spam pre-empts rspamd entirely — i.e. it becomes your spam processor.
• message.delivering (after-spam) runs after rspamd, so it overrides the verdict rspamd produced.
• message.delivered (after-spam, fire-and-forget) is non-blocking; its response is ignored and it cannot influence the verdict. It receives the final is_spam value for logging/sync.

The webhook payload carries the pending verdict in the X-StMsg-Is-Spam header (pending while undecided during the before-spam phase, else true/false). The response body may also drive other actions (action: "drop", labels, assignment, skip_autoreply, etc.) — see webhooks.md.

Webhook channels are scoped global / maildomain / mailbox and must have Channel.is_active = True to fire. Blocking-webhook results (including the is_spam override) are cached for the deferral window and replayed on retry, so a rspamd outage that forces a RETRY doesn't re-invoke an already-successful before-spam webhook.

What happens to a spam message

Once is_spam = True:

• Foldering — spam is a boolean flag on the message/thread, not a separate folder. The thread list excludes spam by default; the Junk view is the same list with is_spam=1. The thread's is_spam follows its first message, and spam messages are excluded from "active" counters/timestamps. is_spam is indexed for search and filtering. It is independent of is_trashed.
• Auto-reply suppressed — should_send_autoreply() returns early for spam (never vacation-reply to spam).
• Push notifications suppressed — no push is enqueued for a spam message.

Deferral & "processing failed"

RETRY outcomes (rspamd greylist/soft reject, rspamd/webhook errors, or a non-2xx blocking webhook) hold the message and re-try it on the inbound queue (roughly every 5 minutes). If a message is still failing after MESSAGES_INBOUND_DEFERRAL_MAX_AGE (default 48 h), it is force-delivered with is_spam = False, auto-reply skipped, and stamped postmark["processing"] = "fail" — which surfaces the "delivered without our usual safety checks" banner (below). This guarantees mail is never lost to a persistently failing rspamd/webhook dependency, at the cost of one un-checked delivery.

User-facing warnings in the reading UI

The reading view shows up to four banners, derived from the message's stmsg_headers (a serializer projection of the internal Message.postmark JSONField; legacy X-StMsg-* MIME headers are also merged for old messages).

Naming note: "postmark" here is an internal per-delivery pipeline record (Message.postmark), not the Postmark email SaaS.

Banner	Severity	stmsg_headers trigger	Text
Forged sender	error	sender-auth == "fail"	"This message failed sender authentication and is likely a forgery. Do not trust it."
Unverified sender	warning	sender-auth == "none"	"This contact's identity could not be verified. Proceed with caution."
Processing failed	error	processing-failed truthy	"This message was delivered without our usual safety checks. Please review it with caution."
Suspected spam	warning	spam == "likely" / "possible"	"This message is likely spam…" / "This message may be spam. Review it with caution."

The sender chip also shows an icon + tooltip: a warning icon for unverified and a gpp_bad icon for forged senders.

How each warning maps to the pipeline

Banner	Source signal	Produced by
Forged / Unverified sender	postmark["auth"] = fail / none	Inbound-auth step (DKIM/DMARC) — for mode rspamd, from the symbol table above
Suspected spam	postmark["spam"] = possible / likely	rspamd action add header → possible, rewrite subject → likely
Processing failed	postmark["processing"] = fail	Deferral-window expiry force-delivery (any persistently failing step)

Note the two-tier design: rspamd's mid-confidence actions (add header, rewrite subject) produce an inbox banner, while its high-confidence actions (quarantine, reject) set is_spam and route to Junk with no banner.

User "report as spam" action

Users with edit rights can flag a message/thread as spam (or un-flag it) via the flag API (spam flag). This sets the is_spam boolean (cascading to draft children) and recomputes thread stats, moving the thread in/out of the Junk view.

Gap: this action is purely local foldering — it does not send any feedback to rspamd (no learnspam/learnham/fuzzy training call exists in the backend). Wiring the report-as-spam / not-spam actions into rspamd's learn endpoints is tracked as a future enhancement (issue #509).

Not yet implemented / future work

• rspamd learn feedback from the report-as-spam button (#509).
• Per-mailbox spam configuration (currently global → maildomain only).
• SPF as a standalone user-visible verdict (today it only feeds rspamd scoring; the surfaced auth verdict is DKIM + DMARC).

Implementation map

Area	File
Pipeline assembly, rspamd action mapping, deferral constants	core/mda/inbound_pipeline.py
rspamd /checkv2 client, hardcoded header rules	core/mda/spam.py
DKIM/DMARC verdict, rspamd symbol table, AR parsing	core/mda/inbound_auth.py
Task wrapper, internal/selfcheck short-circuit, force-delivery	core/mda/inbound_tasks.py
Webhook dispatch, phases, is_spam override, result cache	core/mda/dispatch_webhooks.py
SPAM_CONFIG resolution (get_spam_config)	core/models.py (MailDomain)
is_spam flag, postmark field, stmsg_headers projection	core/models.py (Message/Thread)
Spam / Junk view filtering	core/api/viewsets/thread.py
Report-as-spam flag action	core/api/viewsets/flag.py
UI warning banners	src/frontend/.../thread-message/thread-message-header.tsx
rspamd engine + proxy container	src/mpa/