Commit Graph
778 Commits
Author SHA1 Message Date
Sylvain Zimmer 667d4b798d _ext instead 2026-06-09 14:25:23 +02:00
Sylvain Zimmer 48f65b9837 finetune api 2026-06-09 14:17:15 +02:00
Sylvain Zimmer 115986468c adhere closer to RFC 2026-06-09 14:17:14 +02:00
Sylvain Zimmer 4d1038d7b6 finetune public api 2026-06-09 14:17:14 +02:00
Sylvain Zimmer 10304fb60b more review fixes 2026-06-09 14:17:14 +02:00
Sylvain Zimmer 336da6f8fc review fixes 2026-06-09 14:17:14 +02:00
Sylvain Zimmer 452cea2bfe continue cleanup 2026-06-09 14:17:14 +02:00
Sylvain Zimmer 1f17de1bb7 adjust perimeter of lib 2026-06-09 14:17:14 +02:00
Sylvain Zimmer fc986b0ac0 review fixes 2026-06-09 14:17:14 +02:00
Sylvain Zimmer 7b2c5e8637 review fixes 2026-06-09 14:17:14 +02:00
Sylvain Zimmer 9a25ac988d review fixes 2026-06-09 14:17:14 +02:00
Sylvain Zimmer 27db845d43 ♻️(rfc5322) move email parsing & composing code to new jmap-email lib 2026-06-09 14:17:14 +02:00
Sylvain Zimmer d85a4dd825 🔒️(smtp) harden SMTP connection & proxies config 2026-06-09 14:17:14 +02:00
638a2de121 ♻️(frontend) drop next.js for vite + tanstack router (#675)
Next.js has been concerned by several CVEs last time and it's clearly
overkill for our needs (static export) so in order to enlight our stack
we migrate to vite & tanstack-router

Co-authored-by: jbpenrath <jb.penrath@gmail.com>
2026-06-09 11:58:51 +02:00
Sylvain Zimmerandjbpenrath ad6521c272 🔒️(smtp) harden SMTP connection & proxies config 2026-06-08 16:34:15 +02:00
Sylvain Zimmerandjbpenrath 906f55894b 🐛(rsvp) fix order and default calendar selection when RSVPing (#699) 2026-06-08 16:34:15 +02:00
jbpenrath c77fd0d3c7 🐛(backend) manage message/delivery-status attachments at compose
DSN/bounce and read-receipt reports embed a message/delivery-status part.
A PST stores it as a flat byte blob, and reconstructing the .eml fed it to
compose_email as a message/delivery-status attachment. There, email.generator
dispatches to _handle_message_delivery_status, which assumes a structured
(list) payload: given our flat base64 string it iterated character by
character and raised "'str' object has no attribute 'policy'", aborting the
whole compose. Every PST message carrying such a report was silently dropped,
and any send/widget/autoreply with the same attachment type would fail too.

Relabel message/delivery-status to text/plain at the single choke point
(create_attachment_part) so all compose callers are covered without
duplicating the guard. The bytes are RFC822-style text, so they stay readable
and intact; no other attachment type reaches a payload-structured generator
branch, so normal mail is unaffected.

Also skip empty / whitespace-only PST attachments: DSN reports expose blank
diagnostic parts that libpff surfaces as attachments, which imported as
0-byte parts rendering as broken in the UI while carrying no information.
2026-06-04 16:22:45 +02:00
jbpenrath bd5eb113a0 🐛(backend) complete pst email folder prefixes list
Previously, we considered as email container only ones
prefixed by `IPF.Note` but it appears `IPF.Imap` can also
contains email so we can miss some mails during import.
2026-06-04 16:22:44 +02:00
Sylvain ZimmerandGitHub e3c183a625 🐛(smtp) fix opportunistic TLS against MXes with mismatched certs (#687)
The "may" level was verifying the peer cert and falling back to
cleartext on mismatch, which then bounced on STARTTLS-required
servers (e.g. Mandrill's SES-backed inbound returns 530 to MAIL
FROM in cleartext). Realigned on Postfix's documented behavior:

- "may":    opportunistic TLS, no cert verification.
- "secure": mandatory TLS + CA chain + hostname check; defers if
            STARTTLS isn't advertised or handshake fails.
- "encrypt" is dropped (replaced by "secure").

Also wires MTA_OUT_SMTP_TLS_SECURITY_LEVEL through both the direct
and relay paths — it had been declared but never read — and
collapses the four proxy_* kwargs + sender_hostname of
send_smtp_mail into a single SmtpProxy dataclass.
2026-06-04 16:20:57 +02:00
Sylvain ZimmerandGitHub 89dac5a86f 🐛(import) fix mbox detection as text/html with some libmagic versions (#696) 2026-06-04 16:20:04 +02:00
Sylvain ZimmerandGitHub d2a10256ef 🐛(backend) harden inbound email parsing (#695)
Two distinct AttributeError crashes were firing on legitimate inbound
  mail and producing 5xx-equivalent failures (parse aborts, autoreply
  skipped).

  1. Flanker scanner crashed on `multipart/report` bounces whose status
     part used an RFC 6533 i18n content type (`message/global-delivery-
     status` and siblings). The same crash path was reachable for every
     `message/*` subtype not in flanker's hardcoded list — `partial`,
     `imdn+xml`, `sip`, `cpim`, future / vendor subtypes. Fixed in the
     flanker fork (i18n predicates + traverse fallback + make_part guard);
     pin bumped to 77f4582044f1a8549d49333d50d9bded1176ca09.

  2. `parsed["headers"]` returned `str` for single occurrences and
     `list[str]` when duplicated. Every scalar consumer (`.strip()`,
     `.lower()`, `.startswith()` on Subject / Message-ID / Precedence /
     Return-Path / etc.) crashed the moment a header was repeated. The
     parser now applies a fixed per-header type contract driven by the
     IANA Provisional Message Header Field Registry: headers registered
     with max=1 (RFC 5322 §3.6, RFC 3834, RFC 2045/2046/2183, RFC 4021,
     RFC 3798, RFC 5703, RFC 8058, RFC 8098) are `str` with first-wins
     semantics matching stdlib `email.message.Message[name]`; every
     other header is `list[str]` in document order. `headers_blocks`
     stays uniformly list-typed for the trusted-relays cut.
2026-06-04 15:43:29 +02:00
Sylvain ZimmerandGitHub f7786b3d0f (backend) report selfcheck status to Sentry crons (#694)
Use the new (documented) env var MESSAGES_SELFCHECK_SENTRY_MONITOR_SLUG
to enable
2026-06-04 00:38:59 +02:00
Sylvain ZimmerandGitHub b68f0c4d37 🐛(mta-in) fix milter socket permission race on startup (#693) 2026-06-03 23:38:11 +02:00
Jean-Baptiste PENRATHandGitHub e58a6f323e ⬆️(backend) bump django-lasuite to 0.0.26 (#689)
Upgrade to this version to be able to use `login_hint` param
during oidc auth.
2026-06-02 15:50:07 +02:00
Sylvain ZimmerandGitHub d91e97d8c2 🐛(cal) fix display of recurring events with exceptions (#686) 2026-05-29 16:03:51 +02:00
jbpenrath 3f200ed04d 🔖(minor) release version 0.7.0
Update all version files and changelog for minor release.
v0.7.0
2026-05-28 18:56:07 +02:00
jbpenrath ea9f33f6ff 🐛(frontend) do not refetch thread messages on draft deletion
If a draft is the single thread message, delete it should not
trigger a request to refresh thread message list because this is
wasteful and it also display a toast error as the thread does not
exist anymore.
2026-05-28 18:00:25 +02:00
Sylvain ZimmerandGitHub 7528d3543b 🐛(caldav) use the OIDC email instead of the mailbox email (#679)
This allows the user to see the exact list of calendars seen on the
other Calendars app
2026-05-28 17:56:25 +02:00
jbpenrath afdfaf938a 🚸(frontend) improve sending experience
First when the sending reach the timeout, instead to display a toast
error
with a message that lets believe the message cannot be sent, we display
a warning message mentionning that sending takes more time than
expected.

Then, once a message is sent, we optimistically update thread cache to
hide
immediately the sent draft and show instead the message in the thread
view.
2026-05-28 11:02:15 +02:00
jbpenrath 77b16e5dc6 🐛(frontend) unmount thread view immediately on unselect thread
On slow machines, a race condition can occur when unselectThread is
trigger and another action is also triggered. e.g: when user mark as
unread a thread, the thread is unselect and the request to mark it as
unread is trigger but sometimes, the request is completed before the
router navigation so the observer in charge to mark visible message has
unread is trigger again and finally, the thread is not marked as
unread...
2026-05-27 18:42:02 +02:00
jbpenrath 4907849faf (frontend) attachments preview
Use the Preview component to allow user to
preview message attachment from a thread.
2026-05-27 17:05:16 +02:00
Jean-Baptiste PENRATHandGitHub 567a73a7e4 🗃️(backend) remove _deprecated model fields (#678)
In f360570798, a
migration in 3 parts has been done. This was
a progressive migration and in order to be able
to restore data model in case of failure, we simply
deprecate some fields. Now that everything is fine in
production, we can safely remove those fields.
2026-05-27 10:06:43 +02:00
Jean-Baptiste PENRATHandGitHub 98eabf96dd (backend) add defusedxml as deps (#677)
defusedxml was not declared into dependencies. In local
environment this one should be installed as transitive
deps but during deployment this deps was not found.
2026-05-27 09:39:13 +02:00
Jean-Baptiste PENRATHandGitHub 550af75214 🔖(minor) release version 0.6.0 (#671)
Update all version files and changelog for minor release.
2026-05-26 16:21:59 +02:00
Sylvain ZimmerandGitHub bd779a15d5 (calendar) add link to a CalDAV instance to accept events directly (#584)
CalDAV-backed mailbox calendar actions: RSVP (Accept/Decline/Maybe), Add to calendar, calendar chooser, calendar listing, and conflict detection — actions enqueue background tasks and return task IDs; new API endpoints documented.
2026-05-26 10:52:00 +02:00
Sylvain Zimmer e6bf029488 🐛(makefile) make sure we create env files before any call to compose
Fixes #674
2026-05-21 23:28:01 +02:00
jbpenrath 1f7427044e 🐛(backend) preserve obs-id-left In-Reply-To via UnstructuredHeader
Follow-up to 6144ccb2: the graceful drop stopped the 500 on /send/,
but every reply to an Outlook/MAPI thread lost threading because
Python 3.14's MsgIDListHeader (now the default for In-Reply-To and
References) truncates obs-id-left ids with multiple '@' in the local
part at the first '@' on serialize. The pre-stdlib flanker composer
used to write those bytes through unchanged.

Route both headers to UnstructuredHeader through a dedicated
HeaderRegistry. The instance has to be dedicated: policy.clone()
shares header_factory by reference with policy.SMTP and policy.default,
so mutating in place would silently change parsing process-wide. The
msg-id regex is loosened to allow multiple '@' now that the value
goes out verbatim; the whitespace ban stays (UnstructuredHeader folds
mid-id and receivers would then truncate at the fold), as does the
graceful drop from 6144ccb2 for the cases that genuinely can't ship
(whitespace, no '@', nested brackets, CR/LF injection attempts).

The PST importer's mirror regex is loosened in sync so archive imports
preserve threading on the same kind of ids. Two parametrized tests
lock the regression surface against silent narrowing — eight real-world
shapes that must round-trip on the wire and eight that must drop.
2026-05-21 14:19:18 +02:00
Sylvain ZimmerandGitHub cdd8be1641 🐛(perf) fix performance issue with large recipient counts (#672) v0.6.0 2026-05-21 01:53:40 +02:00
Sylvain Zimmer b3d3f36e00 ️(admin) avoid N+1 queries in admin & slow lookups 2026-05-21 00:22:31 +02:00
Jean-Baptiste PENRATH 7690d08ef0 👔(backend) allow to delete internal message at any time (#669)
Currently, user can delete/edit an internal message while
it is within the edit timeframe defined by
`MAX_THREAD_EVENT_EDIT_DELAY`. First feedbacks raises that
this limit is not relevant for deletion.
2026-05-20 20:49:50 +02:00
jbpenrath fedc4969bb 🐛(backend) improve pst import logic
- Improve error management for pst

We get some pst that are unparsable by pypff. To help user to understand
that the issue is coming from the PST file, we improve the exception
raised by pst task and display a custom error message according to the
error format.

- Recover Exchange X.500-only senders during PST import

Sent items from shared mailboxes — and many internal Exchange messages —
expose every PR_SENDER_*/PR_SENT_REPRESENTING_* slot as an unresolvable
X.500 DN. compose_email then rejected the EML for lack of a valid From
address and pst_tasks silently dropped the message at debug level, so
entire folders disappeared from the import without a trace.

- Prevent duplicate messages on PST re-import

PST messages without transport_headers (drafts, locally composed items)
were reconstructed with no Message-ID at all, and Exchange/O365 exports
sometimes drop the header even on received items. With no mime_id to
key on, deliver_inbound_message skipped its dedup check and inserted
the same message on every import — and even twice within a single
import when the same message appeared in multiple Outlook folders.
2026-05-20 20:49:01 +02:00
Jean-Baptiste PENRATH 49fcf22438 (frontend) thread deep linking (#664)
Allow user to copy/paste a thread link with other mailbox users.
Currently, if the user copy the current url, the link is broken once
the thread has been moved from the origin folder. It is also possible
to target a message or internal message.
2026-05-20 20:48:00 +02:00
Sylvain Zimmerandjbpenrath 3576968e38 (admin) add mandatory TOTP field + search field (#667)
To enable syncing a role to many users, we had to add a custom
Keycloak plugin.
2026-05-20 20:48:00 +02:00
Jean-Baptiste PENRATH ebab36d577 (frontend) improve message composer
* allow to render table in email

Improve email exporter to support table elements. We do not add
blocknote tool to add explicitly table but we allow user to
copy/paste it and render it properly.

* upgrade to blocknote 0.49.0

Remove a bug that prevent to use backspace in an empty block.
https://github.com/TypeCellOS/BlockNote/pull/2610
2026-05-20 20:48:00 +02:00
Sylvain Zimmerandjbpenrath f360570798 (blobs) implement tiered storage and refactor blobs/attachments
This allows to use S3-compatible object storage to offload blobs, making Postgres much lighter. We design for storing ~1B emails on a single instance.

We also take this opportunity to do model changes on blobs & attachments.

Migration 0027 is one-way, no going back after this one so check your backups.
2026-05-20 20:48:00 +02:00
Sylvain Zimmerandjbpenrath 6eab1af6d6 (rfc5322) switch back to Python's stdlib for email composition
Latest fixes in the stdlib make it a more solid alternative for strict composition than Flanker. We keep Flanker for now for lenient inbound parsing. We add stronger tests and fuzzing to validate we didn't regress.
2026-05-20 18:57:36 +02:00
Jean-Baptiste PENRATH 21255a4c4a (global) allow thread assignation (#645)
As a follow-up of the mention feature, we build upon ThreadEvent & UserEvent
models a feature to assign users to a thread.
We allow to filter mailboxe's inbox through assignation state (assigned to me, unassigned).
The thread share modal has been forked from ui-kit to be able to list users of each
mailbox and add a cta to assign them to the thread. A section above shows assigned users.
2026-05-19 14:44:26 +02:00
Jean-Baptiste PENRATHandGitHub 8e009c5972 🐛(backend) allow thread editor to destroy thread accesses (#668)
Currently, only mailbox editors can remove thread access to their
mailbox.
Actually, thread access deletion must be symmetric with creation rights.
So
any user with thread management ability should be able to delete a
thread
access.
2026-05-19 14:44:06 +02:00
jbpenrath 1b03e1d240 🌐(frontend) localized attachment separator
The `ATTACHMENT_SEPARATORS` only contains a english separator and as
this
string is displayed within other mail client, for non-english users it
can
be weird to see an english string. That's why we add new separators and
according to the sender language, we use the localized separator that
correspond to him.
2026-05-12 13:48:38 +02:00
jbpenrath 8a7cb8e0c2 (frontend) make panel sections resizable
Allow user to resize mailbox folder and mailbox label list sections.
2026-05-11 16:39:30 +02:00