This will catch the case where people try to send email before having
correctly configured their DNS, and would also avoid unwanted sends
of messages not supposed to go out.
Adds new config vars for incoming & outgoing attachment sizes, validated at draft stage
and when receiving emails.
---------
Co-authored-by: Riël Notermans <riel@mosa.cloud>
This PR adds a new build system for embeddable widgets and a first implementation of a "Feedback" popup widget.
It also refactors inbound message routes into channels, of which there are 2 for now: MTA (by default) and Widget. More to come!
This fixes delivery errors seen in the wild where:
* Some servers advertise STARTTLS but have broken certificates.
Because of this, we follow Postfix's recommentations to use
opportunistic TLS by default.
* As we are sending emails straight to other SMTPs in direct mode,
we need to have stricter RFC compliance for our raw mime DATA, namely
CRLF endings and maximum line length. This was transparently done
by Postfix in relay mode.
Some env vars have also been renamed for clarity.
This PR adds support for 2 MTA out modes : Direct-to-MX and SMTP-relay outbound delivery. Direct mode supports SOCKS5 proxies, and we bundle a new `src/socks-proxy` component to support it.
We also add an end-to-end self-check command plus scheduled health-check task with optional Prometheus metrics.
---------
Co-authored-by: Bastien Ogier <bastien.ogier@ext.anct.gouv.fr>
Co-authored-by: Stanislas Bruhiere <stanislas@bruhiere.fr>
* 🐛(scalingo) add an env var to disable Celery beat scheduling
Deployments on fresh PaaS databases would fail because of this, without
giving the user the ability to create the database through a manual
run command.
* 🐛(nginx) fix error when no env vars configured
* ♻️(services) refactor Services into a single directory
I haven't done AI yet because there are pending patches but once done we'll also have core/services/ai/*.py
I used importer as a name to avoid using a Python keyword
* 🚨(lint) move imports to the top
* ✨(dns) add admin page for DNS records. Fixes#177
* 👷(dockerflow) add a flag to disable Dockerflow
* 💚(ghactions) fix check-api-update that was missing Orval
* 💚(ci) fix tests that depended on Keycloak
* ♻️(admin) refactor CopyableInput into a separate component
* 🎨(dns) improve code style
This fixes the admin panel where users searched among the same
maildomain could correctly be added by their uuid, but when entering
an arbitrary email, the email and not the uuid gets sent in the POST
mailbox access. This might get replaced later when we have a proper
invitation system for external emails.
We are going to release a 0.1 version soon, along with our first production deployment. Starting from there, migrations and a consistent developer experience will be officially supported. To make that easier, this large patch cleans up several areas:
* Reset migrations one last time
* Update models for storage efficiency (move textchoices to integerchoices on high-volume tables)
* Use Blobs for mail mime data and draft bodies. Having them in a separate PG table is a first step, we will later start offloading them to object storage.
* Add default ZSTD compression to blobs
* Add per-domain DKIM Keys
* Add DNS check and provisioning, with a first Scaleway provider
* Fix Keycloak user provisioning
* Fix Attachment storage, they are now stored individually only at the drafting stage. Afterwards they are extracted from the main blob. This may be optimized later but at least we only store once. For JMAP compatibility, this requires using fake IDs in the blob API route.
* Add a management command and recurring task to retry unsent messages
* Improve the local developer experience with new ports and make commands
* Repackage MTA-in and MTA-out to be closer to Backend: Poetry, multi-step Dockerfile, move compose and makefile to the root
* Migrate to OpenSearch
* Improve overall documentation and add a self-hosting page
Contributes to #177 and #185
This replaces the two plugins we had (check recipients + delivery)
to a single Milter plugin. The real change is that now the delivery
happens during the SMTP session, and can result in a temporary
failure to the client. This makes the MTA-in entirely stateless.
Migrate from counters to simple flags, because the frontend needs
counts of threads, not sums of counts of messages. Also prepare
for spam and archive features.
This allows user creation, passwords reset and listing users from
Keycloak, plus a new MailDomainAccess model for admins.
* ✨(admin) add maildomainacess model, api route and backend tests
* ♻️(drf) simplify API code
* ✨(keycloack) add deployable keycloak to PaaS
* ✨(keycloak) add integration via API, upgrade to 26.x
* 🗑️(settings) remove invitation setting
* 🐛(tests) fix failing tests after rebase
* 🚨(all) fix lint
Before, sending was done synchronously on the api/send POST call. Now
we just build the MIME message and mark the message for sending, and
queue the actual SMTP (or internal) sending for the worker. We use
this opportunity to add fine-grained delivery statuses for each
recipient, something that traditional email systems are unable to do!
Expanding on that, this (too large) PR adds an initial Elasticsearch
index of threads. It also overhauls the build and CI system, using
Poetry instead of Pip.
