This PR adds support for 2 MTA out modes : Direct-to-MX and SMTP-relay outbound delivery. Direct mode supports SOCKS5 proxies, and we bundle a new `src/socks-proxy` component to support it.
We also add an end-to-end self-check command plus scheduled health-check task with optional Prometheus metrics.
---------
Co-authored-by: Bastien Ogier <bastien.ogier@ext.anct.gouv.fr>
Co-authored-by: Stanislas Bruhiere <stanislas@bruhiere.fr>
The OpenAPI Schema does not embedded abilities property.
We fix that and make two serializers for User model in order to get proper
openapi definition for User instance with abilities and without abilities.
* ✨(dns) add admin page for DNS records. Fixes#177
* 👷(dockerflow) add a flag to disable Dockerflow
* 💚(ghactions) fix check-api-update that was missing Orval
* 💚(ci) fix tests that depended on Keycloak
* ♻️(admin) refactor CopyableInput into a separate component
* 🎨(dns) improve code style
We are going to release a 0.1 version soon, along with our first production deployment. Starting from there, migrations and a consistent developer experience will be officially supported. To make that easier, this large patch cleans up several areas:
* Reset migrations one last time
* Update models for storage efficiency (move textchoices to integerchoices on high-volume tables)
* Use Blobs for mail mime data and draft bodies. Having them in a separate PG table is a first step, we will later start offloading them to object storage.
* Add default ZSTD compression to blobs
* Add per-domain DKIM Keys
* Add DNS check and provisioning, with a first Scaleway provider
* Fix Keycloak user provisioning
* Fix Attachment storage, they are now stored individually only at the drafting stage. Afterwards they are extracted from the main blob. This may be optimized later but at least we only store once. For JMAP compatibility, this requires using fake IDs in the blob API route.
* Add a management command and recurring task to retry unsent messages
* Improve the local developer experience with new ports and make commands
* Repackage MTA-in and MTA-out to be closer to Backend: Poetry, multi-step Dockerfile, move compose and makefile to the root
* Migrate to OpenSearch
* Improve overall documentation and add a self-hosting page
Contributes to #177 and #185
Before, sending was done synchronously on the api/send POST call. Now
we just build the MIME message and mark the message for sending, and
queue the actual SMTP (or internal) sending for the worker. We use
this opportunity to add fine-grained delivery statuses for each
recipient, something that traditional email systems are unable to do!
Expanding on that, this (too large) PR adds an initial Elasticsearch
index of threads. It also overhauls the build and CI system, using
Poetry instead of Pip.
