mirror of
https://github.com/suitenumerique/messages.git
synced 2026-09-28 12:45:00 +02:00
Sylvain ZimmerandGitHub
8e8d9e1261
Local parts are ASCII-folded — never str.lower(), which maps U+212A onto "k" and would resolve a look-alike to someone else's mailbox — for everything we own or resolve: mailbox lookup, login identity, inbound recipients. Domains are IDNA-canonicalized. Addresses we merely carry, in contacts and MIME headers, keep the case they were typed or received in. core.mda.addresses is the single policy, held in place by a pylint checker and an eslint rule that ban hand-rolled '@' splits and Unicode case folding. Migration 0035 folds existing rows and reports the collisions it will not merge on its own. Also drops mta-out, and adds dual RSA+ECDSA certificates, per-refusal logging and an mta_session correlation claim to mta-in.
4.3 KiB
4.3 KiB
Architecture
System Architecture Overview
Core Components
Frontend App
- React Application: React-based SPA with TypeScript, Vite, Tanstack Router and React Query
- Auto-generated API Client: Generated from OpenAPI schema using Orval
- Multi-panel Interface: Mailbox panel, thread list, and message view
- Real-time Updates: Using TanStack Query for efficient state management
Backend Services
- Django REST Framework: Main API service handling business logic, including email processing
- Celery Workers: Asynchronous task processing for heavy operations
- Search Service: OpenSearch integration for full-text search
Mail Transfer Layer
- MTA-In (Inbound): Postfix server with Python-based recipient validation
- Outbound delivery: Celery‑based SMTP delivery (direct to MX or via relay), with optional SOCKS proxying
- Mail Processing Agent: rspamd for spam filtering and mail processing
Data Storage
- PostgreSQL: Primary relational database for all structured data
- Redis: Caching layer and Celery message broker
- OpenSearch: Full-text search index for messages and threads
- S3-Compatible Storage: PST/MBOX imports and tiered blob offload — see tiered-storage.md
Authentication & Authorization
- Keycloak: OIDC provider for user authentication
- Role-based Access: Multi-tenant access control via mailbox and thread permissions
Data Flow
Inbound Email Processing
- External email arrives at MTA-In via SMTP
- MTA-In validates recipients against Django backend
- MDA parses and stores messages in PostgreSQL
- Celery tasks index content in OpenSearch
- Users see new messages in real-time via frontend
Outbound Email Processing
- User composes message in frontend
- Frontend sends draft via REST API
- Backend validates and queues message
- Celery delivers directly to each recipient's MX servers (
MTA_OUT_MODE=direct), or through an external SMTP relay ifMTA_OUT_MODE=relay - In development, MailCatcher intercepts messages and displays them in its web interface
Search Operations
- User submits search query via frontend
- Backend directly queries OpenSearch for real-time results
- Results are ranked and filtered by permissions
- Frontend displays paginated results
Search Indexing
- New messages/threads are saved to PostgreSQL
- Backend queues indexing tasks to Celery
- Celery workers asynchronously index content in OpenSearch
- Heavy operations (bulk imports, reindexing) are handled via Celery
Key Features
Multi-tenancy
- Domain-based: Mail domains with administrative roles
- Mailbox-based: Individual mailbox access permissions
- Thread-based: Granular access control for conversations
Scalability
- Microservices Architecture: Independent scaling of components
- Async Processing: Non-blocking operations via Celery
- Caching Strategy: Redis for session and query caching
- Search Optimization: OpenSearch for fast full-text search
Development Experience
- OpenAPI-First: Auto-generated client from backend schema
- Docker Compose: Complete development environment
- Hot Reloading: Frontend and backend development servers
- Testing Tools: Comprehensive test suites and monitoring
Security Considerations
Authentication
- OIDC integration with Keycloak
- JWT token validation
- Session management via Redis
Authorization
- Role-based access control (RBAC)
- Resource-level permissions
- Multi-tenant isolation
Email Security
- DKIM signing for outbound messages
- SPF and DMARC policy enforcement
- Anti-spam filtering via rspamd — see spam.md for the inbound spam & sender-authentication pipeline
Data Protection
- Encrypted storage for sensitive data
- Secure file upload handling
- CORS and CSRF protection
Deployment Architecture
The system is designed for containerized deployment with:
- Docker containers for all services
- Environment-specific configurations (dev, staging, production)
- Horizontal scaling capability for backend and Celery workers
- Load balancing support via Caddy reverse proxy
- Health checks and monitoring integration (In Progress)