Backend part of #1796.
This changes allows to display an onboarding modal the first time that
the get_me() API view is called.
I originally tried to check if `User.last_login` was `None`, but it is
updated as soon as the user is logged, so I chose to create a flag on
the model.
Added:
- ✨(helm) allow all keys in configMap as env var
Changed:
- 📝(docs) improve README and add documentation hub
- ♿️(frontend) restore focus to triggers after closing menus and modals
- 🚸(frontend) change position elements toolbar
- ♿️(frontend) add focus on open to modals
Fixed:
- 🐛(frontend) analytic feature flags problem
- 🐛(frontend) fix home collapsing panel
- 🐛(frontend) fix disabled color on icon Dropdown
- 🐛(frontend) fix zIndex table of content
- 🐛(frontend) fix bug when language not supported by BN
- 🐛 (backend) prevent privileged users from requesting access
We upgrade pydantic-ai-slim to version 1.63.0, we have an error when
Docs is run in preprod environment.
We have to investigate why and we decided to downgrade to a working
version
In version 5 of django-treebeard they change the usage of NumConv for
something easier. They also introduce a breaking change, we have to
change the usage of this class in the migration 15.
Privileged users of a document (owners and admins) should not be
allowed to create an access request on a document they already control.
Without aguardrail, they could inadvertently inflate the access
request queue with redundant entries.
More critically, if an owner submits an access request on their own
document, another admin could accept it and grant them a lower role
(e.g. reader), which would silently strip them of their ownership.
Signed-off-by: Mohamed El Amine BOUKERFA <boukerfa.ma@gmail.com>
This patch changes the Content-Disposition header set on S3 PutObject
requests when uploading attachments to be encoded using a Django helper,
properly encoding UTF-8 in filenames.
This fixes compatibility for uploading attachments with non ASCII file names
to Garage, since the HTTP library used by Garage validates that HTTP Headers
are ASCII only.
Signed-off-by: networkException <git@nwex.de>
Added:
- ✨(frontend) integrate new Blocknote AI feature
- 👷(docker) add arm64 platform support for image builds
- ✨(tracking) add UTM parameters to shared document links
- ✨(frontend) add floating bar with leftpanel collapse button
- ✨(frontend) Can print a doc
- ✨(backend) manage reconciliation requests for user accounts
- 👷(CI) add GHCR workflow for forked repo testing
- ✨(frontend) Move doc modal
- ⚡️(backend) remove content from Document serializer when asked
- ✨(backend) allow the duplication of subpages
- ✨(backend) Onboarding docs for new users
- 🩺(trivy) add trivyignore file and add minimatch CVE
- 🚩 Add feature flags for the AI feature
Changed:
- ♿️(frontend) prevent dates from being focusable
- ♿️(frontend) Focus main container after navigation
- 💄(frontend) align colors and logo with ui-kit v2
- 🚸(backend) sort user search results by proximity
with the active user
- 🚸(oidc) ignore case when fallback on email
- ⚡️(CI) optimize Docker Hub workflow
Fixed:
- 🐛(frontend) fix broadcast store sync
- 🐛(helm) use celery resources instead of backend resources
- 🐛(helm) reverse liveness and readiness for backend deployment
- 🐛(y-provider) use CONVERSION_FILE_MAX_SIZE settings
- 🐛(frontend) fix callout block spacing for old browsers
The Blocknote AI feature is a bit flaky, we want
to be able to disable it if to much issues arise,
without having to do a new release.
We add a bunch of feature flags to be able to
disable the AI features if needed:
- add AI_FEATURE_BLOCKNOTE_ENABLED, to display
or not the feature powered by blocknote
- add AI_FEATURE_LEGACY_ENABLED, to display or not
the legacy AI features
When the tool applyDocumentOperations is used, we have to force the
usage of a system prompt in order to force the model to use it the right
without inventing different actions. The pydantic Agent class can use a
system prompt but this noe is ignoried when a UI adapter is used like
the VercelAiAdapter.
The frontend application is using Vercel AI SDK and it's data stream
protocol. We decided to use the pydantic AI library to use it's vercel
ai adapter. It will make the payload validation, use AsyncIterator and
deal with vercel specification.
This is a naive first switch from sync to async.
This enables the backend to still answer to incomming requests
while streaming LLM results to the user.
For sure there is room for code cleaning and improvements, but
this provides a nice improvement out of the box.
Standard can vary depending on the AI service used.
To work with Albert API:
- a description field is required in the payload
for every tools call.
- if stream is set to false, stream_options must
be omitted from the payload.
- the response from Albert sometimes didn't respect
the format expected by Blocknote, so we added a
system prompt to enforce it.
The frontend can fetch the retrieve endpoint just for having the title.
Everytime, the content is added and to get the content, a request is
made to the s3 bucket. A query string `without_content` can be used, if
the value is `true` then the content is not added (and not also not
fetch).
The settings CONVERSION_FILE_MAX_SIZE was not used
in the y-provider, which caused a 413 Payload
Too Large error when trying to convert a file larger
than 500kb.
This commit updates the y-provider to use the
CONVERSION_FILE_MAX_SIZE settings, allowing it to
handle larger files without throwing an error.
CONVERSION_FILE_MAX_SIZE should follow the same
value as the one defined in the backend settings,
which is 20mb by default.
Adds two methods to allow new users to start with some docs.
User._handle_onboarding_documents_access() gives READER access to
each document listed in settings.USER_ONBOARDING_DOCUMENTS.
User._duplicate_onboarding_sandbox_document() creates a local copy
of the sandbox document specified in
settings.USER_ONBOARDING_SANDBOX_DOCUMENT.
Adds a new with_descendants parameter to the doc duplication API.
The logic of the duplicate() method has been moved to a new
internal _duplicate_document() method to allow for recursion.
Adds unit tests for the new feature.
Add utm_source=docssharelink and utm_campaign={docId} query parameters
to shared document links (copy link and invitation emails).
Signed-off-by: Chaïb Martinez <chaibax@gmail.com>
On the DSFR instance the title will not
be displayed anymore in favor of
a icon that include the title.
So we make the title optional, it
will be configurable from the
theme configuration.
We were partially overriding the frontend with the
cunningham theme meaning at build time. We stop to
do this way to do it only from the configuration
theme. This way it will be easier to maintain and
to update.
We improve as well the typing with more global types
like Image type from logo and icons, and HTMLLinkElement
type for the favicon, meaning you can really
override compoments from the configuration theme.
Some identity providers might change the case, but in our products we
don't consider case variation to be consider as different email
addresses.
Next step would be to normalize the DB value of email to be lower-case.
For now, the reconciliation requests are imported through CSV in the
Django admin, which sends confirmation email to both addresses. When
both are checked, the actual reconciliation is processed, and all
user-related content is updated.
## Purpose
Fix#1616 // Replaces #1708
For now, the reconciliation requests are imported through CSV in the
Django admin, which sends confirmation email to both addresses. When
both are checked, the actual reconciliation is processed, and all
user-related content is updated.
## Proposal
- [x] New `UserReconciliationCsvImport` model to manage the import of
reconciliation requests through a task
(`user_reconciliation_csv_import_job`)
- [x] New `UserReconciliation` model to store the user reconciliation
requests themselves (a row = a `active_user`/`inactive_user` pair)
- [x] On save, a confirmation email is sent to the users
- [x] A `process_reconciliation` admin action process the action on the
requested entries, if both emails have been checked.
- [x] Bulk update the `DocumentAccess` items, while managing the case
where both users have access to the document (keeping the higher role)
- [x] Bulk update the `LinkTrace` items, while managing the case where
both users have link traces to the document
- [x] Bulk update the `DocumentFavorite` items, while managing the case
where both users have put the document in their favorites
- [x] Bulk update the comment system items (`Thread`, `Comment` and
`Reaction` items)
- [x] Bulk update the `is_active` status on both users
- [x] New `USER_RECONCILIATION_FORM_URL` env variable for the "make a
new request" URL in an email.
- [x] Write unit tests
- [x] Remove the unused `email_user()` method on `User`, replaced with
`send_email()` similar to the one on the `Document` model
## Demo page reconciliation success
<img width="1149" height="746" alt="image"
src="https://github.com/user-attachments/assets/09ba2b38-7af3-41fa-a64f-ce3c4fd8548d"
/>
---------
Co-authored-by: Anthony LC <anthony.le-courric@mail.numerique.gouv.fr>
## Purpose
Allows a user to find more easily the other users they search, with the
following order of priority:
- users they already share documents with (more recent first)
- users that share the same full email domain
- ~~users that share the same partial email domain (last two parts)~~
- ~~other users~~
Edit: We need to ilter out other users in order to not reveal email
addresses from members of other organisations. It's still possible to
invite them by email.
Solves #1521
## Proposal
- [x] Add a new function in `core/utils.py`:
`users_sharing_documents_with()`
- [x] Use it as a key to sort the results of a basic user search
- [x] Filter user results to avoid reveal of users (and email addresses)
of other orgs or that have not been interacted with.
- [x] User research through "full" email address (contains the '@') is
left unaffected.
---------
Co-authored-by: Anthony LC <anthony.le-courric@mail.numerique.gouv.fr>
Add support for the `AWS_S3_SIGNATURE_VERSION` environment variable to
allow configuring S3 signature version for compatibility with
S3-compatible storage services like Linode Object Storage.
Fixes#1788
Signed-off-by: dtinth on MBP M1 <dtinth@spacet.me>
Added:
- ✨(frontend) integrate configurable Waffle
- ✨ Import of documents
- 🚨(CI) gives warning if theme not updated
- ✨(frontend) Add stat for Crisp
- ✨(auth) add silent login
- 🔧(project) add DJANGO_EMAIL_URL_APP environment variable
Changed:
- ♿(frontend) improve accessibility:
- ♿️(frontend) fix subdoc opening and emoji pick focus
- ✨(backend) add field for button label in email template
Fixed:
- ✅(e2e) fix e2e test for other browsers
- 🐛(export) fix export column NaN
- 🐛(frontend) add fallback for unsupported Blocknote
languages
- 🐛(frontend) fix emojipicker closing in tree
- 🐛(frontend) display children in favorite
- 🐛(frontend) preserve typed text after @ on escape
Removed:
- 🔥(project) remove all code related to template
Security:
- 🔒️(trivy) fix vulnerability about jaraco.context
We want to split the cache config between the app cache and the session
cache. In the app cache, the default one, we allow to configure a
prefix. By default this prefix is a fixed string so the cache will be
never revoked because it is changing but it allow every instance to
implement its own strategy like prefixing the keyx cache with a
timestamp.
To not impact session, the session cache is splitted in the settings.
Not every project requires silent login.
This commit adds a new feature flag
FRONTEND_SILENT_LOGIN_ENABLED to enable or
disable silent login functionality.
Currently users already logged in to the SSO have to click on
the login button again to be connected.
This extra step should not be necessary.
This commit uses the "silent=true" parameter to the login
endpoint to avoid the extra step.
Most of Docs app is configured thanks to environment
variables, except the url in the email that
was from the django site table.
Now we can set it with DJANGO_EMAIL_URL_APP
environment variable to have a better consistency.
We keep the previous way to avoid breaking
changes.
The max size and allowed extensions for document
import are now fetched from the application
configuration.
This ensures consistency across the app and
allows for easier updates to these
settings in the future.
Added Helm templates for docspec deployment and service to enable
document specification conversion in the Kubernetes environment.
Updated Tiltfile, compose.yml, and Helm values to
configure docspec integration alongside the
backend converter service for document import functionality.
Refactored converter services based on PR #1609 review comments:
- Renamed parameter to `data` across all convert methods for consistency
- Replaced recursive call with explicit sequential calls for readability
- Hardcoded CONVERSION_API_SECURE=True in Production class for security
- Removed unused YdocConverter import from viewsets.py
- Updated tests to match new error message wording
Signed-off-by: Stephan Meijer <me@stephanmeijer.com>
Added comprehensive tests covering DocSpec converter service,
converter orchestration, and document creation with file uploads.
Tests validate DOCX and Markdown conversion workflows, error
handling, service availability, and edge cases including empty
files and Unicode filenames.
Signed-off-by: Stephan Meijer <me@stephanmeijer.com>
We can now import documents in formats .docx and .md.
To do so we added a new container "docspec", which
uses the docspec service to convert
these formats to Blocknote format.
More here: #1567#1569.
The template feature is removed.
Migration created to drop related tables.
Files modified:
- viewsets
- serializers
- models
- admin
- factories
- urls
- tests
- demo data
## Purpose
The email template is made with the idea that they link to a document.
This change allows to customize the label of the button (currently,
"Open") to allow for a different action verb. Additionally, the
'document_title' parameter is renamed to 'link_label' to reflect that it
can link to other things than documents.
## Proposal
- [x] Email template `template.mjml` updated as proposed
- [x] Method `send_email()` updated
- [x] Translations updated
Since celery version 5.6.0 we have trouble with retrying tasks and it is
impactig the malware_detection workflow. We have to use version 5.5.3
while we found the issue.
Added:
- ✨(backend) add documents/all endpoint with descendants
- ✅(export) add PDF regression tests
- 📝(docs) Add language configuration documentation
- 🔒(helm) Set default security context
- ✨(backend) use langfuse to monitor AI actions
Changed:
- ♿(frontend) improve accessibility:
- ♿(frontend) make html export accessible to screen reader users
- ♿(frontend) add missing label and fix Axes errors to improve a11y
Fixed:
- ✅(backend) reduce flakiness on backend test
- 🐛(frontend) fix clickable main content regression
- 🐛(backend) fix TRASHBIN_CUTOFF_DAYS type error
- 💄(frontend) fix icon position in callout block
Security:
- 🔒️(backend) validate more strictly url used by cors-proxy endpoint
- 🔒️(frontend) fix props vulnerability in Interlinking
