We want to export metrics to prometheus. We install for this
django-prometheus. By default the feature is disabled and must be
explicitly enabled. A complete documentation is available in
documentation/metrics.md
Previously we added to retrieve document response a new property
user_access_since that contains the date from when the user started to
have access to the document. The way t was made added an other
annotation to the Document queryset making the sql query more and more
complex. We decided to lighten the queryset and expose the access the
user has on the document instead and read the history from the
created_at property.
The yhub service will need our public key in order to validate the jwt
token we will used. We choose to expose a jwks endpoint as it is a
standard wat to do this.
Added:
- 🔧(backend) fine tune redis cache options
- ✨(frontend) make the full last-update date available
- 💄(frontend) redesign email confirmation standalone page
Changed:
- ⬆️(backend) upgrade celery to version 5.6.3
- ⚡️(backend) stop using LEFT(value, LENGTH(path)) in sql queries
- 🚚(project) switch docspec image to ghcr.io/docspec/api
- 🚚(global) move favorite documents API endpoint
to `/documents/favorites/`
Fixed:
- 🐛(backend) skip session creation for the liveness probe
- 🐛(frontend) preserve page titles when adding an emoji
- 🐛(frontend) scroll to the linked block in read-only documents
- 🐛(frontend) hide the selection highlight on presenter images
- 🐛(y-provider) prevent process crash on malformed websocket frames
- 🐛(frontend) keep commented text sharp when printing to PDF
- 🐛(docker) pull minio images from quay.io
- ♿️(frontend) restore presenter focus trapping after share links
- 🐛(frontend) export any raster image supported by the browser to a PDF
Added
- ✨(frontend) export presenter slides as PDF #2487
Fixed
- 🐛(frontend) hide Leave in the doc menu when not logged in #2626
- 🐛(backend) allow to configure settings DATA_UPLOAD_MAX_MEMORY_SIZE
We removed previously whitenoise because it was not working with asgi
application. By removing it we also removed the way to serve the static
files in the application. There is an existing fork of whitenoise,
servestatic, that manage async application and we can use it to serve
static files.
Added:
- ✨(frontend) Add "Copy link to block" feature
- ✨(frontend) add word count to doc header toolbox
- ✨(frontend) add find and replace feature to the editor
Changed:
- ♿️(frontend) use anchor links for interlinking sub-documents
- ✨(frontend) reset side panel state between documents
- ♿️(frontend) announce search loading state for screen readers
- ♻️(frontend) change favorite to star
- 🚚(frontend) add doc move to doc options
- ♻️(frontend) unified menu
- ♿(frontend) hide decorative emojis in document titles from SR
- ♻️(frontend) save the doc with a keepalive request when
leaving the page
Fixed:
- 🐛(frontend) fix clipped formatting toolbar in new comment
composer
- 🐛(backend) fix duplicating a document that has no content
- 📄(frontend) allowed partially export when MIT
- 🐛(backend) manage async support for Docs custom middleware
Removed:
- 🔥(backend) remove whitenoise package
whitenoise middleware is failing a lot with a cancelled exception from
asyncio. Using whitenoise is not needed in our case, we are just serving
an API with django and DRF. We decided to completely remove it.
Added:
- ♿️(frontend) restore skip to content link after header redesign
- 🌐(i18n) rename cn_CN to zh_CN, add eo_PL and zh_TW locales
- ✨(backend) conditional email notification in server to server api
- ✨(backend) profile api using django-silk
Changed:
- ♿️(frontend) use semantic `<dl>` structure in document info card
- ⚡️(frontend) replace onboarding assets with webm and webp
- 💄(frontend) use the same highlight color for cells and moves
- ⚡️(backend) optimize media_auth endpoint
- 🚸(frontend) print from document options menu
Fixed:
- 🐛(frontend) refresh pins after document deletion and restoration
- 🐛(frontend) redirect homepage to login when homepage feat
is disabled
- 🐛(backend) ignore CSPs for API docs in development
- 🐛(frontend) export images embedded with a relative url
- 🐛(y-provider) fix sentry init
- 🐛(backend) handle object storage metadata keys case-insensitively
- 🐛(keycloak) fix database env variables in the self-hosting example
- 🐛(helm) show the database error while jobs wait for it to be ready
We have performance issue with the media_auth endpoint. We need to
monitor and profile this endpoint to understand what is happening. We
decided to install and configure django-silk. By default django-silk is
not enabled and not accessible, once enabled migration must be run and
the /silk/ endpoint will be accessible only using a super user.
To have relecant profiles and monitor we must have a database comparable
to a production one because we are facing a scaling issue. Unless a
million documents probably nothing is happening, for this we added
several management commands to help us. A first one to anonymize a
database than can be reused without any risk to leak PII data, can be
called using python manage.py anonymize_database with some mandatory
options. An other command to generate a database profile, this command
should be run on a production database, this profile can then be used by
the generate_volumetry to populate a development database with same
volumetry than describe in the profile.
We upgraded pydantic-ai-slim to the latest version and add mistral has
extra dependency. Doing this, the direct dependance to mistral can be
removed and will be managed with pidantic-ai-slim package. The import of
mistralai has changed, we have to updagre them from mistralai to
mistralai.client
Changed:
- ♻️(backend) reset collaboration connection in cascade for all children
Fixed:
- 💄(frontend) fix some UI/UX in the left panel
- 🐛(frontend) fix tree dnd firefox
Added:
- ✨(y-provider) preserve callouts, PDFs, page breaks, interlinking
links and commented text on HTML/markdown export
- ✨(frontend) add a user menu
- ✨(frontend) new header and responsive harmonization
- ✨(backend) add management command to reset a Document
Changed:
- ♿️(frontend) hide mobile left panel from screen readers when collapsed
- ♿️(frontend) enable blocknote heading ids for toc anchors
- ♿️(frontend) focus export modal on format select
- ♿️(frontend) configurable legal submenu in HelpMenu, remove Crisp
- ♻️(frontend) new create button for docs
- ♿️(frontend) align search modal field label with placeholder
- 🚚(frontend) move Waffle to bottom left
- ♿️(frontend) remove redundant aria-label on table of contents links
- ♻️(core) fix typo in settings COLLABORATION_WS_NOT_CONNECTED_READY_ONLY
- ♻️(backend) scope document search by document id instead of path
Fixed:
- 🐛(backend) prevent owner from leaving a soft-deleted document
- 🐛(frontend) fix removed item in the tree
- 🐛(frontend) fix service worker causing reload on tab focus
- 🐛(backend) update restore ability for inherited deletion
- 🔧(dev) make the dev stack domain-agnostic
- 🐛(frontend) stop force index redirect when delete doc
- 🐛(frontend) fix CTA on move modal on mobile
Removed:
- 🔥(backend) remove unused default authentication backend
Added
- ✨(backend) add limit on distinct reactions per comment #1978
- ✨(frontend) leave a document #2410
- ✨(frontend) add top parent on sub docs search #1952
- ✨(frontend) unauthenticated users can search #2407
- ✨(backend) specific user delete method to delete its relations #2437
Changed
- 👷(CI) remove test-e2e-other-browser job #2404
- ♿️(frontend) use heading element for pinned documents section title #2380
- ♿️(frontend) use anchor links for table of contents entries #2390
- ♿️(frontend) improve presenter mode screen reader and keyboard support #2383
- ♿️(frontend) link export modal name to its heading #2422
Fixed
- 🐛(frontend) overlap of block menu dropdown #2406
- ⚡️(backend) fix N+1 queries when serializing thread comments #2415
Changed:
- 💄(frontend) display emoji button on hover
Fixed:
- 🐛(backend) close thread DB connections to fix test teardown
OperationalError
- 🐛(frontend) fix crash when orphaned threads
- 🐛(backend) order trashbin response by most recently deleted
- 🐛(backend) stream document content with an async
iterator under ASGI
- 🐛(frontend) fix long titles in table of content
When docs is ran using an ASGI server, the django StreamingHttpResponse
expect to consume an async generator. If a sync generator is used, it
fully consumes it and then return all the content and we loose the
benefits of using a StreamingHttpResponse.
Added:
- ✨(backend) support creating subdoc from file
- ✨(frontend) comment side panel
- ✨(buildpack) add PaaS deployment support, tested with Scalingo
- 🔧(backend) allow configuring settings
OIDC_OP_USER_ENDPOINT_FORMAT
- ⚡️(helm) create a dedicated svc and deployment for
yprovider converter
- ✨(backend) allow to leave a document
- ✨(frontend) add the presenter mode
- 📈(backend) create a utils to capture event with posthog
- 🔧(backend) new setting DOCUMENT_ALL_ENDPOINT_ENABLED
Changed:
- ♻️(backend) allow global search in sub documents
- ✨(backend) add a breadcrumb in the search response
- ♻️(frontend) move doc action buttons to fix toolbar
- ♿️(frontend) add aria-hidden to decorative avatar SVGs
in share modal
- 🏗️(frontend) move comments to its own folder feature
Fixed:
- 🐛(docs) run migration 0027 without superuser role
- 🐛(backend) prevent admins/owners from overwriting other
users comments
- 🐛(y-provider) return empty output when converting empty
Yjs document
- 🐛(backend) use computed_link_reach in
handle_onboarding_document
- 🐛(frontend) fix toolbar blocknote hidden
- 🐛(frontend) fix application crashes when using
GTranslate and zoom
- 🐛(frontend) fix emoji pdf not matching
- 🐛(backend) fix UnorderedObjectListWarning for
DocumentAskForAccess viewset
The posthog information were only available using the POSTHOG_KEY
settings. We split it in POSTHOG_KEY and POSTHOG_HOST to use them with
the pyhton posthog sdk
Added
- ⚡️(frontend) add skeleton on content loading #2254
- ⚡️(frontend) close websocket connection when user change tab #2264
Changed
- 🏗️(core) migrate from pip to uv
Fixed
- 🩺(project) reload app if front and back unsync #2276
- 🐛(frontend) fix patch and comments #2273
- 🐛(frontend) interlinking are exported correctly in print mode #2269
- 💬(frontend) add missing link in onboarding description #2233
- 🐛(frontend) sanitize pasted and dropped content in document title #2210
- 🐛(frontend) Emoji menu doesn't display above comment box #2229
- 🐛(frontend) Block menu doesn't stay open on 1st line #2229
- 🐛(frontend) The "+" on the first line of a new doc doesn't work #2229
- 🐛(backend) manage race condition between GET and PATCH content #2271
- 🐛(backend) replace document creation table locks with retry strategy #2274
Security
- 🔒️(frontend) sanitize color during collaboration #2270
We already migrate from pip to uv to manage our dependencies. We can also
migrate the build backend from setuptool to uv_build.
In the pyproject file, the readme property has been removed, because
uv_build try to read it, but the readme is at the root of the project
and not copied into the Dockerfile instructions. This readme can be used
when the package is published on pypi but it is not the case for Docs.
Added
- ✨(backend) create a dedicated endpoint to update document content
- ⚡️(backend) stream s3 file content with a dedicated endpoint
- ✨(backend) allow to use new ai feature using mistral sdk
Changed
- ♻️(backend) rename documents content endpoint in `formatted-content` (BC)
- 🚸(frontend) show Crisp from the help menu #2222
- ♿️(frontend) structure correctly 5xx error alerts #2128
- ♿️(frontend) make doc search result labels uniquely identifiable #2212
- ⬆️(backend) upgrade docspec to v3.0.x and adapt converter API #2220
- ✨(backend) make forward auth request uri header configurable #2241
- ♿️(frontend) fix sidebar resize handle for screen readers #2122
Fixed
- 🚸(frontend) redirect on current url tab after 401 #2197
- 🐛(frontend) abort check media status unmount #2194
- ✨(backend) order pinned documents by last updated at #2028
- 🐛(frontend) fix app shallow reload #2231
- 🐛(frontend) fix interlinking modal clipping #2213
- 🛂(frontend) fix cannot manage member on small screen #2226
- 🐛(backend) load jwks url when OIDC_RS_PRIVATE_KEY_STR is set
- 🐛(backend) Prevent moving document to its own descendant or self #2208
- 🐛(backend) return 400 when restoring a non-deleted document #2225
We give the possibility, for the new ai feature, to choose between using
the OpenAI or Mistral sdk. For instances having access to the mistral
infrastructure, using it is mor appropriated than using the openai
compatible chat model.
Validate emojis in ReactionSerializer (previously accepted
any string), preventing multiple emojis or text uploads in
a single reaction
Signed-off-by: Mohamed El Amine BOUKERFA <boukerfa.ma@gmail.com>
Added:
- 🚸(frontend) allow opening "@page" links with
ctrl/command/middle-mouse click
- ✅ E2E - Any instance friendly
Changed:
- ♻️(backend) do not paginate threads list response
- 💄(frontend) Use StyledLink for sub doc tree
Fixed:
- 🐛(frontend) Fix drop cursor creating columns
- 🐛 Fixed side effects between comments and versioning
Changed
- 💫(frontend) fix the help button to the bottom in tree #2073
- ♿️(frontend) improve version history list accessibility #2033
- ♿️(frontend) fix more options menu feedback for screen readers #2071
- ♿(frontend) focus skip link on headings and skip grid dropzone #1983
- ♿️(frontend) fix search modal accessibility issues #2054
- ♿️(frontend) add sr-only format to export download button #2088
- ♿️(frontend) announce formatting shortcuts for screen readers #2070
- ✨(frontend) add markdown copy icon for Copy as Markdown option #2096
- ♻️(backend) skip saving in database a document when payload is empty #2062
Fixed
- ♿️(frontend) fix aria-labels for table of contents #2065
- 🐛(backend) allow using search endpoint without refresh token enabled #2097
## Purpose
For beta testing purposes we need to be able to activate Find hybrid
search to some users, Find full-text search to some others and leave
remaining users on basic DRF title search.
## Proposal
The solution proposed is based on [django-waffle
](https://waffle.readthedocs.io/en/stable/types/flag.html).
- [x] install waffle and activate the default app in settings.
- [x] implement `_get_search_type` in `DocumentViewset` to determine
which search type (title, hybrid or full-text) to use.
- [x] send the `search_type` in the search query.
## External contributions
Thank you for your contribution! 🎉
Please ensure the following items are checked before submitting your
pull request:
- [x] I have read and followed the [contributing
guidelines](https://github.com/suitenumerique/docs/blob/main/CONTRIBUTING.md)
- [x] I have read and agreed to the [Code of
Conduct](https://github.com/suitenumerique/docs/blob/main/CODE_OF_CONDUCT.md)
- [x] I have signed off my commits with `git commit --signoff` (DCO
compliance)
- [x] I have signed my commits with my SSH or GPG key (`git commit -S`)
- [x] My commit messages follow the required format: `<gitmoji>(type)
title description`
- [x] I have added a changelog entry under `## [Unreleased]` section (if
noticeable change)
- [x] I have added corresponding tests for new features or bug fixes (if
applicable)
---------
Signed-off-by: charles <charles.englebert@protonmail.com>
Since we upgraded to django-treebeard version 5 we have anormal behavior
and a high error rate on the document.path property. We must downgrade
it and avoid future upgrade from renovate.
Added:
- ✨(backend) add a is_first_connection flag to the User model
- ✨(frontend) add onboarding modal with help menu button
Changed:
- ♿(frontend) localize LaGaufre label fallback in Docs
- ✨(backend) add a migration cleaning on-boarding
document accesses
- ⬆️(frontend) upgrade Next.js to v16
- ♿️(frontend) fix aria-label and landmark on document
banner state
- 🌐(i18n) add "new window" translation key for waffle
aria-label
Fixed:
- 🐛(backend) create a link_trace record for on-boarding
documents
- 🐛(backend) manage race condition when creating sandbox
document
- 🐛(frontend) fix flickering left panel
- ♿️(frontend) improve doc tree keyboard navigation
Added:
- ✨(helm) allow all keys in configMap as env var
Changed:
- 📝(docs) improve README and add documentation hub
- ♿️(frontend) restore focus to triggers after closing menus and modals
- 🚸(frontend) change position elements toolbar
- ♿️(frontend) add focus on open to modals
Fixed:
- 🐛(frontend) analytic feature flags problem
- 🐛(frontend) fix home collapsing panel
- 🐛(frontend) fix disabled color on icon Dropdown
- 🐛(frontend) fix zIndex table of content
- 🐛(frontend) fix bug when language not supported by BN
- 🐛 (backend) prevent privileged users from requesting access
We upgrade pydantic-ai-slim to version 1.63.0, we have an error when
Docs is run in preprod environment.
We have to investigate why and we decided to downgrade to a working
version
Manuel Raynaud