The tests are using the local object storage configured in the docker
compose stack. Every time a test using the DocumentFactory is executed,
an object is created in the object storage and will stay here but never
used. The data/media directory is growing and time to time we have to
delete it in order to free some gigabytes. In this commit we add a
global conftest responsible to delete the object created during the test
execution. The space will be really free when the object storage
container will be shutdown.
Changed:
- 💄(frontend) display emoji button on hover
Fixed:
- 🐛(backend) close thread DB connections to fix test teardown
OperationalError
- 🐛(frontend) fix crash when orphaned threads
- 🐛(backend) order trashbin response by most recently deleted
- 🐛(backend) stream document content with an async
iterator under ASGI
- 🐛(frontend) fix long titles in table of content
When docs is ran using an ASGI server, the django StreamingHttpResponse
expect to consume an async generator. If a sync generator is used, it
fully consumes it and then return all the content and we loose the
benefits of using a StreamingHttpResponse.
When all the tests are ran, there is a PytestWarning log present to warn
us that there is remaining database connection open and can not be
closed. It appears that the tests about concurrence are responsible of
leaving connection open. We need to manually close them in these tests.
Added:
- ✨(backend) support creating subdoc from file
- ✨(frontend) comment side panel
- ✨(buildpack) add PaaS deployment support, tested with Scalingo
- 🔧(backend) allow configuring settings
OIDC_OP_USER_ENDPOINT_FORMAT
- ⚡️(helm) create a dedicated svc and deployment for
yprovider converter
- ✨(backend) allow to leave a document
- ✨(frontend) add the presenter mode
- 📈(backend) create a utils to capture event with posthog
- 🔧(backend) new setting DOCUMENT_ALL_ENDPOINT_ENABLED
Changed:
- ♻️(backend) allow global search in sub documents
- ✨(backend) add a breadcrumb in the search response
- ♻️(frontend) move doc action buttons to fix toolbar
- ♿️(frontend) add aria-hidden to decorative avatar SVGs
in share modal
- 🏗️(frontend) move comments to its own folder feature
Fixed:
- 🐛(docs) run migration 0027 without superuser role
- 🐛(backend) prevent admins/owners from overwriting other
users comments
- 🐛(y-provider) return empty output when converting empty
Yjs document
- 🐛(backend) use computed_link_reach in
handle_onboarding_document
- 🐛(frontend) fix toolbar blocknote hidden
- 🐛(frontend) fix application crashes when using
GTranslate and zoom
- 🐛(frontend) fix emoji pdf not matching
- 🐛(backend) fix UnorderedObjectListWarning for
DocumentAskForAccess viewset
In the tests there is a UnorderedObjectListWarning, we also find this same
warning in the logs in production. This warning appears because the
queryset used by the DocumentAskForAccess paginator is not ordered and
can lead to inconsistent result. Ordering the queryset fix this warning.
We want to allow or not the usage of the /documents/all/ endpoint. It has
been created to be used for an other purpose than the js client. For
those who don't use it this new settings allow to disable it. By default
it is set to True to keep the same behavior.
When an AI action is called from ai_translate, ai_transform and
ai_proxy, an event doc_ai_action is captured with a method property
giving the information from where it is called.
The posthog information were only available using the POSTHOG_KEY
settings. We split it in POSTHOG_KEY and POSTHOG_HOST to use them with
the pyhton posthog sdk
We want to allow users to leave a document where they have an access or
they have visited creating a link_trace. All subdocuments should also be
leaved at the same time.
To know if the user can leave a doc we have to check when computing the
abilities if a record is existing in the LinkTrace table. This is a N+1
query situation. To avoid it, we added an annotation in the
DocumentQueryset like we already do to annotate the user role.
There is one edge case where the annotation is made to soon, it is when the
user is visiting a document for the first time, the `get_object` add the
annotation and in the permission, we compute the abilities. The `leave`
property is False because the entry in the LinkTrace table is not made,
when the serializer ask for the abilities again, it is still False. So
in the `retrieve` method in the viewset we force the
`user_has_link_trace` to the correct value.
The mask feature has never been implemented by the frontend application
and it is not what we want to do. We want to implement a feature to
leave a document.
In the model method User::_handle_onboarding_documents_access we do not
allow using documents with restricted link_reach to be added as
onboarding documents. To check the real link_reach of the document, we
must use instead the computed_link_reach to be sure that a sub document
can also be used and compute its correct link_reach.
In the search response, we want to display a breadcrumb for every
document returned. For this we added a "parents" property containing a list
documents, all are the parents of the current document (ordered by their
depth). With this we can easily create a breadcrumb.
The global search endpoint filtering by title in the database was not
searching for accessible sub documents. We change the how global search
is made: it is now returning accessible sub documents matching the
title query.
We need to include resolved threads in the queryset
of the ThreadViewSet, otherwise they won't be
included in the list of threads and we will not be
able to list resolved threads in the side panel.
Admins/owners could overwrite another user's comment text and silently
take ownership of it.
Split write permissions: destroy stays broad (author + admin/owner) for
moderation.
Signed-off-by: Mohamed El Amine BOUKERFA <boukerfa.ma@gmail.com>
The djang-lasuite package manages the settings
OIDC_OP_USER_ENDPOINT_FORMAT allowing to enforce the format use by the
user endpoint. By default it is set to `AUTO`.
The migration 00227 must be run with a postgres superuser, most af
managed postgresql database can not be run using this kind of user.
Ti fix this, we use postgresql unnacent function instead of accessing C
function.
When a document is moved outside its current permission scope (root
document, cross-tree move, or promotion to root), its direct accesses
and pending invitations are now deleted server-side within the same
atomic transaction as the move itself. This ensures consistency: if
the move fails, deletions are rolled back.
Signed-off-by: Mohamed El Amine BOUKERFA <boukerfa.ma@gmail.com>
The children/ endpoint was missing file upload support that the root
documents endpoint already had. Added file-to-YJS conversion handling to
subdocument creation.
Added
- ⚡️(frontend) add skeleton on content loading #2254
- ⚡️(frontend) close websocket connection when user change tab #2264
Changed
- 🏗️(core) migrate from pip to uv
Fixed
- 🩺(project) reload app if front and back unsync #2276
- 🐛(frontend) fix patch and comments #2273
- 🐛(frontend) interlinking are exported correctly in print mode #2269
- 💬(frontend) add missing link in onboarding description #2233
- 🐛(frontend) sanitize pasted and dropped content in document title #2210
- 🐛(frontend) Emoji menu doesn't display above comment box #2229
- 🐛(frontend) Block menu doesn't stay open on 1st line #2229
- 🐛(frontend) The "+" on the first line of a new doc doesn't work #2229
- 🐛(backend) manage race condition between GET and PATCH content #2271
- 🐛(backend) replace document creation table locks with retry strategy #2274
Security
- 🔒️(frontend) sanitize color during collaboration #2270
We observe some cases where the frontend and
backend versions can get out of sync, which can
cause issues.
To mitigate this, we want to implement a mechanism
that detects when the frontend and backend
versions are mismatched and triggers a
reload of the application to ensure they are in sync.
We already migrate from pip to uv to manage our dependencies. We can also
migrate the build backend from setuptool to uv_build.
In the pyproject file, the readme property has been removed, because
uv_build try to read it, but the readme is at the root of the project
and not copied into the Dockerfile instructions. This readme can be used
when the package is published on pypi but it is not the case for Docs.
We have situation where the number of locks in the database can increase
dangerously creating deadlock situation. To remove this situation we
decided to change the strategy to manage document creation concurrency.
We decided to use a retry strategy, trying to create the document
multiple times while a usable path is found. To avoid having an
inifinite loop, we use a max_attempts counter configurable using the
setting TREEBEARD_PATH_COMPUTE_RETRY_MAX_ATTEMPTS
We need to split the core/utils.py in multiple submodule created in
core/utils/*.py. We need to do this to avoir circular import between
this module and the models module.
When a PATCH and a GET on the content endpoint are made at the same time
for different users a race condition can happen and the metadata
returned
by the S3 head_object can be outdated when the object is fetched leading
to an error raised because the Content-Length header does not match the
size of the response body. To avoid this, we no longer used head_object
followed bu get_object, we have to manage
everything in one call with the get_object. The get_object also accepts
as parameters an etag or last-modified header and will return a 304 if
the content has not changed, so we can use this to not return the entire
body if this one has not changed.
