Commit Graph
583 Commits
Author SHA1 Message Date
Anthony LCandAnthony LC 67915151aa (e2e) add a test on doc creation server side
We recently added a new feature to the app, which
is the ability to create a document from server to
server.
Server A will send a request to Server B with
a markdown content, and Server B will create a
the document after converting the markdown to
yjs base64 format.
This test will check all the steps of the process
and assert that the document is displayed correctly
on the frontend in the blocknote editor.
2024-12-17 14:49:23 +01:00
Anthony LCandAnthony LC 59e74e6eeb 🐛(e2e) fix flaky tests
3 tests were flags are flaky or bringing flakiness.
We improved them.
2024-12-17 12:42:02 +01:00
Anthony LCandAnthony LC 4e7f095b0f ️(e2e) set maxFailures with CI
If a test fails (retries included), the test runner
will stop after reaching maxFailures.
We will not have to wait for all tests to
run to see the results.
2024-12-17 12:42:02 +01:00
Anthony LCandAnthony LC 6a0d2e21b5 🐛(frontend) fix rerendering when doc is saving
When the document is saved, the blocknote toolbar
was rerendering, causing the toolbar to close
some panels.
It was creating flakiness in the e2e tests, plus
it was not a good user experience.
This commit fixes this issue.
2024-12-17 12:42:02 +01:00
Anthony LCandAnthony LC b79d5fccbc ⬆️(dependencies) update js dependencies 2024-12-16 18:28:37 +01:00
Anthony LCandAnthony LC 6d77cb1801 ️(docker) improve y-provider image
Improve y-provider image by having the
node_modules as small as possible.
We move split the Dockerfile and
add it to the y-provider folder,
it will be easier to read and maintain.
2024-12-16 17:39:45 +01:00
lebaudantoineandaleb_the_flash e4a45a556c 🐛(backend) fix bucket access in the tilt stack
S3 username was desynchronized with the helmfile. Leading to error,
when patching object or saving any update to the Minio bucket.

@rouja fixed it.
2024-12-16 17:17:42 +01:00
lebaudantoineandaleb_the_flash 3ca39ceb8a ♻️(yprovider) support multiple API keys to separate responsibilities
Support for two API keys has been added to the YProvider microservice to
decouple responsibilities between the collaboration server and other
endpoints. This improves security by scoping keys to specific purposes and
ensures a clearer separation of concerns for easier management and debugging.
2024-12-16 17:17:42 +01:00
lebaudantoineandaleb_the_flash 8a93122882 (yprovider) add test to prevent silent breaking changes
Per Quentin's request, added a test to ensure developers are warned
if the token format is updated, preventing backend compatibility issues.
2024-12-16 17:17:42 +01:00
lebaudantoineandaleb_the_flash 8eb986591a 💡(backend) warm about the token nature of Yprovider microservice
Note to the future myself, using a raw token format is
not common. It should be refactor
2024-12-16 17:17:42 +01:00
lebaudantoineandaleb_the_flash c10808b611 ♻️(backend) generalize YProvider API config
Abstracted base URL and API key under 'y-provider' for
reuse in future endpoints, aligning with microservice naming.

Please note the YProvider API here is internal to the cluster.
In facts, we don't want these endpoints to be exposed by any ingress
2024-12-16 17:17:42 +01:00
lebaudantoineandaleb_the_flash ba63358098 🔧(backend) configure conversion microservice in dev
Update helm values to configure the conversion microservice while
creating document server to server.
2024-12-16 17:17:42 +01:00
lebaudantoineandaleb_the_flash 52534db3e1 🐛(backend) fix issues with conversion microservice integration
Minor adjustments were needed after working in parallel on two PRs.
The microservice now accepts an API key without requiring it as a Bearer token.

A mistake in reading the microservice response was corrected after refactoring
the serializer to delegate logic to the converter microservice.
2024-12-16 17:17:42 +01:00
renovate[bot]andAnthony LC dc9b375ff5 ⬆️(dependencies) update python dependencies 2024-12-16 10:45:49 +01:00
renovate[bot]andAnthony LC 65fdf115be ⬆️(dependencies) update django to v5.1.4 [SECURITY] 2024-12-13 21:28:11 +01:00
Anthony LCandAnthony LC ecb2b35ec8 (email) white brand email
The email was branded "La Suite Numérique",
we updated the template to make it generic, we
will use settings env variables to customize the
email for each brand.
2024-12-13 17:58:43 +01:00
renovate[bot]andSamuel Paccoud 2d13e0985e ⬆️(dependencies) update python dependencies 2024-12-12 18:56:25 +01:00
lebaudantoineandaleb_the_flash 5014443f80 💩(y-provider) init a markdown converter endpoint
This code is quite poor. Sorry, I don't have much time working
on this feature. However, it should be functional.

I've reused the code we created for the Demo with Kasbarian.
I've not tested it yet with all corner case. Error handling
might be improved for sure, same for logging.

This endpoint is not modular. We could easily introduce options
to modify its behavior based on some options. YAGNI

I've added bearer token authentification, because it's unclear
how this micro service would be exposed. It's totally not required
if the microservice is not exposed through an Ingress.
2024-12-12 14:37:30 +01:00
lebaudantoineandaleb_the_flash 3fef7596b3 (y-provider) create utils function toBase64
Add utility function to convert BitArray to Base64 string.
This is required for creating Base64-encoded documents,
as the frontend do.
2024-12-12 14:37:30 +01:00
lebaudantoineandaleb_the_flash 19042907be (y-provider) add BlockNote server utils and yjs
Needed dependencies to mimic frontend code when generating a
document from a markdown string.

Will be used in the upcoming commits.
2024-12-12 14:37:30 +01:00
Samuel Paccoud - DINUMandaleb_the_flash 5cdd06d432 (backend) add server-to-server API endpoint to create documents
We want trusted external applications to be able to create documents
via the API on behalf of any user. The user may or may not pre-exist
in our database and should be notified of the document creation by
email.
2024-12-12 14:01:46 +01:00
Samuel Paccoud - DINUMandaleb_the_flash 47e23bff90 🔥(emails) remove dead template code
The email footer file is not being used and should be deleted.
2024-12-12 14:01:46 +01:00
Anthony LCandAnthony LC 7dfc62b2c5 🔖(minor) release 1.9.0
Added:
- (backend) annotate number of accesses
on documents in list view
- (backend) allow users to mark/unmark
documents as favorite

Changed:
- 🔒️(collaboration) increase collaboration access security
- 🔨(frontend) encapsulated title to its own component
- ️(backend) optimize number of queries on
document list view
- ♻️(frontend) stop to use provider with version
- 🚚(collaboration) change the websocket key name

Fixed:
- 🐛(frontend) fix initial content with collaboration
- 🐛(frontend) Fix hidden menu on Firefox
- 🐛(backend) fix sanitize problem IA
2024-12-12 08:37:10 +01:00
Anthony LCandAnthony LC 39c4af0a7c 🐛(frontend) hide cursor for authenticated users
When a authenticated user was in read-only mode,
the cursor was still visible.
This commit hides the cursor for authenticated users
in read-only mode.
2024-12-11 21:18:40 +01:00
Anthony LCandAnthony LC 57c5c394f5 🐛(backend) improve sanitizer ai_services
The json response of the AI service is badly formatted.
This commit improves the sanitizer to try
to handle the response correctly.
2024-12-11 21:18:40 +01:00
Anthony LCandAnthony LC be6da38a08 🐛(frontend) only owner will make initial content
In some cases, the sync of the initial content
is not being done correctly.
We will let only the owner of the document
to make the initial content.
2024-12-11 21:18:40 +01:00
Anthony LCandAnthony LC fc36ed08f1 🐛(frontend) fix initial content with collaboration
The way the initial content was created was causing
issues with the collaboration server.
As soon a user started typing, the problem was gone.
This commit fixes that by letting Blocknote
managing the initial content, then we update the
Blocknote initial content with our initial content.
2024-12-11 16:08:13 +01:00
Anthony LCandAnthony LC ed90769081 🐛(backend) fix sanitize problem IA
Albert send us back a malformed IA json, the
sanitize function was not able to handle it correctly.
We add a try catch on it, to not use the sanitizer if
the json.loads fails.
2024-12-11 15:21:56 +01:00
Anthony LCandAnthony LC a8310fa0ff ️(frontend) remove debounce on useHeadings
We remove the debounce on useHeadings, it
decreases the user experience and it's not
necessary a big performance improvement.
2024-12-11 14:54:41 +01:00
Anthony LCandAnthony LC a902e31521 🔧(helm) add ingress collaboration api
We need to keep the stickyness between the
collaboration api and the ws server, to do so,
we will use "upstream-hash-by: $arg_room", meaning
that the stickyness will be based on the room query.
We need to ahve 2 ingress to handle the
"collaboration_auth", only the ws routes has to
use the "collaboration_auth" subrequest.
2024-12-11 14:54:41 +01:00
Anthony LCandAnthony LC 932ab13d97 📈(collaboration) add sentry
Add sentry to the collaboration server.
It will be used to log errors and exceptions.
2024-12-11 14:54:41 +01:00
Anthony LCandAnthony LC 94a1ba7989 (backend) notify collaboration server
When an access is updated or removed, the
collaboration server is notified to reset the
access connection; by being disconnected, the
accesses will automatically reconnect by passing
by the ngnix subrequest, and so get the good
rights.
We do the same system when the document link is
updated, except here we reset every access
connection.
2024-12-11 14:54:41 +01:00
Anthony LCandAnthony LC bfecdbf83a (y-provider) add tests for y-provider server
We add jest tests for the y-provider server.
The CI will be able to run the tests.
2024-12-11 14:54:41 +01:00
Anthony LCandAnthony LC ba1cfc3c27 (y-provider) endpoint POST /collaboration/api/reset-connections
We want to be able to reset the connections of a document.
To do this, we need to be able to send a
request to the collaboration server.
To do so, we added the endpoint
POST "/collaboration/api/reset-connections"
to the collaboration server thanks to "express".
2024-12-11 14:54:41 +01:00
Samuel Paccoud - DINUMandAnthony LC 2cba228a67 🧑‍💻(helm) rename minio root user password
Using "impress" as the name of minio's root user in Tilt's
dev environment, was triggering obfuscation of the logs in Tilt's
console each time the word "impress" was used.
This made the logs hard to read.
2024-12-11 14:54:41 +01:00
Samuel Paccoud - DINUMandAnthony LC 66553ee236 (backend) add subrequest auth view for collaboration server
We need to improve security on the access to The collaboration server
We can use the same pattern as for media files leveraging the nginx
subrequest feature.
2024-12-11 14:54:41 +01:00
Samuel Paccoud - DINUMandAnthony LC 64674b6a73 ♻️(backend) rename, factorize and improve the subrequest media auth view
We want to use the same pattern for the websocket collaboration service
authorization as what we use for media files.

This addition comes in the next commit but doing it efficiently
required factorizing some code with the media auth view.
2024-12-11 14:54:41 +01:00
Anthony LCandAnthony LC a9def8cb18 ♻️(frontend) create useHeadings hook
- We create the useHeadings hook to manage the
headings of the document and staty DRY.
- We use the headings store in IconOpenPanelEditor
and TableContent, to avoid prop drilling.
- We add a debounce on the onEditorContentChange
to improve a bit the performance.
2024-12-06 15:23:16 +01:00
Anthony LCandAnthony LC f606826098 ♻️(frontend) stop to use provider with version
Version are not editable, we don't need to activate
the collaboration provider for them.
Simplify the code by removing the provider
from the version.
2024-12-06 15:23:16 +01:00
Anthony LCandAnthony LC aff036d9fb 🚚(collaboration) change the websocket url name
We will have 2 urls targeting the server, better
to improve the naming to avoid confusion.
2024-12-06 15:23:16 +01:00
rvveberandAnthony LC 131eefa1ac 🔨(frontend) encapsulate title component
in order to modularize in the future
the title component is encapsulated.
2024-12-06 14:16:24 +01:00
Anthony LCandAnthony LC b4e639cc24 ♻️(frontend) adapt Blocknote button
Last upgrade of Blocknote changes the editor
method getSelection, the blocks were not being
selected in certain cases.
We updated the methods to select the blocks
correctly.
2024-12-05 23:34:06 +01:00
Samuel Paccoud - DINUMandAnthony LC ba962af914 ⬆️(backend) bump openai library version as it breaks tests
This looks like an instability in the openai library's definition
of dependencies.
2024-12-05 23:34:06 +01:00
Anthony LCandAnthony LC 76514a6e2b 🏷️(frontend) adapt typing with recent upgrade
An upgrade to @sentry/nextjs@8.42.0 changed
some typing. It is not from @sentry/types but
from @sentry/core now.
2024-12-05 23:34:06 +01:00
Anthony LCandAnthony LC b69a5342d9 ⬇️(dependency) downgrade workbox-webpack-plugin to 7.1.0
In the 1.8.0 we experienced issues with the service
worker not updating properly. We suspect that the
workbox-webpack-plugin is the cause of this issue.
Better to downgrade to the last version that worked
until we have time to investigate the issue.
We add workbox-webpack-plugin to the renovate.json
file to avoid future updates.
2024-12-05 23:34:06 +01:00
renovate[bot]andAnthony LC c25682f199 ⬆️(dependencies) update js dependencies 2024-12-05 23:34:06 +01:00
Anthony LCandAnthony LC eec8b4d2c3 ♻️(frontend) adapt frontend with new access types
We don't get the accesses anymore from the backeend,
instead we get the number of accesses.
We remove the list of owners in the doc header because
we don't have easily this informations anymore and
we will have to do a bigger refacto.
2024-11-28 16:02:27 +01:00
Samuel Paccoud - DINUMandAnthony LC 1af7b797bc (backend) test interference btw documents permissions and filtering
We want to make sure that applying filters on the document view list
does not interfere with permissions.
2024-11-28 16:02:27 +01:00
Samuel Paccoud - DINUMandAnthony LC b5c159bf63 (backend) allow filtering on document titles
This is the minimal and fast search feature, while we are working on
a full text search based on opensearch. For the moment we only search
on the title of the document.
2024-11-28 16:02:27 +01:00
Samuel Paccoud - DINUMandAnthony LC bfbdfb2b5c (backend) allow filtering documents by link reach
We want to be able to limit document list views to only public documents,
or only restricted or authenticated documents.
2024-11-28 16:02:27 +01:00