Every other package in this repository is installed
with yarn; yhub-server was the only one on npm,
with its own package-lock.json.
To ensure consistency it now uses yarn like the rest
of the project.
package-lock.json is replaced by yarn.lock, a packageManager
field is added, and the Dockerfile, CI workflow, Makefile,
helm init-db job and the documentation move from
`npm ci` / `npm run init-db` to
`yarn install --frozen-lockfile` / `yarn init-db`.
The S3 persistence plugin records the version id of the object it wrote
and names that version when it deletes it. On a versioned bucket - what
a deployment runs - a delete that names no version deletes nothing: it
writes a delete marker and keeps every version underneath. Each
compaction supersedes the blobs of the one before, so what was kept was
every version of every document ever written, a document someone asked
to erase included, still readable by anyone who can list versions. On
AWS this needs s3:DeleteObjectVersion, which a policy granting
s3:DeleteObject alone does not cover.
Blobs are written to the bucket for every branch of a document.
YHUB_S3_PERSISTENCE now governs only whether new blobs are written
there. The plugin itself is attached whenever the YHUB_S3_* settings
name a bucket, on or off, because reading is the half that must never
be taken away: a row pointing at an object is unreadable without the
plugin that wrote it, and yhub reports such a version as having no
content rather than as an error. Turning the toggle off stops the
writing and leaves the reading alone; it is the settings, not the
toggle, that a deployment whose bucket holds anything must keep. Half a
configuration is a startup error naming what is missing, as before.
The dev stack keeps the toggle off and creates its bucket versioned, so
flipping it on exercises what a deployment runs rather than a simpler
case. Its createbuckets job needed fixing to do so: the folded yaml
block joins its lines with a space, so the trailing backslashes reached
the shell as an escaped space glued to the next word and everything
past the first && silently did nothing - the media bucket never had
versioning enabled either.
Signed-off-by: Kevin Jahns <kevin.jahns@protonmail.com>
The version history has been dead since the migration. It listed S3 object
versions of the legacy `{pk}/file` key, and nothing writes that key any more,
so every document's list has been frozen at its migration date; restoring one
was a stub that closed the modal and did nothing, while still promising that
the document would be replaced.
It now reads the collaboration server, which is what keeps the history: the
list comes from `activity`, a selected version is previewed from `changeset`
as the document stood at that moment, and restoring one is a `rollback`.
A version is a minute of editing — changes less than a minute apart become
one, and none spans more than a minute. The collaboration server groups only
changes by the same author, so the browser merges what is left across authors:
a version is a moment in the document, not a moment in one person's editing.
Both are needed, and both use the same rule.
This grants `history.rollback` to editors, which is the first time a browser
may change the past rather than read it, and publishes the rollback route.
A reader is refused it twice over — the collaboration server treats it as a
dead grant without document write access, and the endpoint is withheld as well.
Mutations refuse where reads clamp, so a rollback reaching further back than
the history a user was granted is rejected rather than trimmed: nobody can
undo work that predates their own access, and a rollback with no bound at all
is refused outright. `prune`, which erases, stays granted to nobody. Restoring
is not destructive: it appends a change that undoes another, so what it
replaced stays in the history and can be restored again.
The backend's version endpoints are untouched and now have no caller. They are
marked deprecated with the condition for removing them, since until a document
has been replayed by `migrate_documents` they hold the only record of what it
looked like before it moved.
Also fixes the e2e helper that waited for the removed content endpoint, so it
never returned, and the three version tests that hung behind it.
Signed-off-by: Kevin Jahns <kevin.jahns@protonmail.com>
The collaboration server's activity and changeset routes are opened to
the browser, so a document's editing history can be read from where it
actually lives now. What a user may see of it is bounded to the moment
they were given access to the document: joining a document that has been
written for a year does not hand them the year.
That rule is not new. It is the one the version endpoints have always
applied - "only those created after the user got access to the document"
- and the date is the same one: the earliest access the user holds on
the document or on any of its ancestors, so sharing a folder shares its
subtree from that moment. It was computed twice in the backend,
differently, and exposed nowhere. It is now a single annotation,
user_access_since, that the version endpoints and the collaboration
server both read, the latter through the document detail response it
already fetches to authorize a connection.
The bound is applied server-side and silently: a client asks for
whatever range it likes and receives only its own share, so there is no
bound for it to get wrong and none it can widen. It is a stored date
rather than a wall-clock-relative one, which is what keeps it stable
across a websocket re-check, and it is never zero - the one value that
would also unlock a full-history connection.
A reader who reaches a document through its link alone holds no access
and so has no date to bound a history with. They get none, which is why
the backend has always refused them their versions. rollback and prune
stay refused to everyone: restoring a version is a separate decision.
Signed-off-by: Kevin Jahns <kevin.jahns@protonmail.com>
The upgrade between version 5 and next version 6 is not simple. Lot of
breaking changes are made and must be documented to ease the upgrade for
the community.
Yhub has a s3 persistent plugin, since now we didn't use it but we wanto
to give the possibility to use it optionnaly. To enable it set the
YHUB_S3_PERSISTENCE environment value to true and configure it.
We have some signature errors when using the minio client to list all
the versions of an existing document. To avoid this error we have
decided to use the S3 sdk and allow to configure the signature versino
the user wants. Also, the check on the document size has been removed,
there is no limitation on the document size.
In the redis section there were still hard coded values, we want to
allow the configurations of this settings. The last part will be the
persistence plugin.
The probes used before was using the only one public available endpoint.
This endpoint is the jwks endpoint but it is not an appropriated one.
For the readyness we check that it is possible to connect to postgres
and to redis. for the liveness we made a ping pong just checking the
http connection.
The new infra we have must be configured in the helm chart. This commit
all the missing templates to deploy yhub, it also automate the creation
of the private keys needed by all services.
Y_PROVIDER_API_KEY is no longer used in the codebase,
so we can remove it from the helm chart and the documentation.
We adapt the documentation to use the new JWT conversion
mechanism instead.
The CollaborationService was doing nothing since we started the
migration to yhub, all the code using it is now removed. Also the
`can-edit` endpoint and all the safeguard mechanism relying on the
presence of other users connected to the websocket will not be used
anymore, it will be possible to replace all of this with yhub, so all
this code is also removed.
Added
- ✨(frontend) export presenter slides as PDF #2487
Fixed
- 🐛(frontend) hide Leave in the doc menu when not logged in #2626
- 🐛(backend) allow to configure settings DATA_UPLOAD_MAX_MEMORY_SIZE
We removed previously whitenoise because it was not working with asgi
application. By removing it we also removed the way to serve the static
files in the application. There is an existing fork of whitenoise,
servestatic, that manage async application and we can use it to serve
static files.
Added:
- ✨(frontend) Add "Copy link to block" feature
- ✨(frontend) add word count to doc header toolbox
- ✨(frontend) add find and replace feature to the editor
Changed:
- ♿️(frontend) use anchor links for interlinking sub-documents
- ✨(frontend) reset side panel state between documents
- ♿️(frontend) announce search loading state for screen readers
- ♻️(frontend) change favorite to star
- 🚚(frontend) add doc move to doc options
- ♻️(frontend) unified menu
- ♿(frontend) hide decorative emojis in document titles from SR
- ♻️(frontend) save the doc with a keepalive request when
leaving the page
Fixed:
- 🐛(frontend) fix clipped formatting toolbar in new comment
composer
- 🐛(backend) fix duplicating a document that has no content
- 📄(frontend) allowed partially export when MIT
- 🐛(backend) manage async support for Docs custom middleware
Removed:
- 🔥(backend) remove whitenoise package
Added:
- ♿️(frontend) restore skip to content link after header redesign
- 🌐(i18n) rename cn_CN to zh_CN, add eo_PL and zh_TW locales
- ✨(backend) conditional email notification in server to server api
- ✨(backend) profile api using django-silk
Changed:
- ♿️(frontend) use semantic `<dl>` structure in document info card
- ⚡️(frontend) replace onboarding assets with webm and webp
- 💄(frontend) use the same highlight color for cells and moves
- ⚡️(backend) optimize media_auth endpoint
- 🚸(frontend) print from document options menu
Fixed:
- 🐛(frontend) refresh pins after document deletion and restoration
- 🐛(frontend) redirect homepage to login when homepage feat
is disabled
- 🐛(backend) ignore CSPs for API docs in development
- 🐛(frontend) export images embedded with a relative url
- 🐛(y-provider) fix sentry init
- 🐛(backend) handle object storage metadata keys case-insensitively
- 🐛(keycloak) fix database env variables in the self-hosting example
- 🐛(helm) show the database error while jobs wait for it to be ready
The migrate and createsuperuser jobs polled `manage.py check` with its
output sent to /dev/null, then printed "Database not ready" whatever
the reason. Any failure the check reports, a missing setting or a bad
credential for instance, looked like a database that had not started
yet, and the job looped forever without a single clue in its logs.
The check output is now captured and printed on each failed attempt,
along with the attempt number, so the reason the job is still waiting
is readable with kubectl logs. The retry behaviour is unchanged.
Applied to both jobs in the chart values and in the dev and feature
helmfile environments. The chart README table is regenerated
accordingly.
Signed-off-by: risk-alt <aldu6974@gmail.com>
Changed:
- ♻️(backend) reset collaboration connection in cascade for all children
Fixed:
- 💄(frontend) fix some UI/UX in the left panel
- 🐛(frontend) fix tree dnd firefox
Added:
- ✨(y-provider) preserve callouts, PDFs, page breaks, interlinking
links and commented text on HTML/markdown export
- ✨(frontend) add a user menu
- ✨(frontend) new header and responsive harmonization
- ✨(backend) add management command to reset a Document
Changed:
- ♿️(frontend) hide mobile left panel from screen readers when collapsed
- ♿️(frontend) enable blocknote heading ids for toc anchors
- ♿️(frontend) focus export modal on format select
- ♿️(frontend) configurable legal submenu in HelpMenu, remove Crisp
- ♻️(frontend) new create button for docs
- ♿️(frontend) align search modal field label with placeholder
- 🚚(frontend) move Waffle to bottom left
- ♿️(frontend) remove redundant aria-label on table of contents links
- ♻️(core) fix typo in settings COLLABORATION_WS_NOT_CONNECTED_READY_ONLY
- ♻️(backend) scope document search by document id instead of path
Fixed:
- 🐛(backend) prevent owner from leaving a soft-deleted document
- 🐛(frontend) fix removed item in the tree
- 🐛(frontend) fix service worker causing reload on tab focus
- 🐛(backend) update restore ability for inherited deletion
- 🔧(dev) make the dev stack domain-agnostic
- 🐛(frontend) stop force index redirect when delete doc
- 🐛(frontend) fix CTA on move modal on mobile
Removed:
- 🔥(backend) remove unused default authentication backend
The settings COLLABORATION_WS_NOT_CONNECTED_READY_ONLY contains a typo.
We don't want to see READY_ONLY but READ_ONLY. This commit fix the typo
and change everywhere in the codebase the settings name.
In the yprovider deployment we now need to mount the cacert containing
the certificates generated by mkcert. Then, we have to declare the
environment variable NODE_EXTRA_CA_CERTS with the path where this cacert
is mounted. With this new config we don't have tls issue anymore.
Added
- ✨(backend) add limit on distinct reactions per comment #1978
- ✨(frontend) leave a document #2410
- ✨(frontend) add top parent on sub docs search #1952
- ✨(frontend) unauthenticated users can search #2407
- ✨(backend) specific user delete method to delete its relations #2437
Changed
- 👷(CI) remove test-e2e-other-browser job #2404
- ♿️(frontend) use heading element for pinned documents section title #2380
- ♿️(frontend) use anchor links for table of contents entries #2390
- ♿️(frontend) improve presenter mode screen reader and keyboard support #2383
- ♿️(frontend) link export modal name to its heading #2422
Fixed
- 🐛(frontend) overlap of block menu dropdown #2406
- ⚡️(backend) fix N+1 queries when serializing thread comments #2415
Changed:
- 💄(frontend) display emoji button on hover
Fixed:
- 🐛(backend) close thread DB connections to fix test teardown
OperationalError
- 🐛(frontend) fix crash when orphaned threads
- 🐛(backend) order trashbin response by most recently deleted
- 🐛(backend) stream document content with an async
iterator under ASGI
- 🐛(frontend) fix long titles in table of content
Added:
- ✨(backend) support creating subdoc from file
- ✨(frontend) comment side panel
- ✨(buildpack) add PaaS deployment support, tested with Scalingo
- 🔧(backend) allow configuring settings
OIDC_OP_USER_ENDPOINT_FORMAT
- ⚡️(helm) create a dedicated svc and deployment for
yprovider converter
- ✨(backend) allow to leave a document
- ✨(frontend) add the presenter mode
- 📈(backend) create a utils to capture event with posthog
- 🔧(backend) new setting DOCUMENT_ALL_ENDPOINT_ENABLED
Changed:
- ♻️(backend) allow global search in sub documents
- ✨(backend) add a breadcrumb in the search response
- ♻️(frontend) move doc action buttons to fix toolbar
- ♿️(frontend) add aria-hidden to decorative avatar SVGs
in share modal
- 🏗️(frontend) move comments to its own folder feature
Fixed:
- 🐛(docs) run migration 0027 without superuser role
- 🐛(backend) prevent admins/owners from overwriting other
users comments
- 🐛(y-provider) return empty output when converting empty
Yjs document
- 🐛(backend) use computed_link_reach in
handle_onboarding_document
- 🐛(frontend) fix toolbar blocknote hidden
- 🐛(frontend) fix application crashes when using
GTranslate and zoom
- 🐛(frontend) fix emoji pdf not matching
- 🐛(backend) fix UnorderedObjectListWarning for
DocumentAskForAccess viewset
We want to isolate the converter service built in the yprovider server in
order to not mix conversion and collaboration together. We decided to
create a dedicated service and deployment, by default they use the
values defined in yProvider values but all can be overridden in
yProvider.converter.
Also, by default these new service and deployment are not enabled, they
must be explictly enabled and configured. If not it works like now with
only one service and deployment for the yProvider server.
Added
- ⚡️(frontend) add skeleton on content loading #2254
- ⚡️(frontend) close websocket connection when user change tab #2264
Changed
- 🏗️(core) migrate from pip to uv
Fixed
- 🩺(project) reload app if front and back unsync #2276
- 🐛(frontend) fix patch and comments #2273
- 🐛(frontend) interlinking are exported correctly in print mode #2269
- 💬(frontend) add missing link in onboarding description #2233
- 🐛(frontend) sanitize pasted and dropped content in document title #2210
- 🐛(frontend) Emoji menu doesn't display above comment box #2229
- 🐛(frontend) Block menu doesn't stay open on 1st line #2229
- 🐛(frontend) The "+" on the first line of a new doc doesn't work #2229
- 🐛(backend) manage race condition between GET and PATCH content #2271
- 🐛(backend) replace document creation table locks with retry strategy #2274
Security
- 🔒️(frontend) sanitize color during collaboration #2270
Added
- ✨(backend) create a dedicated endpoint to update document content
- ⚡️(backend) stream s3 file content with a dedicated endpoint
- ✨(backend) allow to use new ai feature using mistral sdk
Changed
- ♻️(backend) rename documents content endpoint in `formatted-content` (BC)
- 🚸(frontend) show Crisp from the help menu #2222
- ♿️(frontend) structure correctly 5xx error alerts #2128
- ♿️(frontend) make doc search result labels uniquely identifiable #2212
- ⬆️(backend) upgrade docspec to v3.0.x and adapt converter API #2220
- ✨(backend) make forward auth request uri header configurable #2241
- ♿️(frontend) fix sidebar resize handle for screen readers #2122
Fixed
- 🚸(frontend) redirect on current url tab after 401 #2197
- 🐛(frontend) abort check media status unmount #2194
- ✨(backend) order pinned documents by last updated at #2028
- 🐛(frontend) fix app shallow reload #2231
- 🐛(frontend) fix interlinking modal clipping #2213
- 🛂(frontend) fix cannot manage member on small screen #2226
- 🐛(backend) load jwks url when OIDC_RS_PRIVATE_KEY_STR is set
- 🐛(backend) Prevent moving document to its own descendant or self #2208
- 🐛(backend) return 400 when restoring a non-deleted document #2225
When developping using the tilt environment, we need to allow the domain
docs.127.0.0.1.nip.io in the next configuration.
Also the the customization is removed as it is not maintained.
Summary
- Bump docspec Docker image from `2.6.3` to `3.0.0` and adapt
`DocSpecConverter` to the new API (raw body upload with explicit
`Content-Type`/`Accept` headers instead of multipart form)
Important
**The Docker image (`ghcr.io/docspecio/api:3.0.0`) must be updated
alongside the code changes.** The new request format is incompatible
with v2.x — deploying only the code without updating the image (or vice
versa) will break document conversion.
The django application is running in ASGI in production, to have the
same behavior we run the development container with uvicorn too with
options more appropriated for a development evironment.
Added:
- 🚸(frontend) allow opening "@page" links with
ctrl/command/middle-mouse click
- ✅ E2E - Any instance friendly
Changed:
- ♻️(backend) do not paginate threads list response
- 💄(frontend) Use StyledLink for sub doc tree
Fixed:
- 🐛(frontend) Fix drop cursor creating columns
- 🐛 Fixed side effects between comments and versioning
By default, Node.js has a memory limit of
around 512MB, which can lead to out-of-memory
errors when processing large documents.
This commit increases the memory limit to
2GB for the y-provider server, allowing
it to handle larger documents without crashing.
Changed
- 💫(frontend) fix the help button to the bottom in tree #2073
- ♿️(frontend) improve version history list accessibility #2033
- ♿️(frontend) fix more options menu feedback for screen readers #2071
- ♿(frontend) focus skip link on headings and skip grid dropzone #1983
- ♿️(frontend) fix search modal accessibility issues #2054
- ♿️(frontend) add sr-only format to export download button #2088
- ♿️(frontend) announce formatting shortcuts for screen readers #2070
- ✨(frontend) add markdown copy icon for Copy as Markdown option #2096
- ♻️(backend) skip saving in database a document when payload is empty #2062
Fixed
- ♿️(frontend) fix aria-labels for table of contents #2065
- 🐛(backend) allow using search endpoint without refresh token enabled #2097
The annotation in the values.yaml have not been updated since a while.
This commit update them and generate the readme using the generate-readme.sh
script
We want the possibility to configure specific environment variables on
backend and celery deployment. Most of them are common but in the case
of the newly added settings DB_PSYCOPG_POOL_MIN_SIZE we want to
configure ot only on the backend deployment, not on the celery or with a
different value.
Added:
- ✨(backend) add a is_first_connection flag to the User model
- ✨(frontend) add onboarding modal with help menu button
Changed:
- ♿(frontend) localize LaGaufre label fallback in Docs
- ✨(backend) add a migration cleaning on-boarding
document accesses
- ⬆️(frontend) upgrade Next.js to v16
- ♿️(frontend) fix aria-label and landmark on document
banner state
- 🌐(i18n) add "new window" translation key for waffle
aria-label
Fixed:
- 🐛(backend) create a link_trace record for on-boarding
documents
- 🐛(backend) manage race condition when creating sandbox
document
- 🐛(frontend) fix flickering left panel
- ♿️(frontend) improve doc tree keyboard navigation
Anthony LC