Added
- 👷(ci) lint, typecheck, build and test the collaboration server (yhub) #2757
- 👷(ci) check the load-test tooling: swarm, canary, k6 and dashboards #2757
- 👷(ci) cancel the superseded runs of a pull request #2757
- ✨(backend) add mention endpoint with cooldown-limited email
notification #2447
- 🚩(setting) add feature flag on Duplicate with Children #2721
- 💄(frontend) redesign 404 error standalone page #2696
- 💄(frontend) redesign 403 access denied page #2720
- ✨(frontend) duplicate with subdocuments #2584
- ✨(frontend) add markdown download option #2608
- ✨(frontend) turn pasted doc links into interlinks #2713
- ✨(helm) allow disallowing search engine indexing per instance #2694
- ✨(backend) expose the attachment max size in the config endpoint #2577
- ✨(frontend) warn before uploading an attachment over the size limit #2577
- ✨(loadtest) add a websocket load generator for the collaboration
server #2556
- ✨(loadtest) add k6 scenarios for the page-open sequence and the heavy
endpoints #2556
- ✨(loadtest) add browser canaries measuring what a user feels under
load #2556
- ✨(loadtest) add the grafana dashboards of the load-test campaign, valkey
included #2556
- 📝(documentation) add the load-testing guide #2556
- 🔧(helm) run grafana with those dashboards in the dev cluster, in place of
the prometheus console #2556
- ✨(backend) measure the calls to yhub and to the converters, the database
pool and the celery queue #2556
- ✨(backend) add a `LoadTest` configuration and its `loadtest` application,
minting sessions for load tests #2556
- ✨(collaboration) add opt-in prometheus metrics to yhub, server and worker,
protected by a bearer token #2556
- ✨(collaboration) report yhub errors to sentry, configured through
`SENTRY_*` #2556
- ✨(backend) add opt-in prometheus metrics on `/metrics`, protected by a
bearer token #2556
- ✨(helm) add a dedicated ingress for the prometheus metrics of the backend
and of yhub #2556
- ✨(helm) add a ServiceMonitor and a PodMonitor per component whose metrics
are enabled, and `backend.metrics.enabled` #2556
- 🔧(helm) scrape the metrics of the dev cluster with a trimmed
kube-prometheus-stack and the ServiceMonitors of the chart #2556
- 🔧(helm) serve the example console of django-prometheus on the dev
Prometheus #2556
- ✨(backend) add a service generating cached RS256 JWT tokens #2556
- ✨(backend) publish the JWT public key on a JWKS endpoint #2556
- 🔧(dev) generate the JWT signing key when bootstrapping the dev stack #2556
- ✨(collaboration) add an admin reset-connections endpoint on yhub #2556
- ✨(collaboration) add a create-ydoc endpoint on yhub #2556
- ✨(collaboration) soft-migrate legacy S3 documents into yhub #2556
- ✨(collaboration) replay legacy s3 version history into yhub #2556
- ✨(backend) add a service to call the yhub REST API #2556
- ✨(backend) call YHubService to seed initial document content #2556
- ✨(collaboration) add a get-ydoc endpoint on yhub #2556
- ✨(backend) duplicate a document through the collaboration server #2556
- ✨(backend) serve `documents/{id}/formatted-content/` from yhub #2556
- ✨(collaboration) notify the backend when the worker persists new
content #2556
- ✨(collaboration) let a user read the document's editing history #2556
- ✨(frontend) fall back to http polling when the websocket cannot be
opened. #2556
- 🔧(collaboration) make the version-history granularity configurable through
`COLLABORATION_VERSION_GRANULARITY_MS` #2556
- ✨(frontend) keep a local copy of documents, so they open and stay editable
offline #2556
Changed
- 🛂(backend) let users allowed to comment list each other's accesses #2447
- ♻️(collaboration) migrate the collaboration server from hocuspocus to
yhub #2556
- 💥(y-provider) y-provider becomes converter-only #2556
- 💥(backend) move the resource server JWKS from `/api/{version}/jwks` to
`/external_api/{version}/jwks` #2556
- 🔧(collaboration) adapt docker stack for development purpose #2556
- 🔧(helm) run a valkey for the backend and one for yhub in dev and
feature #2556
- 💄(frontend) redesign 401 error standalone page #2716
Fixed
- 🐛(documentation) fix minio port and keycloak realm in the k8s guide #2751
- 🐛(collaboration) bound the calls yhub makes to the backend with a timeout,
`YHUB_BACKEND_REQUEST_TIMEOUT_MS`, 5s by default #2753
- 🐛(backend) compensate document duplication failures #2755
- 🐛(frontend) open search results in a new tab with ctrl/cmd+click #2719
- 🐛(docker) pull minio images from pgsty #2739
- 🐛(backend) retry the duplicate of a document on a tree path collision #2726
- 🐛(frontend) keep caption and alignment when replacing an image #2730
- 🐛(backend) reset the collaboration connections on every access change #2725
- ⚡️(backend) coalesce and pace the resets of collaboration connections #2725
- 🐛(frontend) clear callout background on Backspace #2052
- 🐛(export) keep image aspect ratio in PDF columns #2670
- 🐛(frontend) fix redirect after deleting a document #2706
- 🐛(frontend) keep documents draggable with a mouse when zoomed #2727
- 📝(docs) fix markdown typo in installation README #2766
- 🐛(i18n) export locale correctly #2750
- 🐛(frontend) keep login and logout out of the service worker cache #2771
- 🐛(frontend) reduce PostHog volume from web vitals and opt_in spam #2701
- 🐛(frontend) stop reconnecting to the websocket based on the status
code #2556
- 🐛(frontend) stop the service worker from caching the collaboration server's
rest api #2556
Removed
- 🔥(backend) remove the unused `CollaborationService` #2556
- 💥(backend) remove the `documents/{id}/can-edit/` endpoint #2556
- 💥(backend) remove the `documents/{id}/content/` endpoint #2556
- 🔥(backend) remove the document version endpoints. #2556
- 🔥(backend) remove `Document.content` #2556
Security
- 🔒️(collaboration) stop read-only users from sharing their cursor #2556
Every other package in this repository is installed
with yarn; yhub-server was the only one on npm,
with its own package-lock.json.
To ensure consistency it now uses yarn like the rest
of the project.
package-lock.json is replaced by yarn.lock, a packageManager
field is added, and the Dockerfile, CI workflow, Makefile,
helm init-db job and the documentation move from
`npm ci` / `npm run init-db` to
`yarn install --frozen-lockfile` / `yarn init-db`.
The content is not managed anymore in the Document object nor the django
application. The last piece using it was the versioning API and it has
been deleted previously. There is no more reason to keep code related of
the content in the backend application. The only remaining code is the
`file_key` property in the model. It is kept as a safe guard use by the
clean_document management command. If the file is kept on the bucket,
then the document will be reseed with it the first time the document
will be reopen.
The S3 persistence plugin records the version id of the object it wrote
and names that version when it deletes it. On a versioned bucket - what
a deployment runs - a delete that names no version deletes nothing: it
writes a delete marker and keeps every version underneath. Each
compaction supersedes the blobs of the one before, so what was kept was
every version of every document ever written, a document someone asked
to erase included, still readable by anyone who can list versions. On
AWS this needs s3:DeleteObjectVersion, which a policy granting
s3:DeleteObject alone does not cover.
Blobs are written to the bucket for every branch of a document.
YHUB_S3_PERSISTENCE now governs only whether new blobs are written
there. The plugin itself is attached whenever the YHUB_S3_* settings
name a bucket, on or off, because reading is the half that must never
be taken away: a row pointing at an object is unreadable without the
plugin that wrote it, and yhub reports such a version as having no
content rather than as an error. Turning the toggle off stops the
writing and leaves the reading alone; it is the settings, not the
toggle, that a deployment whose bucket holds anything must keep. Half a
configuration is a startup error naming what is missing, as before.
The dev stack keeps the toggle off and creates its bucket versioned, so
flipping it on exercises what a deployment runs rather than a simpler
case. Its createbuckets job needed fixing to do so: the folded yaml
block joins its lines with a space, so the trailing backslashes reached
the shell as an escaped space glued to the next word and everything
past the first && silently did nothing - the media bucket never had
versioning enabled either.
Signed-off-by: Kevin Jahns <kevin.jahns@protonmail.com>
The upgrade between version 5 and next version 6 is not simple. Lot of
breaking changes are made and must be documented to ease the upgrade for
the community.
notify the backend when the worker persists new content for
a document, so the lists ordered by `updated_at` follow the edits made on the
collaboration server. The backend serves it on
`POST /api/v1.0/documents/{id}/content-updated/`, authenticated with a short
lived RS256 JWT the collaboration server signs (`aud: "docs-backend"`) and
the backend verifies against the JWKS the collaboration server publishes on
`/collaboration/jwks/v1` — the mirror of the admin token the backend signs to
call it, so no long lived secret is shared and either side can roll its key
on its own
Both its PATCH and its GET: the content of a document is saved and
served by the collaboration server. The `content_patch` and
`content_retrieve` abilities go with it.
The yhub service will need our public key in order to validate the jwt
token we will used. We choose to expose a jwks endpoint as it is a
standard wat to do this.
Added
- ✨(backend) create a dedicated endpoint to update document content
- ⚡️(backend) stream s3 file content with a dedicated endpoint
- ✨(backend) allow to use new ai feature using mistral sdk
Changed
- ♻️(backend) rename documents content endpoint in `formatted-content` (BC)
- 🚸(frontend) show Crisp from the help menu #2222
- ♿️(frontend) structure correctly 5xx error alerts #2128
- ♿️(frontend) make doc search result labels uniquely identifiable #2212
- ⬆️(backend) upgrade docspec to v3.0.x and adapt converter API #2220
- ✨(backend) make forward auth request uri header configurable #2241
- ♿️(frontend) fix sidebar resize handle for screen readers #2122
Fixed
- 🚸(frontend) redirect on current url tab after 401 #2197
- 🐛(frontend) abort check media status unmount #2194
- ✨(backend) order pinned documents by last updated at #2028
- 🐛(frontend) fix app shallow reload #2231
- 🐛(frontend) fix interlinking modal clipping #2213
- 🛂(frontend) fix cannot manage member on small screen #2226
- 🐛(backend) load jwks url when OIDC_RS_PRIVATE_KEY_STR is set
- 🐛(backend) Prevent moving document to its own descendant or self #2208
- 🐛(backend) return 400 when restoring a non-deleted document #2225
We also want to use the mistral sdk with the legacy AI feature when this
one is configured with the settings. In order to separate bot feature,
they now live in their own module.
Added:
- ✨(frontend) integrate new Blocknote AI feature
- 👷(docker) add arm64 platform support for image builds
- ✨(tracking) add UTM parameters to shared document links
- ✨(frontend) add floating bar with leftpanel collapse button
- ✨(frontend) Can print a doc
- ✨(backend) manage reconciliation requests for user accounts
- 👷(CI) add GHCR workflow for forked repo testing
- ✨(frontend) Move doc modal
- ⚡️(backend) remove content from Document serializer when asked
- ✨(backend) allow the duplication of subpages
- ✨(backend) Onboarding docs for new users
- 🩺(trivy) add trivyignore file and add minimatch CVE
- 🚩 Add feature flags for the AI feature
Changed:
- ♿️(frontend) prevent dates from being focusable
- ♿️(frontend) Focus main container after navigation
- 💄(frontend) align colors and logo with ui-kit v2
- 🚸(backend) sort user search results by proximity
with the active user
- 🚸(oidc) ignore case when fallback on email
- ⚡️(CI) optimize Docker Hub workflow
Fixed:
- 🐛(frontend) fix broadcast store sync
- 🐛(helm) use celery resources instead of backend resources
- 🐛(helm) reverse liveness and readiness for backend deployment
- 🐛(y-provider) use CONVERSION_FILE_MAX_SIZE settings
- 🐛(frontend) fix callout block spacing for old browsers
Added:
- ✨ Add comments feature to the editor
- ✨(backend) Comments on text editor
- ✨(frontend) link to create new doc
Changed:
- ⚡️(sw) stop to cache external resources likes videos
- 💥(frontend) upgrade to ui-kit v2
- ⚡️(frontend) improve perf on upload and table of contents
- ♿(frontend) improve accessibility:
- ♿(frontend) improve share modal button accessibility
- ♿(frontend) improve screen reader support in DocShare modal
Fixed:
- 🐛(frontend) fix toolbar not activated when reader
- 🐛(frontend) preserve left panel width on window resize
- 🐛(frontend) prevent duplicate as first character in title
We want to be able to define whether AI features are available to
anonymous users who gained editor access on a document, or if we
demand that they be authenticated or even if we demand that they
gained their editor access via a specific document access.
Being authenticated is now the default value. This will change the
default behavior on your existing instance (see UPGRADE.md)
Manuel Raynaud