Cover both paths off the legacy Django store end to end: the lazy seed on
first access, and the migrate endpoint replaying every S3 version. The tests
need no database — the admin JWT short-circuits document authorization, so a
fixture is an S3 object on a random uuid — and read the timeline through
yhub 0.5.0's `Accept: application/json`, which spares python a lib0 decoder.
CI grows a valkey service and starts a collaboration server alongside the
backend test job; the tests skip themselves when nothing answers on the new
COLLABORATION_API_URL setting, so `make test` without the dev stack still
passes.
Writing them turned up three things worth fixing in the server.
Backend reads now seed too. getAccessType short-circuited on the admin token
before reaching the legacy store, so a server-side read of an unmigrated
document answered with an empty one, and a create-ydoc against it would have
written a second lineage beside the content the first user access was about
to seed in.
Seeding no longer decides access; the backend's answer alone does. A legacy
object that cannot be migrated — it does not decode, or it exceeds the size
we load — opens as a new document instead of denying, since no retry can fix
it and refusing would leave the document unopenable by anyone. The cause is
logged once per attempt with the bucket, key and stack, and every later access
logs that it admitted a caller without migrating.
That made the failure classifier dangerous, so it is inverted. It was an
allowlist of retryable errors — eight socket errnos — which left every way S3
can refuse (AccessDenied on a rotated key, NoSuchBucket, a region redirect)
counting as "this object is unusable". Denying, that was survivable; opening
empty, one misscoped credential would fork every document touched during the
window. Now only a failure raised while interpreting bytes we already hold is
permanent, marked at the throw site, and everything else answers a retryable
503. Guessing wrong that way costs a retry; the other way costs the document.
The admin seed is also fenced to the org and to main, like the user path
above it. The legacy store is branchless — {docid}/file is main — and the
bookkeeping is per document, so seeding ?branch=draft would have written
main's content into an orphan room and left the real one permanently empty.
Signed-off-by: Kevin Jahns <kevin.jahns@protonmail.com>
Add POST /collaboration/migrate/v1/docs/{id}, which replays every S3 version
of a document's legacy `{id}/file` object into one gc:false Yjs document and
stores it as a single row at clock 0, crediting each version with its own S3
timestamp. Nothing existing is deleted and nothing goes on the stream, so the
next compaction merges that row like any other. The clock-0 insert is ON
CONFLICT DO NOTHING and migrated ids are kept in a valkey set, so the endpoint
is idempotent without a lock. The activity api then reports the same timeline
as the backend's /documents/{id}/versions/, instead of the single
migration-time change the lazy soft migration leaves behind.
That lazy seed now writes no insertAt/deleteAt. Persisted contentmaps are
merged rather than de-duplicated, so a seed timestamp would survive next to
the real per-version one on the same ids and the activity api would report
whichever the unordered row scan put last. A seed is not an editing event and
has no honest time to report.
Upgrade yhub to 0.5.0, where error codes encode retry semantics (4xx
permanent, 5xx and 429 retryable) and auth plugins may throw apiError(503). A
temporarily unreachable Django backend, JWKS endpoint or legacy S3 store is
now reported as 503 rather than denied like a permission failure, so clients
retry instead of giving up.
The legacy-store code moves out of server.js into migration.js, with the
shared *_FILE secret helper in env.js.
Signed-off-by: Kevin Jahns <kevin.jahns@protonmail.com>
yhub verified Django's RS256 admin JWT without checking "aud", so the
y-converter token Django hands to the converter process was replayable
here — and admin: true short-circuits getAccessType to "rw" on every
document, plus the backend-internal reset-connections purpose and the
X-User-Id attribution override. Require aud: "yhub", as y-provider
already does for its own audience. Nothing in the backend calls yhub's
admin endpoints yet, so no caller is affected.
Signed-off-by: Kevin Jahns <kevin.jahns@protonmail.com>
With SOFT_MIGRATION=true, the first access to a document yhub does not
know yet fetches the legacy snapshot from Django's S3 media bucket
({id}/file, UTF-8 base64 of a raw Yjs update), seeds the room through
the compute pool - attributed to "system" with a migration=s3 custom
attribution - and only then admits the connection, so the initial sync
always includes the seed. Now that the frontend no longer bootstraps
rooms client-side (content GET/PATCH removal), this is the only path
that brings legacy content into yhub; keep the flag on until a batch
backfill has migrated the full corpus.
A missing S3 object is the brand-new-document case and yields an empty
room; every real failure fails closed (opaque 401, y-websocket retries
with backoff). Existence is probed postgres-first (bare SELECT, then
the valkey stream, then the SELECT again to close the compaction
race). Guard rails: a per-docid verdict cache (poison objects cannot
sustain an S3 retry storm, transient errors expire in 15s, per-replica
seed backpressure denies once without caching), in-flight dedup, a
token-owned cross-replica valkey lock released by compare-and-delete,
a 10s S3 fetch timeout that also destroys a late-arriving response
stream, and the same 10MiB decoded cap as create-ydoc. Concurrent
seeds stay correct regardless: the frozen snapshots share one Yjs
lineage, so duplicates merge as CRDT no-ops.
Also reject non-lowercase docids (Django serializes UUIDs lowercase; a
case variant would open a parallel room and miss its S3 object) and
refuse to boot when AWS_S3_ENDPOINT_URL carries a path the minio
client cannot address. On AWS the read-only credentials must include
s3:ListBucket so a missing object surfaces as NoSuchKey rather than
AccessDenied - see the README for the full guarantees and ops notes.
Signed-off-by: Kevin Jahns <kevin.jahns@protonmail.com>
The Playwright MCP browser session writes snapshots and console logs
into .playwright-mcp/ at the repository root while driving the app
during development — keep them out of version control.
Signed-off-by: Kevin Jahns <kevin.jahns@protonmail.com>
We remove the code related to the content GET and
PATCH endpoints, as they are no longer used in the
codebase. The yhub server will handle the content
management directly, providing the content and
managing the updates. This change simplifies the code
and reduces the complexity of the frontend application.
We will need to reimplement the saving mechanism
in the service worker when we are offline.
Let's wait that the service is fully developed
on the yhub side before we implement this feature.
We will not block anymore the users not connected
to the collaboration server from editing the document,
we will have an HTTP fallback instead, so we can
remove the "can-edit" mechanism and the related code.
Address the findings of an adversarial review of the new endpoint:
- Only the backend admin token may attribute content to another user via
the X-User-Id header. The endpoint uses the default access purpose, so
any editor with update ability can call it — honoring the header for
them would let an editor forge the attribution history of the first
revision (the websocket path likewise stamps the server-side
identity). Regular callers now always author as themselves; verified:
an editor session posting X-User-Id gets its own userid stamped.
- Reject non-main ?branch= requests (400). Cookie users are main-only
via getAccessType, but the admin token bypasses it and could seed an
orphan (org, docid, branch) room no user-facing path reads — while
dodging the branch-scoped 409 existence check.
- Correct the concurrent-create comment: two racing creates merge as
independently generated updates (fresh clientIDs), so the seeded
content appears twice — user-visible duplication, not merely a
doubly-attributed revision. Still accepted (Django creates each doc
once and a duplicated seed is user-fixable), but the tradeoff is now
stated accurately.
Signed-off-by: Kevin Jahns <kevin.jahns@protonmail.com>
Python cannot call yhub's built-in PATCH ydoc api because its body must
be lib0-any encoded - a lib0-specific binary framing with no
implementation outside javascript. The new endpoint
POST /collaboration/create-ydoc/v1/{org}/{docid} accepts the raw binary
Yjs update (pycrdt get_update() / Y.encodeStateAsUpdate output) as
application/octet-stream, so Django can seed a document's initial state
with a plain requests.post(url, data=raw_bytes) - needed by the
server-side creation flows (file import, create-for-owner, duplication,
template instantiation) whose yhub rooms currently stay empty until the
first browser connects.
Strict create semantics: 409 when the room already has content
(checked via getDoc, covering persisted state and uncompacted stream
messages; yhub has no atomic create, concurrent creates merge via CRDT
and never corrupt). The initial content is attributed to the optional
X-User-Id header, else to the caller's identity. Access uses the
default purpose, i.e. standard document write access like the built-in
ydoc routes: the admin JWT, or a user session with update ability.
Malformed updates map to 400 (the compute worker rejects them and the
pool replaces the thread), empty updates to 400, bodies over 10MiB to
413.
Gotcha worth noting: req.bytes() resolves to a Node Buffer, but yhub's
compute-task schema validates with lib0's exact-constructor Uint8Array
check, so the body is re-viewed as a plain Uint8Array before it is
handed to the compute pool.
Signed-off-by: Kevin Jahns <kevin.jahns@protonmail.com>
Y_PROVIDER_API_KEY is no longer used in the codebase,
so we can remove it from the helm chart and the documentation.
We adapt the documentation to use the new JWT conversion
mechanism instead.
The /api/convert route no longer accepts the Y_PROVIDER_API_KEY shared
secret. It now verifies the admin JWT signed by Django against the
JWKS published on its /api/v1.0/jwks endpoint.
Add POST /collaboration/reset-connections/v1/{org}/{docid} (optional
X-User-Id header) to yhub-server. It distributes yhub recheckAuth: every
server re-runs the access check per matching connection and closes only
those whose access actually changed (close code 4401), so unaffected
clients see no reconnect churn. The endpoint authenticates with the
RS256 admin JWT issued by JWTService, verified against the backend JWKS
(new jose dependency); the admin token acts as the "system" user and is
the only principal granted the reset-connections access purpose. The
backend does not trigger it on permission changes yet - that wiring
comes separately, now that CollaborationService is gone.
yhub is upgraded to 0.4.0 and serves every route under the
/collaboration/ prefix (server.apiPrefix): the websocket moves to
/collaboration/ws/v1/docs, and the built-in document apis are meant to
be publicly exposed alongside it, with reset-connections as the one
backend-internal exception.
Also harden websocket auth: fail closed when the backend errors (only a
genuine 401/403 falls back to the anonymous identity, so a signed-in
editor can never hide from a targeted recheck under an anon userid) and
tolerate small clock skew when verifying the cached admin token.
Signed-off-by: Kevin Jahns <kevin.jahns@protonmail.com>
The CollaborationService was doing nothing since we started the
migration to yhub, all the code using it is now removed. Also the
`can-edit` endpoint and all the safeguard mechanism relying on the
presence of other users connected to the websocket will not be used
anymore, it will be possible to replace all of this with yhub, so all
this code is also removed.
For now the only token we will need is ont with the admin claim set to
True. To not repeat the creation of this token again and again, we
created a dedicated method to issue this token in the JWTService class.
The yhub service will need our public key in order to validate the jwt
token we will used. We choose to expose a jwks endpoint as it is a
standard wat to do this.
We can now choose to duplicate documents along with
their subdocuments. A confirmation modal will now
appear to let the user choose whether to include
subdocuments in the duplication, only if the document
has subdocuments.
A toast notification will inform the user and
give the possibility to undo the duplication if
needed.
The duplicate endpoint used to respond with only `{"id": ...}`. It now
returns the complete duplicated document representation, consistent
with the other document detail endpoints, so the frontend doesn't have
to make a follow-up request to get the new document's data.
This required setting `is_favorite` explicitly on the duplicated
document before serializing it: it is normally set by the
`annotate_is_favorite` queryset method, which the newly created
document never goes through. Being a read-only serializer field, it
was silently dropped from the response instead of raising an error. A
document can't be a favorite right after being created, so it is set
to `False` directly.
Update lightly the ui to match the new design mockups,
to fit with what was proposed for the duplicate subdocs
feature.
It updates:
- modal closing position
- checkbox style
- toast style
- create a ButtonLink component, a wrapper above
the ui-kit component to fit the Next.js routing system.
We fixed the redirect behavior after deleting a
document to ensure the user stays on the correct page
after deleting a document.
We added a test to avoid regression in the redirect
behavior.
useToastProvider()'s `toast` function is recreated by the provider
every time any toast is shown anywhere in the app. useUploadFile
listed it as a dependency of `uploadFile`, which is itself a
dependency of useCreateBlockNote, so an unrelated toast (e.g. "Link
copied!") recreated `uploadFile` and made BlockNote rebuild the whole
editor instance, resetting the scroll position to the top of the
document.
Added useToast, a drop-in wrapper around useToastProvider that keeps
`toast`'s identity stable behind a ref, and switched every call site
in the app to use it instead of the provider's hook directly.
Dropping a file larger than the allowed size showed a bare "unknown
error" in the editor. The proxy in front of the API cuts the request
and answers a 413 with an HTML body, so errorCauses threw while
parsing it as JSON and no cause ever reached the error panel.
The size limit the backend already enforces is now exposed by the
config endpoint, and the editor checks the file against it before
sending anything, with the same toast wording the document import
uses. errorCauses no longer throws on a body it cannot parse, and a
413 without a usable cause falls back to an explicit message, which
covers the instances whose proxy limit is lower than the application
one.
The size formatting duplicated in the import hook moved to a shared
util.
Signed-off-by: risk-alt <aldu6974@gmail.com>
$opt_in fired on every reconnect since opt_in_capturing() captures an
event by default, even though this call just re-enables tracking after
a network drop and isn't a real consent action. Also disable web
vitals capture, which fed a single low-value insight.
Add an optional ConfigMap-backed robots.txt mounted into the frontend
static export (frontend.robotsTxt.enabled/content), following the same
pattern as backend.themeCustomization, and enable it with a full
Disallow on the feature review-app environment.
Added:
- 🔧(backend) fine tune redis cache options
- ✨(frontend) make the full last-update date available
- 💄(frontend) redesign email confirmation standalone page
Changed:
- ⬆️(backend) upgrade celery to version 5.6.3
- ⚡️(backend) stop using LEFT(value, LENGTH(path)) in sql queries
- 🚚(project) switch docspec image to ghcr.io/docspec/api
- 🚚(global) move favorite documents API endpoint
to `/documents/favorites/`
Fixed:
- 🐛(backend) skip session creation for the liveness probe
- 🐛(frontend) preserve page titles when adding an emoji
- 🐛(frontend) scroll to the linked block in read-only documents
- 🐛(frontend) hide the selection highlight on presenter images
- 🐛(y-provider) prevent process crash on malformed websocket frames
- 🐛(frontend) keep commented text sharp when printing to PDF
- 🐛(docker) pull minio images from quay.io
- ♿️(frontend) restore presenter focus trapping after share links
- 🐛(frontend) export any raster image supported by the browser to a PDF
BlockNoteReader (read-only docs) doesn't register the find & replace
tiptap extension. If the panel is left open while the active editor
switches to it -- e.g. a collaborative doc turning read-only after a
WebSocket disconnect -- calling into its commands threw
"commands.clearSearch is not a function".
useFindReplace now exposes whether the current editor actually
supports find & replace, and FindReplace closes the panel and renders
nothing when it doesn't, instead of crashing.
When a WebSocket connection is rejected for a missing Origin or cookies,
or hits a route with no matching handler, express-ws still completes the
upgrade handshake and only closes the socket afterwards. The `ws`
library keeps parsing incoming frames during the close handshake, which
can take up to 30 seconds, and no 'error' listener was attached to these
sockets in the meantime. A single malformed frame (e.g. reserved bits
set) made the parser emit an unhandled 'error' event, crashing the whole
process and taking down realtime collaboration for every connected user.
We now attach the error listener on every WebSocket as soon as it is
created, before any routing or middleware runs, so malformed frames are
logged instead of crashing the server.
Recent changes broke the way the titles were displayed
when a emoji was present. It was displayed on 2
lines instead of a single line.
We fixed the CSS to ensure that titles with emojis
are displayed on a single line.
We improved the way the underline is displayed with
interlinking blocks.
Add a configuration knob for the trace sampling rate, so we can
enable tracing on middleware and cache spans when debugging slow
requests in production.
Sampling is set to 0 by default, so tracing stays fully off unless
explicitly enabled.
Copied from suitenumerique/meet#1690
Moving a doc via drag-and-drop already applies the move locally through
`treeData.handleMove`, but `useMoveDoc`'s `onSuccess` also
invalidated the `KEY_DOC_TREE` query, which `useDocTree`
reacts to by nulling `treeContext.root` and forcing the whole tree to
refetch and remount. Add a `skipTreeInvalidation`
option so the drag-and-drop handler can skip that invalidation
and rely on its own local update, while other callers (e.g. the
move modal) keep reloading the tree as before.
Gecko cannot express a blended run in PDF, so it falls back to a bitmap of that
run: every commented sentence came out of Print as a 72 dpi image while the rest
of the page stayed text. The print stylesheet already hid the comment highlight,
but only its background and border, so mix-blend-mode stayed and that is what
forces the fallback.
Measured on the engine the report came from, printing the same page twice:
with the current rule the commented run is not extractable and the file carries
an image plus its alpha mask, with mix-blend-mode neutralised it is text again
and the file carries no image at all.
Signed-off-by: tanguy chenier <tanguychenier@gmail.com>
MinIO stopped publishing images on Docker Hub in October 2025 and
archived its GitHub repository in February 2026. `minio/mc` and
`minio/minio` can no longer be pulled, so the `createbuckets` service
fails with `pull access denied for minio/mc`. The same images are still
served from quay.io.
Signed-off-by: Nicolas Clerc <kernicpanel@nclerc.fr>
