Cover both paths off the legacy Django store end to end: the lazy seed on
first access, and the migrate endpoint replaying every S3 version. The tests
need no database — the admin JWT short-circuits document authorization, so a
fixture is an S3 object on a random uuid — and read the timeline through
yhub 0.5.0's `Accept: application/json`, which spares python a lib0 decoder.
CI grows a valkey service and starts a collaboration server alongside the
backend test job; the tests skip themselves when nothing answers on the new
COLLABORATION_API_URL setting, so `make test` without the dev stack still
passes.
Writing them turned up three things worth fixing in the server.
Backend reads now seed too. getAccessType short-circuited on the admin token
before reaching the legacy store, so a server-side read of an unmigrated
document answered with an empty one, and a create-ydoc against it would have
written a second lineage beside the content the first user access was about
to seed in.
Seeding no longer decides access; the backend's answer alone does. A legacy
object that cannot be migrated — it does not decode, or it exceeds the size
we load — opens as a new document instead of denying, since no retry can fix
it and refusing would leave the document unopenable by anyone. The cause is
logged once per attempt with the bucket, key and stack, and every later access
logs that it admitted a caller without migrating.
That made the failure classifier dangerous, so it is inverted. It was an
allowlist of retryable errors — eight socket errnos — which left every way S3
can refuse (AccessDenied on a rotated key, NoSuchBucket, a region redirect)
counting as "this object is unusable". Denying, that was survivable; opening
empty, one misscoped credential would fork every document touched during the
window. Now only a failure raised while interpreting bytes we already hold is
permanent, marked at the throw site, and everything else answers a retryable
503. Guessing wrong that way costs a retry; the other way costs the document.
The admin seed is also fenced to the org and to main, like the user path
above it. The legacy store is branchless — {docid}/file is main — and the
bookkeeping is per document, so seeding ?branch=draft would have written
main's content into an orphan room and left the real one permanently empty.
Signed-off-by: Kevin Jahns <kevin.jahns@protonmail.com>
Add POST /collaboration/migrate/v1/docs/{id}, which replays every S3 version
of a document's legacy `{id}/file` object into one gc:false Yjs document and
stores it as a single row at clock 0, crediting each version with its own S3
timestamp. Nothing existing is deleted and nothing goes on the stream, so the
next compaction merges that row like any other. The clock-0 insert is ON
CONFLICT DO NOTHING and migrated ids are kept in a valkey set, so the endpoint
is idempotent without a lock. The activity api then reports the same timeline
as the backend's /documents/{id}/versions/, instead of the single
migration-time change the lazy soft migration leaves behind.
That lazy seed now writes no insertAt/deleteAt. Persisted contentmaps are
merged rather than de-duplicated, so a seed timestamp would survive next to
the real per-version one on the same ids and the activity api would report
whichever the unordered row scan put last. A seed is not an editing event and
has no honest time to report.
Upgrade yhub to 0.5.0, where error codes encode retry semantics (4xx
permanent, 5xx and 429 retryable) and auth plugins may throw apiError(503). A
temporarily unreachable Django backend, JWKS endpoint or legacy S3 store is
now reported as 503 rather than denied like a permission failure, so clients
retry instead of giving up.
The legacy-store code moves out of server.js into migration.js, with the
shared *_FILE secret helper in env.js.
Signed-off-by: Kevin Jahns <kevin.jahns@protonmail.com>
yhub verified Django's RS256 admin JWT without checking "aud", so the
y-converter token Django hands to the converter process was replayable
here — and admin: true short-circuits getAccessType to "rw" on every
document, plus the backend-internal reset-connections purpose and the
X-User-Id attribution override. Require aud: "yhub", as y-provider
already does for its own audience. Nothing in the backend calls yhub's
admin endpoints yet, so no caller is affected.
Signed-off-by: Kevin Jahns <kevin.jahns@protonmail.com>
The hocuspocus-to-yhub migration entry claimed both the kick and
get-connections APIs were deferred pending a yhub equivalent. The kick
flow now has its server-side replacement (the reset-connections
endpoint, backend wiring pending), and get-connections lost its only
consumer when the can-edit mechanism was removed — it is dropped, not
awaiting reimplementation.
Signed-off-by: Kevin Jahns <kevin.jahns@protonmail.com>
With SOFT_MIGRATION=true, the first access to a document yhub does not
know yet fetches the legacy snapshot from Django's S3 media bucket
({id}/file, UTF-8 base64 of a raw Yjs update), seeds the room through
the compute pool - attributed to "system" with a migration=s3 custom
attribution - and only then admits the connection, so the initial sync
always includes the seed. Now that the frontend no longer bootstraps
rooms client-side (content GET/PATCH removal), this is the only path
that brings legacy content into yhub; keep the flag on until a batch
backfill has migrated the full corpus.
A missing S3 object is the brand-new-document case and yields an empty
room; every real failure fails closed (opaque 401, y-websocket retries
with backoff). Existence is probed postgres-first (bare SELECT, then
the valkey stream, then the SELECT again to close the compaction
race). Guard rails: a per-docid verdict cache (poison objects cannot
sustain an S3 retry storm, transient errors expire in 15s, per-replica
seed backpressure denies once without caching), in-flight dedup, a
token-owned cross-replica valkey lock released by compare-and-delete,
a 10s S3 fetch timeout that also destroys a late-arriving response
stream, and the same 10MiB decoded cap as create-ydoc. Concurrent
seeds stay correct regardless: the frozen snapshots share one Yjs
lineage, so duplicates merge as CRDT no-ops.
Also reject non-lowercase docids (Django serializes UUIDs lowercase; a
case variant would open a parallel room and miss its S3 object) and
refuse to boot when AWS_S3_ENDPOINT_URL carries a path the minio
client cannot address. On AWS the read-only credentials must include
s3:ListBucket so a missing object surfaces as NoSuchKey rather than
AccessDenied - see the README for the full guarantees and ops notes.
Signed-off-by: Kevin Jahns <kevin.jahns@protonmail.com>
The Playwright MCP browser session writes snapshots and console logs
into .playwright-mcp/ at the repository root while driving the app
during development — keep them out of version control.
Signed-off-by: Kevin Jahns <kevin.jahns@protonmail.com>
We remove the code related to the content GET and
PATCH endpoints, as they are no longer used in the
codebase. The yhub server will handle the content
management directly, providing the content and
managing the updates. This change simplifies the code
and reduces the complexity of the frontend application.
We will need to reimplement the saving mechanism
in the service worker when we are offline.
Let's wait that the service is fully developed
on the yhub side before we implement this feature.
We will not block anymore the users not connected
to the collaboration server from editing the document,
we will have an HTTP fallback instead, so we can
remove the "can-edit" mechanism and the related code.
Address the findings of an adversarial review of the new endpoint:
- Only the backend admin token may attribute content to another user via
the X-User-Id header. The endpoint uses the default access purpose, so
any editor with update ability can call it — honoring the header for
them would let an editor forge the attribution history of the first
revision (the websocket path likewise stamps the server-side
identity). Regular callers now always author as themselves; verified:
an editor session posting X-User-Id gets its own userid stamped.
- Reject non-main ?branch= requests (400). Cookie users are main-only
via getAccessType, but the admin token bypasses it and could seed an
orphan (org, docid, branch) room no user-facing path reads — while
dodging the branch-scoped 409 existence check.
- Correct the concurrent-create comment: two racing creates merge as
independently generated updates (fresh clientIDs), so the seeded
content appears twice — user-visible duplication, not merely a
doubly-attributed revision. Still accepted (Django creates each doc
once and a duplicated seed is user-fixable), but the tradeoff is now
stated accurately.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Kevin Jahns <kevin.jahns@protonmail.com>
Python cannot call yhub's built-in PATCH ydoc api because its body must
be lib0-any encoded - a lib0-specific binary framing with no
implementation outside javascript. The new endpoint
POST /collaboration/create-ydoc/v1/{org}/{docid} accepts the raw binary
Yjs update (pycrdt get_update() / Y.encodeStateAsUpdate output) as
application/octet-stream, so Django can seed a document's initial state
with a plain requests.post(url, data=raw_bytes) - needed by the
server-side creation flows (file import, create-for-owner, duplication,
template instantiation) whose yhub rooms currently stay empty until the
first browser connects.
Strict create semantics: 409 when the room already has content
(checked via getDoc, covering persisted state and uncompacted stream
messages; yhub has no atomic create, concurrent creates merge via CRDT
and never corrupt). The initial content is attributed to the optional
X-User-Id header, else to the caller's identity. Access uses the
default purpose, i.e. standard document write access like the built-in
ydoc routes: the admin JWT, or a user session with update ability.
Malformed updates map to 400 (the compute worker rejects them and the
pool replaces the thread), empty updates to 400, bodies over 10MiB to
413.
Gotcha worth noting: req.bytes() resolves to a Node Buffer, but yhub's
compute-task schema validates with lib0's exact-constructor Uint8Array
check, so the body is re-viewed as a plain Uint8Array before it is
handed to the compute pool.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Kevin Jahns <kevin.jahns@protonmail.com>
Y_PROVIDER_API_KEY is no longer used in the codebase,
so we can remove it from the helm chart and the documentation.
We adapt the documentation to use the new JWT conversion
mechanism instead.
The /api/convert route no longer accepts the Y_PROVIDER_API_KEY shared
secret. It now verifies the admin JWT signed by Django against the
JWKS published on its /api/v1.0/jwks endpoint.
Add POST /collaboration/reset-connections/v1/{org}/{docid} (optional
X-User-Id header) to yhub-server. It distributes yhub recheckAuth: every
server re-runs the access check per matching connection and closes only
those whose access actually changed (close code 4401), so unaffected
clients see no reconnect churn. The endpoint authenticates with the
RS256 admin JWT issued by JWTService, verified against the backend JWKS
(new jose dependency); the admin token acts as the "system" user and is
the only principal granted the reset-connections access purpose. The
backend does not trigger it on permission changes yet - that wiring
comes separately, now that CollaborationService is gone.
yhub is upgraded to 0.4.0 and serves every route under the
/collaboration/ prefix (server.apiPrefix): the websocket moves to
/collaboration/ws/v1/docs, and the built-in document apis are meant to
be publicly exposed alongside it, with reset-connections as the one
backend-internal exception.
Also harden websocket auth: fail closed when the backend errors (only a
genuine 401/403 falls back to the anonymous identity, so a signed-in
editor can never hide from a targeted recheck under an anon userid) and
tolerate small clock skew when verifying the cached admin token.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Kevin Jahns <kevin.jahns@protonmail.com>
The CollaborationService was doing nothing since we started the
migration to yhub, all the code using it is now removed. Also the
`can-edit` endpoint and all the safeguard mechanism relying on the
presence of other users connected to the websocket will not be used
anymore, it will be possible to replace all of this with yhub, so all
this code is also removed.
For now the only token we will need is ont with the admin claim set to
True. To not repeat the creation of this token again and again, we
created a dedicated method to issue this token in the JWTService class.
The yhub service will need our public key in order to validate the jwt
token we will used. We choose to expose a jwks endpoint as it is a
standard wat to do this.
Support ?view=present&slide=N deep-links (clamped, URL kept in sync) and
a "Copy link to slide" action in the floating bar. Ignore the deep-link
on mobile.
Closes#2397
Recent upgrade of tsc-alias introduced a breaking
change in the build command, which now requires the
`--project` flag to specify the path to the
tsconfig file.
@gouvfr-lasuite/ui-kit pin @gouvfr-lasuite/cunningham-react,
so multiple versions of
@gouvfr-lasuite/cunningham-react were being installed,
which was causing issues.
We pin the version to 4.3.1, forcing all packages to
use the same version.
Blocknote is not compatible with the latest version
of the ai packages:
- @ai-sdk/openai
- ai
Better to downgrade to the previous version until the
compatibility issue is resolved.
We upgraded prettier in a previous commit. Some files
were not formatted with the new prettier version.
This commit applies the new prettier formatting to
those files.
We are downgrading TypeScript from version 7.0.2
to 6.0.3.
To bump the version of TypeScript, we will need
to update the codebase to be compatible with the
new version.
FRONTEND_HOMEPAGE_FEATURE_ENABLED=false makes the Auth guard send
anonymous visitors on / straight to login instead of /home, but the
/home page itself still rendered the full homepage when reached via a
direct link. Redirect it to login too, so disabling the feature really
disables the page.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PhqB4SHsHCuMfprwLEN3rR
Signed-off-by: Maarten Draijer <maarten@madra.nl>
We have renamed the Chinese locale from `cn_CN`
to `zh_CN` to align with standard locale naming
conventions.
Additionally, we have added support for two new
locales: Esperanto (`eo_PL`) and
Traditional Chinese (`zh_TW`).
A selected image (or any selected node) leaked its editing outline into
the read-only slide. Suppress the selected-node outline so nothing looks
focused unless the user tabs or clicks.
Closes#2393
Dividers with children are kept as structural parents so their slide
keeps its indentation; the divider's own rule is hidden. Empty
paragraphs around a divider are dropped so habitual spacing no longer
offsets slides.
Closes#2398
Prepend a slide with the document title (emoji stripped) so every
presentation opens on a cover, announced to screen readers like any
other slide (#2466).
Pull the BlockNote rendering out of PresenterSlide into
PresenterSlideContent so it can be reused by the title slide and the PDF
export. Add a PresenterSlideData union (title | content) and a shared
PresenterBlock type.
Move presenter ownership out of DocToolBox into a top-level
PresenterRoot driven by a zustand store, lazy-load the overlay and paint
a boot cover while the editor boots. Groundwork for the deep-link and
present-from-here features (#2466).
Changed:
- ♻️(backend) reset collaboration connection in cascade for all children
Fixed:
- 💄(frontend) fix some UI/UX in the left panel
- 🐛(frontend) fix tree dnd firefox
Manuel Raynaud