The offline support couldn't work with the existing
implementation anymore, because there is no request
to get or save data anymore, everything is handled
with web sockets.
In order to support offline functionality, we
leveraged y-indexeddb to store and synchronize
local changes, ensuring that the application remains
functional even when offline.
Version grouping used a hard-coded 60s window, and its
timestamps are minted server-side.
We now expose COLLABORATION_VERSION_GRANULARITY_MS
through /config, read it in useDocActivity.
By doing so, we can control the granularity of
version history with settings, it can be adjusted as
needed. It will help us to test different version
history granularities in our e2e tests.
Previously we added to retrieve document response a new property
user_access_since that contains the date from when the user started to
have access to the document. The way t was made added an other
annotation to the Document queryset making the sql query more and more
complex. We decided to lighten the queryset and expose the access the
user has on the document instead and read the history from the
created_at property.
The check was dropped during the yhub migration, on the grounds that
`reset_connections` was a no-op. It is not one any more: the backend calls
yhub's reset-connections route, which re-runs the authorization of each
connection and closes the sockets whose access is no longer the one they
were opened with, with the code 4401.
The old check cannot come back as it was. It had the author of the document
change its visibility and waited for the author's own socket to close, which
is what the collaboration server used to do - close every connection to the
document, whatever it was that changed. yhub leaves alone the connections
that are unaffected, and an owner's access is the same before and after, so
the author's socket is now deliberately kept.
The client to watch is therefore another one: the anonymous reader holding
the public link, while the author switches that link from reading to
editing. Their socket goes down and the client comes back on its own.
That the new connection is live is asserted by what flows through it rather
than by the socket being open - the author writes after the change and the
text reaches the reader, with nothing reloaded in between.
Signed-off-by: Kevin Jahns <kevin.jahns@protonmail.com>
The export regression tests injected their fixture by intercepting
`GET /documents/{id}/content/`, and the 401 routing test intercepted the
`PATCH .../content/` a document used to send on the way out. Both endpoints
went with the yhub migration, so the interceptions matched nothing: the
exported document was whatever the two images added to an empty page, and
the 401 page was never reached.
The fixture now goes to the collaboration server, which is the only thing
that holds document content. It is seeded before the editor mounts: its
first block carries the id BlockNote gives the empty paragraph it writes
into any document it opens empty, so seeding an already open document would
leave a stray paragraph in front of the fixture and shift every page. The
reference PDFs are unchanged and still match, page count, text and pixels.
The 401 test reloads rather than navigating to another document, nothing
being sent on the way out any more.
Signed-off-by: Kevin Jahns <kevin.jahns@protonmail.com>
A document's imported past is attributed to `system`, and until now it went
through the same grouping as ordinary editing. That is wrong twice over.
It loses history. Before the migration the editor saved the whole document on
an interval of exactly 60s, each save becoming one version of the legacy file,
and the migration replays them at their original timestamps — so an imported
history arrives as a chain of entries spaced almost exactly the width of the
grouping window. Whether two of them survived as two came down to whether the
round trip that wrote them ran a few milliseconds fast or slow: measured over
simulated sessions, about a third of the chain collapsed. A document whose
saves happened closer together lost far more — four versions two seconds apart
became one.
And it is the wrong question to ask of them. Grouping summarises someone's
editing into the moments that are worth listing; there was no editing session
here, only a record of saves that already happened, and the only honest thing
to do with that is to show it one for one.
`system` is now excluded from grouping on both sides. The collaboration server
takes the exclusion as a parameter, which is what the yhub bump in this commit
is for. The panel applies it again and more strictly: the server declines to
merge an excluded entry into the one before it, having already refused on the
author comparison it makes, while the panel merges across authors and so must
refuse in both directions — an edit made moments after a document was migrated
must not absorb the imported entry beside it, or be absorbed by it.
Live editing is unchanged: several edits by one person seconds apart are still
one version.
Signed-off-by: Kevin Jahns <kevin.jahns@protonmail.com>
The version history has been dead since the migration. It listed S3 object
versions of the legacy `{pk}/file` key, and nothing writes that key any more,
so every document's list has been frozen at its migration date; restoring one
was a stub that closed the modal and did nothing, while still promising that
the document would be replaced.
It now reads the collaboration server, which is what keeps the history: the
list comes from `activity`, a selected version is previewed from `changeset`
as the document stood at that moment, and restoring one is a `rollback`.
A version is a minute of editing — changes less than a minute apart become
one, and none spans more than a minute. The collaboration server groups only
changes by the same author, so the browser merges what is left across authors:
a version is a moment in the document, not a moment in one person's editing.
Both are needed, and both use the same rule.
This grants `history.rollback` to editors, which is the first time a browser
may change the past rather than read it, and publishes the rollback route.
A reader is refused it twice over — the collaboration server treats it as a
dead grant without document write access, and the endpoint is withheld as well.
Mutations refuse where reads clamp, so a rollback reaching further back than
the history a user was granted is rejected rather than trimmed: nobody can
undo work that predates their own access, and a rollback with no bound at all
is refused outright. `prune`, which erases, stays granted to nobody. Restoring
is not destructive: it appends a change that undoes another, so what it
replaced stays in the history and can be restored again.
The backend's version endpoints are untouched and now have no caller. They are
marked deprecated with the condition for removing them, since until a document
has been replayed by `migrate_documents` they hold the only record of what it
looked like before it moved.
Also fixes the e2e helper that waited for the removed content endpoint, so it
never returned, and the three version tests that hung behind it.
Signed-off-by: Kevin Jahns <kevin.jahns@protonmail.com>
yhub 0.8.0 retires the 'r' | 'rw' | null access vocabulary. The auth
plugin now answers a typed permission object stating, facet by facet,
what a subject may do with a document, and yhub enforces every facet
itself - on the websocket and on the REST routes alike. Three rules we
wanted but could not express under the old vocabulary become one-line
facets. Our whole access policy now lives in
src/yhub-server/permissions.js, apart from the server so that it can be
read and tested without standing up redis and postgres.
Read-only users no longer share their cursor #2544. A read-only
connection could still propagate awareness updates to everyone else in
the document, even though its document updates were already dropped.
Presence is now a permission of its own, separate from the right to
edit: a reader receives it and never publishes it. The collaboration
server enforces that rather than trusting the editor to stay quiet, so a
modified or stale client changes nothing. The frontend has to know it
too - the http fallback provider has no receive-only mode, so a reader's
provider is built with no awareness instance at all, or its first PATCH
would take a 403 and close it for good.
The browser is granted only the two routes it uses, the websocket and
ydoc for the http fallback. Everything else - history, rollback, prune,
and every backend-internal endpoint - is refused to it, as is any
endpoint a future release adds, because the grant names no wildcard.
create-ydoc in particular was reachable by any signed-in editor and is
now the backend's alone.
Anonymous visitors are given the userid "anonymous" rather than no
identity at all, which is what lets them keep editing public documents:
yhub refuses the upgrade of a caller that holds the write but cannot be
attributed. Their edits share one author.
Room is DocRef throughout, which is a rename of object keys and not only
of types: the worker event payload and the stream message lookup both
carry it, and both fail silently rather than loudly when missed.
Signed-off-by: Kevin Jahns <kevin.jahns@protonmail.com>
Document content stopped going through Django when the collaboration
server took it over, and these two are what the move left behind. Both
are unreachable rather than merely unused, so they are removed instead
of being carried forward.
useUpdateDoc sent websocket: true whenever the provider was synced, to
unlock a cache lock the backend used to hold while another user was
connected. That lock is gone - there is no websocket field on the
serializer and nothing reads one - so DRF has been silently dropping the
key. Removing it also drops the store's only isSynced reader.
The service worker still wrote an entry to a doc-content table keyed on
documents/{id}/content/, an endpoint that no longer exists. Nothing ever
read it back: handlerDidErrorRead only ever asks for doc-list and
doc-item. The table is dropped on the next upgrade so it does not linger
in browsers that already have it.
This also clears the five standing typescript errors in the repository,
which were all stale doc-content literals in the tests for the code
being removed.
Signed-off-by: Kevin Jahns <kevin.jahns@protonmail.com>
Some networks refuse a websocket upgrade - corporate proxies, captive
portals - and a browser is told nothing more than "the connection
closed", so those users could not edit at all. The editor now runs a
second transport next to the socket, polling the collaboration server's
REST api on the same room, with the same session cookie and the same
authorization, and only while the socket is down. Local changes go out
about a second after the last keystroke and remote ones arrive within
ten seconds, so editing works with visibly more latency rather than not
at all. The socket keeps being retried underneath, so a client that fell
back during an outage returns to it on its own, and nothing is lost in
either direction - both transports publish from the same document.
This makes /collaboration/ydoc/ a route browsers call, so
COLLABORATION_SERVER_ORIGIN is now handed to yhub as its cors
configuration and gates the http routes as well as the websocket.
Signed-off-by: Kevin Jahns <kevin.jahns@protonmail.com>
The yhub server returns custom status code when the websocket is not
accessible, like 4401 when an access is removed and 4404 when a document
is deleted. the websocket client now use these custom status code to
stop reconnection forever.
We remove the code related to the content GET and
PATCH endpoints, as they are no longer used in the
codebase. The yhub server will handle the content
management directly, providing the content and
managing the updates. This change simplifies the code
and reduces the complexity of the frontend application.
We will need to reimplement the saving mechanism
in the service worker when we are offline.
Let's wait that the service is fully developed
on the yhub side before we implement this feature.
We will not block anymore the users not connected
to the collaboration server from editing the document,
we will have an HTTP fallback instead, so we can
remove the "can-edit" mechanism and the related code.
The /api/convert route no longer accepts the Y_PROVIDER_API_KEY shared
secret. It now verifies the admin JWT signed by Django against the
JWKS published on its /api/v1.0/jwks endpoint.
Add POST /collaboration/reset-connections/v1/{org}/{docid} (optional
X-User-Id header) to yhub-server. It distributes yhub recheckAuth: every
server re-runs the access check per matching connection and closes only
those whose access actually changed (close code 4401), so unaffected
clients see no reconnect churn. The endpoint authenticates with the
RS256 admin JWT issued by JWTService, verified against the backend JWKS
(new jose dependency); the admin token acts as the "system" user and is
the only principal granted the reset-connections access purpose. The
backend does not trigger it on permission changes yet - that wiring
comes separately, now that CollaborationService is gone.
yhub is upgraded to 0.4.0 and serves every route under the
/collaboration/ prefix (server.apiPrefix): the websocket moves to
/collaboration/ws/v1/docs, and the built-in document apis are meant to
be publicly exposed alongside it, with reset-connections as the one
backend-internal exception.
Also harden websocket auth: fail closed when the backend errors (only a
genuine 401/403 falls back to the anonymous identity, so a signed-in
editor can never hide from a targeted recheck under an anon userid) and
tolerate small clock skew when verifying the cached admin token.
Signed-off-by: Kevin Jahns <kevin.jahns@protonmail.com>
Reset the background when Backspace converts a callout to a paragraph.
Preserve content, alignment, regular text deletion, and undo behavior.
Fixes#2052
Signed-off-by: Manixhor <manigururam@hotmail.com>
Export Markdown as a ZIP containing the document and same-origin media.
Normalize filenames and restore the export UI after asynchronous failures.
Cover media packaging, modal behavior, filenames, and the browser flow.
Signed-off-by: Gokul Krishna <gokul.06krishna@gmail.com>
Ctrl+Alt+P (Cmd+Option+P on macOS) opens the current document in
presentation mode, saving two clicks through the document menu.
Signed-off-by: Anveet Pal <anveetpal12@gmail.com>
We can now choose to duplicate documents along with
their subdocuments. A confirmation modal will now
appear to let the user choose whether to include
subdocuments in the duplication, only if the document
has subdocuments.
A toast notification will inform the user and
give the possibility to undo the duplication if
needed.
Update lightly the ui to match the new design mockups,
to fit with what was proposed for the duplicate subdocs
feature.
It updates:
- modal closing position
- checkbox style
- toast style
- create a ButtonLink component, a wrapper above
the ui-kit component to fit the Next.js routing system.
We fixed the redirect behavior after deleting a
document to ensure the user stays on the correct page
after deleting a document.
We added a test to avoid regression in the redirect
behavior.
useToastProvider()'s `toast` function is recreated by the provider
every time any toast is shown anywhere in the app. useUploadFile
listed it as a dependency of `uploadFile`, which is itself a
dependency of useCreateBlockNote, so an unrelated toast (e.g. "Link
copied!") recreated `uploadFile` and made BlockNote rebuild the whole
editor instance, resetting the scroll position to the top of the
document.
Added useToast, a drop-in wrapper around useToastProvider that keeps
`toast`'s identity stable behind a ref, and switched every call site
in the app to use it instead of the provider's hook directly.
Dropping a file larger than the allowed size showed a bare "unknown
error" in the editor. The proxy in front of the API cuts the request
and answers a 413 with an HTML body, so errorCauses threw while
parsing it as JSON and no cause ever reached the error panel.
The size limit the backend already enforces is now exposed by the
config endpoint, and the editor checks the file against it before
sending anything, with the same toast wording the document import
uses. errorCauses no longer throws on a body it cannot parse, and a
413 without a usable cause falls back to an explicit message, which
covers the instances whose proxy limit is lower than the application
one.
The size formatting duplicated in the import hook moved to a shared
util.
Signed-off-by: risk-alt <aldu6974@gmail.com>
$opt_in fired on every reconnect since opt_in_capturing() captures an
event by default, even though this call just re-enables tracking after
a network drop and isn't a real consent action. Also disable web
vitals capture, which fed a single low-value insight.
Added:
- 🔧(backend) fine tune redis cache options
- ✨(frontend) make the full last-update date available
- 💄(frontend) redesign email confirmation standalone page
Changed:
- ⬆️(backend) upgrade celery to version 5.6.3
- ⚡️(backend) stop using LEFT(value, LENGTH(path)) in sql queries
- 🚚(project) switch docspec image to ghcr.io/docspec/api
- 🚚(global) move favorite documents API endpoint
to `/documents/favorites/`
Fixed:
- 🐛(backend) skip session creation for the liveness probe
- 🐛(frontend) preserve page titles when adding an emoji
- 🐛(frontend) scroll to the linked block in read-only documents
- 🐛(frontend) hide the selection highlight on presenter images
- 🐛(y-provider) prevent process crash on malformed websocket frames
- 🐛(frontend) keep commented text sharp when printing to PDF
- 🐛(docker) pull minio images from quay.io
- ♿️(frontend) restore presenter focus trapping after share links
- 🐛(frontend) export any raster image supported by the browser to a PDF
BlockNoteReader (read-only docs) doesn't register the find & replace
tiptap extension. If the panel is left open while the active editor
switches to it -- e.g. a collaborative doc turning read-only after a
WebSocket disconnect -- calling into its commands threw
"commands.clearSearch is not a function".
useFindReplace now exposes whether the current editor actually
supports find & replace, and FindReplace closes the panel and renders
nothing when it doesn't, instead of crashing.
When a WebSocket connection is rejected for a missing Origin or cookies,
or hits a route with no matching handler, express-ws still completes the
upgrade handshake and only closes the socket afterwards. The `ws`
library keeps parsing incoming frames during the close handshake, which
can take up to 30 seconds, and no 'error' listener was attached to these
sockets in the meantime. A single malformed frame (e.g. reserved bits
set) made the parser emit an unhandled 'error' event, crashing the whole
process and taking down realtime collaboration for every connected user.
We now attach the error listener on every WebSocket as soon as it is
created, before any routing or middleware runs, so malformed frames are
logged instead of crashing the server.
Recent changes broke the way the titles were displayed
when a emoji was present. It was displayed on 2
lines instead of a single line.
We fixed the CSS to ensure that titles with emojis
are displayed on a single line.
We improved the way the underline is displayed with
interlinking blocks.
Moving a doc via drag-and-drop already applies the move locally through
`treeData.handleMove`, but `useMoveDoc`'s `onSuccess` also
invalidated the `KEY_DOC_TREE` query, which `useDocTree`
reacts to by nulling `treeContext.root` and forcing the whole tree to
refetch and remount. Add a `skipTreeInvalidation`
option so the drag-and-drop handler can skip that invalidation
and rely on its own local update, while other callers (e.g. the
move modal) keep reloading the tree as before.
Gecko cannot express a blended run in PDF, so it falls back to a bitmap of that
run: every commented sentence came out of Print as a 72 dpi image while the rest
of the page stayed text. The print stylesheet already hid the comment highlight,
but only its background and border, so mix-blend-mode stayed and that is what
forces the fallback.
Measured on the engine the report came from, printing the same page twice:
with the current rule the commented run is not extractable and the file carries
an image plus its alpha mask, with mix-blend-mode neutralised it is text again
and the file carries no image at all.
Signed-off-by: tanguy chenier <tanguychenier@gmail.com>
Keep the relative timestamp in the document header while exposing the
localized full date through the existing tooltip on hover and keyboard focus.
Signed-off-by: fch-aa <21101725+fch-aa@users.noreply.github.com>
The ws library emits an unhandled 'error' event when a client sends a
frame with unexpected RSV bits (e.g. permessage-deflate mismatch),
which was crashing the whole y-provider process since no listener was
attached to the socket. Add an error listener to log and drop the
offending connection instead.
