The yhub database schema have new update and will probably be modified
in the future. We don't want to maintain this sql schema in the Docs
repo, we want to reuse what is directly made in the yhub project. For
this we reuse the existing bin/init-db.js script
Cover both paths off the legacy Django store end to end: the lazy seed on
first access, and the migrate endpoint replaying every S3 version. The tests
need no database — the admin JWT short-circuits document authorization, so a
fixture is an S3 object on a random uuid — and read the timeline through
yhub 0.5.0's `Accept: application/json`, which spares python a lib0 decoder.
CI grows a valkey service and starts a collaboration server alongside the
backend test job; the tests skip themselves when nothing answers on the new
COLLABORATION_API_URL setting, so `make test` without the dev stack still
passes.
Writing them turned up three things worth fixing in the server.
Backend reads now seed too. getAccessType short-circuited on the admin token
before reaching the legacy store, so a server-side read of an unmigrated
document answered with an empty one, and a create-ydoc against it would have
written a second lineage beside the content the first user access was about
to seed in.
Seeding no longer decides access; the backend's answer alone does. A legacy
object that cannot be migrated — it does not decode, or it exceeds the size
we load — opens as a new document instead of denying, since no retry can fix
it and refusing would leave the document unopenable by anyone. The cause is
logged once per attempt with the bucket, key and stack, and every later access
logs that it admitted a caller without migrating.
That made the failure classifier dangerous, so it is inverted. It was an
allowlist of retryable errors — eight socket errnos — which left every way S3
can refuse (AccessDenied on a rotated key, NoSuchBucket, a region redirect)
counting as "this object is unusable". Denying, that was survivable; opening
empty, one misscoped credential would fork every document touched during the
window. Now only a failure raised while interpreting bytes we already hold is
permanent, marked at the throw site, and everything else answers a retryable
503. Guessing wrong that way costs a retry; the other way costs the document.
The admin seed is also fenced to the org and to main, like the user path
above it. The legacy store is branchless — {docid}/file is main — and the
bookkeeping is per document, so seeding ?branch=draft would have written
main's content into an orphan room and left the real one permanently empty.
Signed-off-by: Kevin Jahns <kevin.jahns@protonmail.com>
We don't maintain the test-e2e-other-browser job,
and it often fails due to various reasons.
To avoid unnecessary noise in our CI pipeline, we
will remove this job.
We are adding workflow_dispatch trigger to the e2e-tests
workflow, so that we can still run e2e tests on other
browsers when needed from the Github actions UI.
We had to maintains 2 jobs, test-e2e-chromium and
test-e2e-other-browser, in the impress-frontend
workflow.
By factorising the E2E tests into a separate
workflow, we can now maintain only one job for
each browser, which is much simpler and easier
to maintain.
We got cases where the last-run is empty, but the
last-failed flag is set to true. If that happens,
the workflow will fail because the last-run is empty.
We now check if the last-run is filled before
setting the last-failed flag.
We improved the test suite to only replay failed
tests when rerunning the test suite.
This allows us to focus on fixing the failed
tests without having to wait for the entire
test suite to run again.
We add the Browser in cache to speed up the test
execution and reduce the time it takes to run the tests.
We add permissions level on workflow to avoid
warning in github action logs.
This is a warning and not an error,
but it is good to have a clean log without warnings.
The psycopg pool config was enabled by default forcing its usage. Using
psycopg pool can be difficult, finding the good configuration take time.
By default its usage should be disable and the maintainer of the
instance should decide to enable it or not.
In order to run the tests we need to increase the max pool size. Only
having 4 connections in the pool is not enough and all the tests using a
transaction are failing with a tiemout error.
We have the same problem running locally so the same value is added to
the postgresql environment file
Update all GitHub Actions to their latest major versions for improved
performance, security patches, and Node.js runtime compatibility.
Signed-off-by: Stephan Meijer <me@stephanmeijer.com>
Building twice the image take lof of time. In soma cases, building the
arm64 image using the artifacts build in the amd64 and thant can be
reused should speed up the build of the arm64 image.
We extract the docker build and push job to a dedicated action called
to make the docker job for each application. This action avoid
copy/paste between each job.
We use the Docker Hub Workflow to build and push
our images to Docker Hub, but to check if we
have vulnerabilities in our images as well.
When we are just checking for vulnerabilities,
we don't need to do all the builing steps.
This commit optimizes the workflow by only doing the
necessary steps when we are just checking for
vulnerabilities, so during pull requests
without label "preview" we skip the build steps,
and we do not activate QEMU.
We added a .trivyignore file to ignore the
CVE-2026-26996 vulnerability coming from the y-provider
image.
We also updated the docker-hub.yml workflow to
use this .trivyignore file when scanning our
Docker images with Trivy.
Add workflow_dispatch-triggered workflow that builds and pushes images
to GitHub Container Registry (ghcr.io). This allows forked repositories
to build their own images for testing without requiring Docker Hub
credentials.
Images are tagged with branch names, semver tags, and commit SHA for
easy testing of specific builds.
Signed-off-by: dtinth on MBP M1 <dtinth@spacet.me>
We added a feature to print documents directly
from the browser. The function is called
`window.print()`, this name collides with Python's
`print()` function. To avoid false positives in our
CI when checking for print statements, we limit
the search to only the backend code.
On the release pull request workflows, we sometimes
face issues with disk space. We clean up some space
before starting Docker services or after finishing
to build images to avoid these issues.
The label preview will deploy a full environment. This environment is
accessible using a specific url. This commit will publish a comment with
the good url.
Replace custom Docker Hub authentication with standard, secure,
official GitHub actions for improved security and maintainability.
Uses officially supported actions that follow security best practices
and receive regular updates from GitHub.
Avoid unsecure handling of GitHub secrets.
Thanks to @lebaudantoine
git-lint steps are independant and we would like to have all checks at
once. Using the `if: always()` instruction should ensure all steps
should be run event if the previous fails.
We don't need to run the bundle-size-check job if
the app didn't change.
If the yarn.lock file or the app have changed, the
bundle-size-check job will be triggered.
The setup-python action is able to cache the dependencies and reuse this
cache while the pyproject file has not changed. It is easy to setup,
just the package manager used has to be declared in the cache settings.
Job that will give a report on the bundle size
of the frontend application.
Good to know if the bundle size is increasing or
decreasing and if the changes are acceptable.
We had lot of problems with the previous env.d system.
Users were often confused by the need to change
the env.d files manually, leading to issues
when using the project locally.
This commit introduces a new system that uses
.env.local files, which are automatically created
and can be modified by users without affecting
the original env.d files. This should simplify
the development process and reduce confusion by
removing the need to manually edit env.d files.
In order to speed the rebuild of images, the --no-cache option is
removed by default. If we want to build the images without cache, the
cache paramter must be used.
We want to run the e2e tests using the frontend and y-provider
production images. We created a dedicated compose file adding just
missing services. These services are built in the CI.
Manuel Raynaud