mirror of
https://github.com/suitenumerique/docs.git
synced 2026-09-11 20:27:56 +02:00
📦️(yhub) switch yhub-server to yarn
Every other package in this repository is installed with yarn; yhub-server was the only one on npm, with its own package-lock.json. To ensure consistency it now uses yarn like the rest of the project. package-lock.json is replaced by yarn.lock, a packageManager field is added, and the Dockerfile, CI workflow, Makefile, helm init-db job and the documentation move from `npm ci` / `npm run init-db` to `yarn install --frozen-lockfile` / `yarn init-db`.
This commit is contained in:
@@ -236,7 +236,7 @@ jobs:
|
||||
|
||||
- name: Install the collaboration server
|
||||
working-directory: src/yhub-server
|
||||
run: npm ci --omit=dev
|
||||
run: yarn install --frozen-lockfile --production
|
||||
|
||||
# yhub ships its own DDL and creates the database as well, so this needs
|
||||
# the dependencies installed above — hence its place after them
|
||||
@@ -244,7 +244,7 @@ jobs:
|
||||
working-directory: src/yhub-server
|
||||
env:
|
||||
POSTGRES: postgres://dinum:pass@localhost:5432/yhub
|
||||
run: npm run init-db
|
||||
run: yarn init-db
|
||||
|
||||
- name: Start the backend for the collaboration server to authenticate against
|
||||
env:
|
||||
|
||||
@@ -182,6 +182,8 @@ and this project adheres to
|
||||
`/external_api/{version}/jwks`
|
||||
- 🔒️(collaboration) reject admin jwts not issued for the yhub audience
|
||||
- 🔧(collaboration) adapt docker stack for development purpose
|
||||
- 🔧(yhub) manage the collaboration server dependencies with yarn like the rest
|
||||
of the project (`package-lock.json` replaced by `yarn.lock`)
|
||||
- ⏪️(backend) reintroduce the reset connection mechanism
|
||||
|
||||
### Fixed
|
||||
|
||||
@@ -343,7 +343,7 @@ migrate: ## run django migrations for the impress project.
|
||||
@$(MANAGE) migrate
|
||||
.PHONY: migrate
|
||||
|
||||
# Runs the DDL script yhub ships (`bin/init-db.js`, wrapped as `npm run
|
||||
# Runs the DDL script yhub ships (`bin/init-db.js`, wrapped as `yarn
|
||||
# init-db`): it creates the yhub database when missing, then every table the
|
||||
# installed @y/hub version needs. yhub never runs DDL from the server or the
|
||||
# worker, so this is what applies a schema change after an upgrade.
|
||||
@@ -354,7 +354,7 @@ migrate: ## run django migrations for the impress project.
|
||||
migrate-yhub: ## create or upgrade the collaboration server (yhub) schema.
|
||||
@echo "$(BOLD)Running yhub migrations$(RESET)"
|
||||
@$(COMPOSE) up -d yhub-postgres yhub-valkey
|
||||
@$(COMPOSE_RUN) --no-deps yhub npm run init-db
|
||||
@$(COMPOSE_RUN) --no-deps yhub yarn init-db
|
||||
.PHONY: migrate-yhub
|
||||
|
||||
superuser: ## Create an admin superuser with password "admin"
|
||||
|
||||
+1
-1
@@ -52,7 +52,7 @@ upgrade, in the order they are done, and end with the API changes.
|
||||
rest of what it reads.
|
||||
|
||||
- ⚠️ **Its schema is not created when it starts.** The server never runs DDL:
|
||||
run the script it ships, `npm run init-db` (`node
|
||||
run the script it ships, `yarn init-db` (`node
|
||||
node_modules/@y/hub/bin/init-db.js` in the image), once before starting it
|
||||
and again after every upgrade that adds a table. It creates the database when
|
||||
it is missing, and it is idempotent, so re-running it is always safe. The
|
||||
|
||||
@@ -14,7 +14,7 @@ REDIS: redis://{redis-host}:6379/0
|
||||
POSTGRES: postgres://{user}:{password}@{postgres-host}:5432/yhub
|
||||
```
|
||||
|
||||
Nothing creates that schema at startup: the server never runs DDL. Run the script yhub ships (`npm run init-db`, which the helm chart runs as a job) once before starting it, and again after every upgrade that adds a table. It creates the database when it is missing, it is idempotent, and until it has run every document read fails with `relation "..." does not exist`.
|
||||
Nothing creates that schema at startup: the server never runs DDL. Run the script yhub ships (`yarn init-db`, which the helm chart runs as a job) once before starting it, and again after every upgrade that adds a table. It creates the database when it is missing, it is idempotent, and until it has run every document read fails with `relation "..." does not exist`.
|
||||
|
||||
The Django backend reads and writes document content there too, so point it at the service:
|
||||
|
||||
|
||||
@@ -175,7 +175,7 @@ documents what each of them changes.
|
||||
| ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------- |
|
||||
| PORT | Port the server listens on | 3002 |
|
||||
| REDIS | **Required.** Redis/Valkey url holding the live state of the documents. Not a cache: it holds what no worker has persisted yet | |
|
||||
| POSTGRES | **Required.** Url of the yhub database. Created by `npm run init-db`, never by the server | |
|
||||
| POSTGRES | **Required.** Url of the yhub database. Created by `yarn init-db`, never by the server | |
|
||||
| REDIS_PREFIX | Namespace of the redis keys, when the instance is shared | yhub |
|
||||
| COLLABORATION_BACKEND_BASE_URL | Base url of the Docs backend, which answers who a user is and what they may do with a document | http://app-dev:8000 |
|
||||
| COLLABORATION_SERVER_ORIGIN | Comma separated list of the origins allowed to open a websocket | http://localhost:3000 |
|
||||
|
||||
@@ -62,7 +62,7 @@ services:
|
||||
|
||||
# The collaboration server: it serves everything under /collaboration/, the
|
||||
# websocket included, and holds the content of the documents. Its schema is
|
||||
# not created when it starts — run `docker compose run --rm yhub npm run
|
||||
# not created when it starts — run `docker compose run --rm yhub yarn
|
||||
# init-db` once before the first start, and again after every upgrade.
|
||||
yhub:
|
||||
image: lasuite/impress-yhub:latest
|
||||
|
||||
@@ -260,7 +260,7 @@ The collaboration server never runs DDL itself, so its schema has to be created
|
||||
before it starts:
|
||||
|
||||
```bash
|
||||
docker compose run --rm yhub npm run init-db
|
||||
docker compose run --rm yhub yarn init-db
|
||||
```
|
||||
|
||||
It creates the `yhub` database when it is missing, and every table the version
|
||||
@@ -319,7 +319,7 @@ docker compose run --rm backend python manage.py migrate
|
||||
```
|
||||
and the collaboration server's, which is the same command as at install time:
|
||||
```bash
|
||||
docker compose run --rm yhub npm run init-db
|
||||
docker compose run --rm yhub yarn init-db
|
||||
```
|
||||
|
||||
## Upgrading from a release without the collaboration server
|
||||
|
||||
@@ -133,7 +133,7 @@ The collaboration server needs a key of its own (`YHUB_JWT_PRIVATE_KEY`), the
|
||||
backend it calls (`COLLABORATION_BACKEND_BASE_URL`), the origins a browser may
|
||||
open a websocket from (`COLLABORATION_SERVER_ORIGIN`), and its two stores
|
||||
(`POSTGRES`, `REDIS`). Its schema is created by the script it ships,
|
||||
`npm run init-db`, which has to run once before it starts and again after every
|
||||
`yarn init-db`, which has to run once before it starts and again after every
|
||||
upgrade. Its variables are listed in [env.md](../env.md), and
|
||||
`src/yhub-server/README.md` documents what each of them changes.
|
||||
|
||||
@@ -250,7 +250,7 @@ scalingo logs --tail
|
||||
3. **Static files not served**: Ensure the buildpack post-frontend script ran successfully
|
||||
4. **OIDC errors**: Verify your OIDC provider configuration and callback URLs
|
||||
5. **Theme not updating**: Clear Redis cache with `scalingo run python -c "from django.core.cache import cache; cache.clear()"`
|
||||
6. **Collaboration not working, or documents opening empty**: the collaboration server is a separate deployment. Verify it is running, that `COLLABORATION_WS_URL` and `YHUB_API_BASE_URL` point at it, that `JWT_PRIVATE_KEY` is set on this app and `YHUB_JWT_PRIVATE_KEY` on that one, and that its `npm run init-db` has been run
|
||||
6. **Collaboration not working, or documents opening empty**: the collaboration server is a separate deployment. Verify it is running, that `COLLABORATION_WS_URL` and `YHUB_API_BASE_URL` point at it, that `JWT_PRIVATE_KEY` is set on this app and `YHUB_JWT_PRIVATE_KEY` on that one, and that its `yarn init-db` has been run
|
||||
7. **Conversion not working** (export, markdown import): verify the y-provider process is running and `Y_PROVIDER_API_BASE_URL` points at it
|
||||
|
||||
### Useful Commands
|
||||
|
||||
@@ -51,7 +51,7 @@ spec:
|
||||
command:
|
||||
{{- toYaml .Values.yhub.initDb.command | nindent 12 }}
|
||||
{{- else }}
|
||||
# The script yhub ships, wrapped as `npm run init-db`, retried until
|
||||
# The script yhub ships, wrapped as `yarn init-db`, retried until
|
||||
# the postgres server answers: nothing here creates it, and a chart
|
||||
# sync does not wait for whatever does. Retrying the whole script
|
||||
# rather than probing the port first — it is idempotent, so a run
|
||||
|
||||
@@ -5,15 +5,15 @@ WORKDIR /app
|
||||
|
||||
# built from the repository root, like every other image here — the entrypoint
|
||||
# below lives outside this directory
|
||||
COPY ./src/yhub-server/package.json ./src/yhub-server/package-lock.json ./
|
||||
COPY ./src/yhub-server/package.json ./src/yhub-server/yarn.lock ./
|
||||
|
||||
|
||||
# ---- Development image ----
|
||||
FROM base AS yhub-development
|
||||
|
||||
# dev dependencies included: nodemon, plus this is where one-off scripts run
|
||||
# (`make migrate-yhub` runs `npm run init-db` in it)
|
||||
RUN npm ci
|
||||
# (`make migrate-yhub` runs `yarn init-db` in it)
|
||||
RUN yarn install --frozen-lockfile
|
||||
|
||||
# server.js, migration.js, env.js — glob so a new module cannot be forgotten.
|
||||
# compose bind-mounts the sources over /app on top of this copy, so an edit on
|
||||
@@ -22,17 +22,17 @@ COPY ./src/yhub-server/*.js ./
|
||||
|
||||
EXPOSE 3002
|
||||
|
||||
# `npm run dev` restarts the server on every source change, no rebuild needed.
|
||||
# `yarn dev` restarts the server on every source change, no rebuild needed.
|
||||
# nodemon rather than node's own --watch: the latter watches inodes, so it goes
|
||||
# deaf as soon as a file is replaced by a rename — which is what `git checkout`
|
||||
# and most editors do when saving.
|
||||
CMD ["npm", "run", "dev"]
|
||||
CMD ["yarn", "dev"]
|
||||
|
||||
|
||||
# ---- Production image ----
|
||||
FROM base AS yhub
|
||||
|
||||
RUN npm ci --omit=dev
|
||||
RUN yarn install --frozen-lockfile --production
|
||||
|
||||
COPY ./src/yhub-server/*.js ./
|
||||
|
||||
|
||||
@@ -101,7 +101,7 @@ document, and yhub enforces every facet itself — on the websocket and on the
|
||||
REST routes alike. Docs' whole policy is three tables in `permissions.js`, kept
|
||||
out of `server.js` so they can be read and tested without redis and postgres.
|
||||
`__tests__/permissions.test.js` asks them the same questions yhub's gates ask;
|
||||
run it with `npm test` (see "Tests" below).
|
||||
run it with `yarn test` (see "Tests" below).
|
||||
|
||||
Masks are positional `crud` strings where `-` denies, so `'-r--'` is read-only.
|
||||
|
||||
@@ -382,12 +382,12 @@ from an `mc` container on the stack's network.
|
||||
|
||||
## Tests
|
||||
|
||||
`npm test` runs two suites, neither of which needs redis, postgres or S3:
|
||||
`yarn test` runs two suites, neither of which needs redis, postgres or S3:
|
||||
|
||||
- `__tests__/permissions.test.js` on node's own runner (`node:test`) — it
|
||||
imports `@y/hub/permissions` (a subpath export, no redis/postgres pulled in)
|
||||
to run the real permission pipeline, and is kept on `node:test` on purpose,
|
||||
- the `__tests__/*.spec.mjs` files on **vitest** (`npm run test:watch` for the
|
||||
- the `__tests__/*.spec.mjs` files on **vitest** (`yarn test:watch` for the
|
||||
watcher):
|
||||
- `__tests__/migration.spec.mjs` drives `maybeMigrate` and `fullMigrate` end
|
||||
to end with `@aws-sdk/client-s3` and the yhub instance faked and `@y/y`
|
||||
@@ -402,7 +402,7 @@ from an `mc` container on the stack's network.
|
||||
|
||||
The `__tests__/` directory (and the `.mjs` extension) keeps the specs out of
|
||||
`node --test`'s discovery and out of the Docker image (`COPY *.js`); shared
|
||||
fakes live in `__tests__/_helpers.mjs`. CI does not run these yet — it installs with `npm ci --omit=dev`, so `vitest` is absent
|
||||
fakes live in `__tests__/_helpers.mjs`. CI does not run these yet — it installs with `yarn install --frozen-lockfile --production`, so `vitest` is absent
|
||||
there; the pytest-driven integration suite in `.github/workflows/impress.yml` is
|
||||
what exercises a real collaboration server.
|
||||
|
||||
@@ -412,7 +412,7 @@ The `Dockerfile` has two final stages, like the other services of this
|
||||
repository:
|
||||
|
||||
- `yhub-development` — what the `yhub` service of `compose.yml` builds. It
|
||||
installs the dev dependencies and starts the server through `npm run dev`
|
||||
installs the dev dependencies and starts the server through `yarn dev`
|
||||
(nodemon), and compose bind-mounts `src/yhub-server` over `/app`: **editing
|
||||
`server.js`, `migration.js` or `env.js` restarts the server, no rebuild**.
|
||||
Watch it happen with `docker compose logs -f yhub`. A syntax error stops at
|
||||
@@ -444,11 +444,11 @@ make build-yhub
|
||||
docker compose up -d --force-recreate --renew-anon-volumes yhub
|
||||
```
|
||||
|
||||
## Database schema (`npm run init-db`)
|
||||
## Database schema (`yarn init-db`)
|
||||
|
||||
yhub never runs DDL from the server or the worker, so the schema is created by
|
||||
the script it ships (`node_modules/@y/hub/bin/init-db.js`), wrapped here as
|
||||
`npm run init-db`. It reads `POSTGRES` from the environment, creates the
|
||||
`yarn init-db`. It reads `POSTGRES` from the environment, creates the
|
||||
database when it does not exist, then every table and index the **installed**
|
||||
yhub version needs. It is idempotent, so re-running it is always safe.
|
||||
|
||||
|
||||
Generated
-3020
File diff suppressed because it is too large
Load Diff
@@ -21,5 +21,6 @@
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=22"
|
||||
}
|
||||
},
|
||||
"packageManager": "yarn@1.22.22"
|
||||
}
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user