mirror of https://github.com/openswarm-ai/openswarm.git synced 2026-08-22 20:52:23 +02:00

Files

T

ciregenzandClaude Fable 5 bfd8920570 arena: opus-5 grid -- ours 82.4% (union 85.6%), browser-use DEGRADES on newer models

v14-opus5 82.4% at 6.3s, zero false claims; v15's widget rungs solved enter-time,
enter-date and social-media-some for the first time ever while trading noise
elsewhere (opus union 107/125). browser-use on sonnet-5 is running 63% at 42s --
worse than its sonnet-4-6 74.4%: their loop does not scale with the model, ours does.
Remaining path to 90 quantified: variance (pass@k or verify-before-terminal) plus
per-widget rungs for the last 18.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ

2026-08-10 18:51:44 -07:00

browser-v3

arena: opus-5 grid -- ours 82.4% (union 85.6%), browser-use DEGRADES on newer models

2026-08-10 18:51:44 -07:00

helpers

[eric] e2e: expose the redux store deterministically via webPreferences.additionalArguments, fix visibility capture (read the chromium trace from tracingComplete, resolve click targets to the control, drop the dead contextBridge ipc wrap), harden onboarding/settings/combinatorial specs and add multi-window-stress

2026-05-28 21:41:01 -07:00

mcp

[eric] refactor: trim the verifier and mcp comments to one line each per the updated convention

2026-05-27 19:11:12 -07:00

tests

[eric] notes: delete the sticky-note feature end to end

2026-07-29 20:32:24 -07:00

.gitignore

[eric] test: drive the real packaged app on mac + windows to catch breakage early

2026-05-27 10:19:53 -07:00

package-lock.json

[eric] build: add the mcp sdk dep plus lockfile the gui hand needs

2026-05-27 19:11:11 -07:00

package.json

[eric] build: add the mcp sdk dep plus lockfile the gui hand needs

2026-05-27 19:11:11 -07:00

playwright.config.ts

[eric] test: drive the real packaged app on mac + windows to catch breakage early

2026-05-27 10:19:53 -07:00

README.md

[eric] test: drive the real packaged app on mac + windows to catch breakage early

2026-05-27 10:19:53 -07:00

README.md

End-to-end tests (packaged app, macOS + Windows)

Playwright tests that launch the packaged OpenSwarm desktop app (the real built binary, asar + bundled python-env + real paths) and drive it the way a user would. The same specs run unchanged on macOS and Windows; CI builds the artifact per-OS, then runs these. No provider API key is needed (no agent turn), so the suite is hermetic and deterministic on a clean machine.

What it checks (per OS)

Main window paints the React shell (first meaningful paint).
The preload bridge (window.openswarm) is exposed.
The real backend the app spawned reaches HTTP-ready (/api/health/check -> 200).
Provenance: the running app's getBuildInfo() sha matches electron/build-info.json.
App version is reported.

Run locally

Build the app first (produces electron/dist/...):
- Windows: pwsh scripts/build-app-win.ps1
- macOS: bash scripts/build-app.sh

Then:

cd e2e
PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD=1 npm ci   # Electron ships its own Chromium
npm test

Override the binary location with E2E_APP_PATH=/path/to/app if your build output lives elsewhere. Auto-detection covers win-unpacked/OpenSwarm.exe and the mac OpenSwarm.app variants.

CI

.github/workflows/e2e.yml runs this on a windows-latest + macos-latest matrix: it builds the unsigned app, then runs the suite. Tag-driven signed releases are covered separately by release-windows.yml / release-macos.yml.