mirror of https://github.com/openswarm-ai/openswarm.git synced 2026-08-23 13:02:23 +02:00

Files

T

ciregenzandClaude Fable 5 bcbb97d657 arena: definitive close -- v10 beats the real browser-use 75.2 to 69.6 at 8.5x speed, 0 false claims

Twelve versions, every technique measured on all 125 tasks, negative results recorded
alongside the wins (chain-split -4.7, nudge inside noise, 36-step runway bought
nothing). Perception was most of the gap: subtree-text names alone were +12 points.
Category ledger vs their whole stack: lead 5, tie 3, behind 1 (forms -- long widget
flows, needs a scripted autocomplete primitive, not more steps). Product-port list
staged in priority order with each item's measured arena win attached.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ

2026-08-10 07:32:19 -07:00

browser-v3

arena: definitive close -- v10 beats the real browser-use 75.2 to 69.6 at 8.5x speed, 0 false claims

2026-08-10 07:32:19 -07:00

helpers

[eric] e2e: expose the redux store deterministically via webPreferences.additionalArguments, fix visibility capture (read the chromium trace from tracingComplete, resolve click targets to the control, drop the dead contextBridge ipc wrap), harden onboarding/settings/combinatorial specs and add multi-window-stress

2026-05-28 21:41:01 -07:00

mcp

[eric] refactor: trim the verifier and mcp comments to one line each per the updated convention

2026-05-27 19:11:12 -07:00

tests

[eric] notes: delete the sticky-note feature end to end

2026-07-29 20:32:24 -07:00

.gitignore

[eric] test: drive the real packaged app on mac + windows to catch breakage early

2026-05-27 10:19:53 -07:00

package-lock.json

[eric] build: add the mcp sdk dep plus lockfile the gui hand needs

2026-05-27 19:11:11 -07:00

package.json

[eric] build: add the mcp sdk dep plus lockfile the gui hand needs

2026-05-27 19:11:11 -07:00

playwright.config.ts

[eric] test: drive the real packaged app on mac + windows to catch breakage early

2026-05-27 10:19:53 -07:00

README.md

[eric] test: drive the real packaged app on mac + windows to catch breakage early

2026-05-27 10:19:53 -07:00

README.md

End-to-end tests (packaged app, macOS + Windows)

Playwright tests that launch the packaged OpenSwarm desktop app (the real built binary, asar + bundled python-env + real paths) and drive it the way a user would. The same specs run unchanged on macOS and Windows; CI builds the artifact per-OS, then runs these. No provider API key is needed (no agent turn), so the suite is hermetic and deterministic on a clean machine.

What it checks (per OS)

Main window paints the React shell (first meaningful paint).
The preload bridge (window.openswarm) is exposed.
The real backend the app spawned reaches HTTP-ready (/api/health/check -> 200).
Provenance: the running app's getBuildInfo() sha matches electron/build-info.json.
App version is reported.

Run locally

Build the app first (produces electron/dist/...):
- Windows: pwsh scripts/build-app-win.ps1
- macOS: bash scripts/build-app.sh

Then:

cd e2e
PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD=1 npm ci   # Electron ships its own Chromium
npm test

Override the binary location with E2E_APP_PATH=/path/to/app if your build output lives elsewhere. Auto-detection covers win-unpacked/OpenSwarm.exe and the mac OpenSwarm.app variants.

CI

.github/workflows/e2e.yml runs this on a windows-latest + macos-latest matrix: it builds the unsigned app, then runs the suite. Tag-driven signed releases are covered separately by release-windows.yml / release-macos.yml.