mirror of https://github.com/openswarm-ai/openswarm.git synced 2026-08-26 14:32:22 +02:00

Files

T

ciregenzandClaude Fable 5 b43123ae20 arena: v20 champion 86.4 + public-baseline positioning -- ~12 points above published generic agents

User-supplied 2026 survey gives the right comparison class: generic-harness MiniWoB
tops out at 71.5 (GPT-5) / 74.9 (best harness). v20's 86.4 zero-tuning single-run sits
far above it, and our +60-of-harness vs +7-of-model finding reproduces the survey's
Orby insight at scale. MiniWoB demoted to regression-suite status per the survey
rubric; Fable 5 sweep launched to complete the first known Claude-5 triplet and answer
model-limited-vs-harness-limited directly.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ

2026-08-12 14:06:36 -07:00

browser-v3

arena: v20 champion 86.4 + public-baseline positioning -- ~12 points above published generic agents

2026-08-12 14:06:36 -07:00

helpers

[eric] e2e: expose the redux store deterministically via webPreferences.additionalArguments, fix visibility capture (read the chromium trace from tracingComplete, resolve click targets to the control, drop the dead contextBridge ipc wrap), harden onboarding/settings/combinatorial specs and add multi-window-stress

2026-05-28 21:41:01 -07:00

mcp

[eric] refactor: trim the verifier and mcp comments to one line each per the updated convention

2026-05-27 19:11:12 -07:00

tests

[eric] notes: delete the sticky-note feature end to end

2026-07-29 20:32:24 -07:00

.gitignore

[eric] test: drive the real packaged app on mac + windows to catch breakage early

2026-05-27 10:19:53 -07:00

package-lock.json

[eric] build: add the mcp sdk dep plus lockfile the gui hand needs

2026-05-27 19:11:11 -07:00

package.json

[eric] build: add the mcp sdk dep plus lockfile the gui hand needs

2026-05-27 19:11:11 -07:00

playwright.config.ts

[eric] test: drive the real packaged app on mac + windows to catch breakage early

2026-05-27 10:19:53 -07:00

README.md

[eric] test: drive the real packaged app on mac + windows to catch breakage early

2026-05-27 10:19:53 -07:00

README.md

End-to-end tests (packaged app, macOS + Windows)

Playwright tests that launch the packaged OpenSwarm desktop app (the real built binary, asar + bundled python-env + real paths) and drive it the way a user would. The same specs run unchanged on macOS and Windows; CI builds the artifact per-OS, then runs these. No provider API key is needed (no agent turn), so the suite is hermetic and deterministic on a clean machine.

What it checks (per OS)

Main window paints the React shell (first meaningful paint).
The preload bridge (window.openswarm) is exposed.
The real backend the app spawned reaches HTTP-ready (/api/health/check -> 200).
Provenance: the running app's getBuildInfo() sha matches electron/build-info.json.
App version is reported.

Run locally

Build the app first (produces electron/dist/...):
- Windows: pwsh scripts/build-app-win.ps1
- macOS: bash scripts/build-app.sh

Then:

cd e2e
PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD=1 npm ci   # Electron ships its own Chromium
npm test

Override the binary location with E2E_APP_PATH=/path/to/app if your build output lives elsewhere. Auto-detection covers win-unpacked/OpenSwarm.exe and the mac OpenSwarm.app variants.

CI

.github/workflows/e2e.yml runs this on a windows-latest + macos-latest matrix: it builds the unsigned app, then runs the suite. Tag-driven signed releases are covered separately by release-windows.yml / release-macos.yml.