mirror of
https://github.com/openswarm-ai/openswarm.git
synced 2026-08-26 22:42:23 +02:00
ciregenzandClaude Opus 5
56c113462a
Added to compare browser-use against us with the BROWSER held constant, and the
result is that it cannot be held constant, which is worth writing down.
What works: with the port exposed, browser-use attaches, sees the browser cards
(they are CDP targets of type 'webview'), navigates one, and completes a task that
verifies independently. Cards persist across restarts because they live in the saved
dashboard layout, so there is something to attach to.
What does not: browser-use's screenshot path cannot capture a webview and falls back
to the app's own 'page' target -- 49 fallbacks in 49 agent steps, i.e. it drove the
web page while LOOKING at the OpenSwarm UI for every single step. A comparison run
that way handicaps the rival for an infrastructural reason and measures neither
agent, so the shared-browser numbers are not publishable as a fair head-to-head.
Two hazards found the hard way, both worth knowing before anyone repeats this:
- browser-use filters CDP targets to page/tab, so 'webview' is invisible to it, and
Electron refuses Target.createTarget ("Not supported") so it cannot open its own.
Pointed at Electron unrestricted it therefore grabs the app's page target and
navigates the OpenSwarm UI away, killing the renderer: the next sweep failed
infra_no_renderer 1/1 because of exactly that.
- An external agent must never be allowed to kill the shared session. Doing so after
trial 1 made trials 2-15 fail ConnectError, which a naive summary published as
"browser-use REACH 4/15" -- ten rows of my own teardown scored against the agent.
Off by default: an open debugging port is a local attack surface, and this only
exists for measurement.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
29 lines
1.8 KiB
Bash
Executable File
29 lines
1.8 KiB
Bash
Executable File
#!/bin/bash
|
|
# Keep an Electron renderer alive for the length of a sweep.
|
|
#
|
|
# A 45-run sweep takes over an hour and the app does not reliably survive it: one run quit partway
|
|
# through and 24 of 45 rows came back "dispatch refused: no dashboard", which scored as a 29% reach
|
|
# that measured nothing but my own dead window. Relaunching by hand between runs is not a fix, it
|
|
# just moves the gap to whenever I am not looking.
|
|
#
|
|
# Restarts on exit, and waits for webpack first: Electron launched against a dead :3026 loads a
|
|
# blank page and quits immediately, which is how the window went missing the first time.
|
|
# Where logs, profiles and run output go. Defaults to runs/ beside this harness; override with
|
|
# OSW_BENCH_DIR to keep multi-gigabyte browser profiles off the repo disk.
|
|
SP="${OSW_BENCH_DIR:-$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/runs}"
|
|
mkdir -p "$SP"
|
|
cd "$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)/electron" || exit 1
|
|
|
|
while true; do
|
|
until curl -s -o /dev/null --max-time 4 "http://localhost:${OPENSWARM_DEV_PORT:-3026}"; do sleep 5; done
|
|
# OSW_CDP_PORT exposes Electron's DevTools protocol so an EXTERNAL agent can be pointed at the very
|
|
# same browser our own agent drives. Without it a cross-agent comparison runs one side in Electron
|
|
# and the other in vanilla Chrome, and then every difference is confounded by the browser rather
|
|
# than the agent. Off unless asked for: an open debugging port is a local attack surface.
|
|
ELECTRON_DEV=1 OPENSWARM_DEV_PORT="${OPENSWARM_DEV_PORT:-3026}" OPENSWARM_PORT="${OPENSWARM_PORT:-8326}" \
|
|
./node_modules/.bin/electron . --user-data-dir="$SP/udd" \
|
|
${OSW_CDP_PORT:+--remote-debugging-port=$OSW_CDP_PORT} >> "$SP/renderer.log" 2>&1
|
|
echo "[keep_renderer] electron exited at $(date +%H:%M:%S), restarting" >> "$SP/renderer.log"
|
|
sleep 5
|
|
done
|