v3 (44.8%, beats the browser-use-shaped arm's 36.0% by 8.8 points at 3x the speed):
page-text panel, select options on the row, input twins exempt from dedupe. v4 queued:
DOM-attribute names for nameless icons and deep sibling context, which turn the email
suite's unlabeled trash/star icons into addressable rows.
Product fix landed with tests: interactiveRanking.ts collapsed adjacent same-named
inputs and adjacent nameless icons as twins -- measured on MiniWoB, that single
collapse cost enter-password and the whole email suite. Inputs and nameless rows are
now exempt; named icon+label pairs still collapse. 17/17 tests, tsc clean.
ARENA.md carries the full scoreboard, per-category verdicts, what was ingested from
whom, and the honest gaps (drag and vision-less spatial) neither stack solves.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
Read browser-use's own DOM code (dom/service.py,
is_element_visible_according_to_all_parents) and it decides visibility on CSS
alone: display, visibility, opacity, plus bounds existence and a viewport
intersection with 1000px of slack. There is NO minimum size gate anywhere.
Ours had `r.width >= 80 && r.height >= 16`, and that floor was the entire reason
ACE / CodeMirror-5 / Monaco composers were unreachable: their real input is a ~1x1
offscreen textarea that paints into a sibling div, so it fails a size test by
construction while being the correct fill target. Measured: w3schools and onlinegdb
scored 0/3 while every contenteditable editor (CodeMirror 6, Quill, TinyMCE,
CKEditor) passed -- the input MECHANISM split the suite, not the site.
This replaces the proxyHost patch from the previous commit, which reached the same
sites by walking up to a composer-sized ancestor. Their rule is simpler, more
general, and needs no ancestor heuristic, so proxyHost is deleted rather than kept
alongside. The floor was also doing less work than it looked: junk is excluded by
requiring a rich role or a nearby submit AND score >= 2, and a honeypot is
display:none / opacity:0, which still fails.
Measured on the merged tree, same lane (OSW_MODEL=opus-4-8-cc):
- anon w3schools 3/3, i.e. ACE still reached with proxyHost gone
- holdout 18/20 = 90%, byte-identical to before: no regression
- other_ms unchanged at 1211ms
deepl regressed to not_measurable in the same sweep and it is NOT this change:
A/B'd by restoring the old floor and re-running, it fails identically. deepl now
redirects to /en/translator/l/en/pt-BR and serves one textbox; the redirect predates
the change in the logs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Six of the eleven defects here were in the MEASUREMENT, not the product, and they
were wrong in both directions.
Harness, all of which silently produced wrong numbers:
- coverage.py preflight refused every sweep on a box holding exactly one backend:
stack.sh's supervisor is a `bash -c` quoting the whole uvicorn line, so it carries
both "-m uvicorn backend.main" AND the venv python path. Discriminate on POSITION.
- stack.sh status reported 2 backends over 1 and 0 webpack over a live dev server
(webpack retitles its process). A status check whose job is preventing a second
stack, failing in the direction that lets one land.
- c7_run.sh/c8_run.sh slice r6_be.log while stack.sh names logs by TAG: a stack under
any other tag hands every trial an empty slice and the sweep reports a confident
0/108. Now refuses loudly; it caught this exact mistake on first use.
- "Browser command timed out" was bucketed infra. It is ONE command blowing its own
budget, not a dead webview: all 4 such rows were BrowserFindComposer at exactly its
30s cap, every run completed after, zero card-gone markers in the whole log. Filed
as infra it read as 11.8% flake AND lifted holdout reach 70% -> 84%.
- api_retry / rate_limit_error now grade as infra. A provider 429 storm turned clean
15-21s exclusions into 188s product_no_composer rows.
- bench.py prints reach BOTH ways when a row is UNVERIFIED. An exclusion resting on
the agent's own word quietly flatters the score, and coverage.py's own instruction
to confirm it by hand goes unread (I quoted a 100% that excluded onlinegdb).
Timing was measuring 0.2% of the run: prestage completes BEFORE metrics_started_at,
so other_ms was 25ms of a 12700ms median while prestage (4146ms, ~61%) sat in no
bucket at all. prestage_ms/task_ms are now recorded; total_ms is deliberately NOT
redefined, which would invalidate every before/after already taken against it.
Product:
- find_composer rejected ACE/CodeMirror-5/Monaco composers. Their input is a ~1x1
offscreen textarea that paints into a sibling div, so it can never pass a size
gate. Accept it when a VISIBLE ancestor is composer-sized; honeypots stay out
because the input itself must not be display:none/visibility:hidden/opacity:0.
anon reach 80% -> 100%, holdout 89% -> 90%, p95 38.6s -> 9.7s.
- the composer poll slept a blind 0+1.2+1.4 = 2.6s whenever prestage staged nothing,
which is nearly every run, and it was the whole of other_ms's suspicious constancy
(2610-2613ms regardless of tools_ms). Stop when two reads are identical, the rule
the opener poll 40 lines below already applies. other_ms -53.7%, tools_ms flat.
- prestage no longer navigates to the page it is already on, nor sleeps 0.35s before
its first settle probe.
- is_replay_boundary reasoned from the NAME alone, so x.com's composer textbox named
"Post text" was ruled an irreversible send and truncated its replay to a bare
navigate. Excluded by ROLE; first_unsafe_step now passes role through at all.
Measured on this box: reach 100% (83% if onlinegdb's unverified exclusion is bogus),
0 false successes in ~155 runs, prestage tier-0/1 2702ms, other_ms -53.7%, infra
flake 0/158, holdout 18/20. Criteria 2/4/9 need live writes and are untouched.
Full evidence, including what did NOT work, in e2e/browser-v3/RESULTS_2026-08-06.md.
Also drops the tracked electron/node_modules symlink pointing at another machine's
Downloads folder; it is dangling on every other checkout and re-breaks the install on
any stash or checkout.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
