From 1db9f85f1fc43ef96985fbd189720484608c0b6c Mon Sep 17 00:00:00 2001 From: ciregenz Date: Wed, 12 Aug 2026 05:43:11 -0700 Subject: [PATCH] arena: v17 takes the single-run crown -- 84.0%, every episode clean, still zero false claims Per-task process isolation recovered the entire playwright-wedge class and mechanical fill-verify held the targeted text/email categories; the 20 remaining losses collapse onto exactly the four engineering clusters named in the plan (long forms, pixel precision, console emulation, stateful games). Seed-43 sweep running as the cross-seed generalization check. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ --- e2e/browser-v3/arena/ARENA.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/e2e/browser-v3/arena/ARENA.md b/e2e/browser-v3/arena/ARENA.md index fc88553d..3a7398d6 100644 --- a/e2e/browser-v3/arena/ARENA.md +++ b/e2e/browser-v3/arena/ARENA.md @@ -46,6 +46,11 @@ it — the constraint is flow competence, not steps. | sonnet-5 | 76.0% @ 5.5s, 0 false | 63% running @ 42s | their loop DEGRADES on the newest model | | **opus-5** | **82.4% @ 6.3s, 0 false** (v14); v15 81.6; v16 82.1 @ 9.5s | 66.4% @ 28.3s, 2 false (clean re-run; replaces tainted 48.0%*) | ours leads +16 pts at 4.5x speed | +**v17 (isolation + mechanical fill-verify) FINAL: 84.0% (105/125) @ 9.6s, 0 false -- new +single-run champion**, all 125 clean (one process per task killed the copy-paste wedge class). +Its 20 losses are now almost purely the four named engineering clusters: long transactional +forms (6), pixel-precision spatial (5), console/editor emulation, and stateful games. + v16 (verify-terminal, look-act-look, rapid-fire, sub-step confirm) FINAL: 82.1% on clean episodes -- statistically tied with v14, but the fixes hit their targets: email 10/10 (their best 6/10), text_entry 16/17, click_basic 12/13. The wins moved WHERE predicted while the untouched hard