From 8f4a25ff390bc5a93c5b29d9c9d56643b7a83c10 Mon Sep 17 00:00:00 2001 From: ciregenz Date: Mon, 17 Aug 2026 17:01:37 -0700 Subject: [PATCH] arena: v48 plan-state fair pilot MARGINAL (5 vs 4 strict, -partial, within noise); bigger 65-task resolving test launched; corrected too-loose auto-verdict --- e2e/browser-v3/arena/ARENA.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/e2e/browser-v3/arena/ARENA.md b/e2e/browser-v3/arena/ARENA.md index a46215a0..c2c304cc 100644 --- a/e2e/browser-v3/arena/ARENA.md +++ b/e2e/browser-v3/arena/ARENA.md @@ -1123,3 +1123,13 @@ tasks to lift; booking 'memory flat' off it would have been dishonest. Rebuilt a near-miss (partial-credit) tasks + 4 strict controls where v42 baseline = 11.8 partial-sum / 4 strict -- tasks with genuine room for a completion mechanism to push 0.5->1.0. v48 running on it now; THAT result is the decisive, honest test of whether plan-state moves completion. + +## v48 plan-state: fair pilot = MARGINAL/inconclusive (2026-08-17) +Fair near-miss pilot (20 tasks, v42 base 4 strict/11.8 partial): v48 = 5 strict / 9.3 partial. ++1 strict but -2.5 partial-sum -- WITHIN NOISE at n=20, NOT a clear win (my auto-verdict script's +threshold was too loose; corrected). Plan-state is the ONLY one of four capability mechanisms +(v45/v46/v47/v48) to nudge strict up at all, so worth resolving: launched a bigger test on all 65 +v42-informative tasks (v42 base 13 strict / 38.2 partial). If v48 clearly beats that -> real +completion lift, ingest. If flat/within-noise -> completion wall confirmed not scaffolding-movable, +book the full negative ledger. Honest status: no mechanism has clearly broken the WebArena +completion wall; plan-state is the one open thread.