WEBDIENSTE/openswarm
1
0
Fork 0
mirror of https://github.com/openswarm-ai/openswarm.git synced 2026-09-28 04:24:51 +02:00
Code Issues Packages Projects Releases Wiki Activity

arena: v48 plan-state fair pilot MARGINAL (5 vs 4 strict, -partial, within noise); bigger 65-task resolving test launched; corrected too-loose auto-verdict

Browse Source
This commit is contained in:
ciregenz
2026-08-17 17:01:37 -07:00
parent 2529aa3772
commit 8f4a25ff39
1 changed files with 10 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
e2e/browser-v3/arena/ARENA.md
+10
View File
@@ -1123,3 +1123,13 @@ tasks to lift; booking 'memory flat' off it would have been dishonest. Rebuilt a
near-miss (partial-credit) tasks + 4 strict controls where v42 baseline = 11.8 partial-sum / 4
strict -- tasks with genuine room for a completion mechanism to push 0.5->1.0. v48 running on it
now; THAT result is the decisive, honest test of whether plan-state moves completion.
## v48 plan-state: fair pilot = MARGINAL/inconclusive (2026-08-17)
Fair near-miss pilot (20 tasks, v42 base 4 strict/11.8 partial): v48 = 5 strict / 9.3 partial.
+1 strict but -2.5 partial-sum -- WITHIN NOISE at n=20, NOT a clear win (my auto-verdict script's
threshold was too loose; corrected). Plan-state is the ONLY one of four capability mechanisms
(v45/v46/v47/v48) to nudge strict up at all, so worth resolving: launched a bigger test on all 65
v42-informative tasks (v42 base 13 strict / 38.2 partial). If v48 clearly beats that -> real
completion lift, ingest. If flat/within-noise -> completion wall confirmed not scaffolding-movable,
book the full negative ledger. Honest status: no mechanism has clearly broken the WebArena
completion wall; plan-state is the one open thread.