WEBDIENSTE/openswarm
1
0
Fork 0
mirror of https://github.com/openswarm-ai/openswarm.git synced 2026-09-22 01:24:52 +02:00
Code Issues Packages Projects Releases Wiki Activity

arena: CompWoB spot-check -- no paren-bug suppression (3/12 same as v34); 81.1 stands

Browse Source
This commit is contained in:
ciregenz
2026-08-16 19:18:33 -07:00
parent c44308ed94
commit bc225d11ae
1 changed files with 8 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
e2e/browser-v3/arena/ARENA.md
+8
View File
@@ -1005,3 +1005,11 @@ score-inflation lever, not a measurement. These
stay SUPPLEMENTARY, never a primary goal clause (the 11 clauses are deterministic, judge-free).
Note: Polar's gemini-self-judge + 'stricter than others' claim is the same vendor-picks-own-grader
pattern we avoid.
## CompWoB suppression spot-check (2026-08-16): no suppression, 81.1 stands
Reran 12 bu-edge CompWoB tasks under v41 (has the quote-aware parser fix). 3/12 won -- IDENTICAL
to v34. The paren-parser bug did NOT suppress CompWoB (its goals are MiniWoB-style with short
answers, no long JSON answer channel where parens appeared). 81.1 stands honest, no correction.
The instrument bug was WebArena-specific (that benchmark appends a JSON answer schema + long
answers with parens).