WEBDIENSTE/openswarm
1
0
Fork 0
mirror of https://github.com/openswarm-ai/openswarm.git synced 2026-08-17 18:25:42 +02:00
Code Issues Packages Projects Releases Wiki Activity

arena: pre-register v38/v39 pilots

Browse Source 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
This commit is contained in:
ciregenz
2026-08-16 01:30:31 -07:00
co-authored by Claude Fable 5
parent 2e07047ff7
commit 0e2aac1afc
1 changed files with 14 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
e2e/browser-v3/arena/ARENA.md
+14
View File
@@ -595,6 +595,20 @@ VERDICTS (2026-08-16):
don't need durable memory; the REAL test is chore-scale aggregation, which needs an uncapped
chore rerun (expensive, deferred). Flag stays off in champion until that evidence exists.
## PRE-REGISTERED (2026-08-16, before any v38/v39 episode): remaining Tier-1 pilots
v38 TABLE->MARKDOWN pilot — targets (8): read-table, read-table-2, social-media, book-flight
pair, order-food, stock-market, phone-book (table/list-heavy pages). Prediction: >=2/8 flip
(read-table family + one flight/list task), controls 12/12 (no table on the page -> block
renders nothing).
v39 MUTATION-DIFF pilot — targets (8): book-flight pair, use-autocomplete pair, choose-list,
form-sequence-3, search-engine, click-menu (pages that REACT to actions with popups/new rows).
Prediction: >=2/8 flip (autocomplete family especially), controls 12/12 (delta line is small and
only appears when the page actually changed).
Order: v38 then v39, sequential, auto-chained.
## Open-source / SOTA reference points (2026 leaderboards, for honest comparison)
- WebArena: SOTA WebTactix/DeepSeek-v3.2 74.3%; frontier models 64-68%; human 78. (leaderboard.steel.dev)