WEBDIENSTE/openswarm
1
0
Fork 0
mirror of https://github.com/openswarm-ai/openswarm.git synced 2026-09-10 11:47:43 +02:00
Code Issues Packages Projects Releases Wiki Activity
2,679 Commits 93 Branches 129 Tags
76eada698727eaacded1802f6bc804815f4431a4
Commit Graph
2679 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
ciregenzandClaude Fable 5 76eada6987 arena: v40 union FAILS confirm (interaction regression) -- pairwise isolation running (v40c first) 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
 2026-08-16 02:33:55 -07:00
ciregenzandClaude Fable 5 7581706f5f arena: v38 7/8 + v39 5/8 both PASS -- v40 champion = union of three passed ingestions; combined confirm pilot next 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
 2026-08-16 02:02:03 -07:00
ciregenzandClaude Fable 5 0e2aac1afc arena: pre-register v38/v39 pilots 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
 2026-08-16 01:30:31 -07:00
ciregenzandClaude Fable 5 2e07047ff7 arena: v36 PASSES (3/8 incl click-collapsible pair; first ingestion win) -> v40 champion; v37 deferred to chore-scale 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
 2026-08-16 01:30:09 -07:00
ciregenzandClaude Fable 5 25892f30b1 arena: WebChoreArena primary Claude column final -- 1/89 at 500s cap, 39 answers, 0 false; budget is the binding constraint (both models agree) 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
 2026-08-16 00:58:44 -07:00
ciregenzandClaude Fable 5 8574ab4d2a arena: pre-register v36/v37 ingestion pilots (targets + predictions) before any episode 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
 2026-08-15 23:01:32 -07:00
ciregenzandClaude Fable 5 3abe31fb01 arena: implement v39 mutation-diff action feedback (Agent-E) -- all four Tier-1 ingestions now flag-gated and ready to pilot 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
 2026-08-15 22:53:46 -07:00
ciregenzandClaude Fable 5 a3d2096982 arena: implement v38 table->markdown rendering (AgentOccam) -- offline-verified on read-table, flag-gated 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
 2026-08-15 22:22:33 -07:00
ciregenzandClaude Fable 5 6c245d3245 arena: implement v36 no_match refusal token (SeeAct) + v37 persistent NOTE scratchpad (AgentOccam/Agent-E), flag-gated, offline-verified -- pilots queued for lane-free 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
 2026-08-15 21:49:56 -07:00
ciregenzandClaude Fable 5 42ba2b897f arena: competitor ingestion ledger from AgentOccam/Agent-E/SeeAct study -- 4 Tier-1 generic candidates (escape token, note scratchpad, table-markdown, mutation-diff), pre-registered pilot-gated 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
 2026-08-15 21:47:10 -07:00
ciregenzandClaude Fable 5 3e55a95117 arena: 2026 open-source/SOTA reference points + browser-use library-vs-cloud distinction (honest comparison scope) 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
 2026-08-15 21:41:04 -07:00
ciregenzandClaude Fable 5 0c149a671d arena: WebChoreArena GPT-5.4 column final -- 2/86 solved at 500s cap, 0 false claims, upstream evaluator crashes disclosed 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
 2026-08-15 20:44:37 -07:00
ciregenzandClaude Fable 5 7dfd65cb00 arena: CORRECTION -- lane outage was a stale router OAuth token, not plan exhaustion (user dashboard evidence) 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
 2026-08-15 20:34:42 -07:00
ciregenzandClaude Fable 5 ee68667b26 arena: primary Claude v35 MiniWoB = 90.5% (v35 picker fix recovers choose-date; net-flat vs v22, +1.5 vs v34-buggy); residual >=95 cluster named 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
 2026-08-15 17:13:01 -07:00
ciregenzandClaude Fable 5 00172966ae arena: GPT-5.4 cross-model MiniWoB = 86.1% (same harness, ~3pt of Claude; harness-generality claim measured directly) 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
 2026-08-15 13:34:39 -07:00
ciregenzandClaude Fable 5 597931725c arena: book Claude-sub-exhausted -> GPT-5.4 cross-model column decision 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
 2026-08-15 11:30:14 -07:00
ciregenzandClaude Fable 5 5d6a14c936 arena: '@lane' arm alias for cross-model runs on a separate subscription (Claude sub exhausted; GPT-5.4 lane live, own data file) 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
 2026-08-15 11:29:46 -07:00
ciregenzandClaude Fable 5 1845e6d4c8 arena: WASP adapter design -- deterministic injection-refusal scoring (their ASR classifier excluded) 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
 2026-08-15 09:14:08 -07:00
ciregenzandClaude Fable 5 1bb3b0810a arena: v35 readonly-picker fallback (unit-verified 1.0 without a model) -- pre-registered for lane return 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
 2026-08-15 08:42:14 -07:00
ciregenzandClaude Fable 5 e6ecb83744 arena: bound 429 backoff to fit the step watchdog (flat 12s; unbounded waits booked as step_hang) 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
 2026-08-14 22:23:30 -07:00
ciregenzandClaude Fable 5 61af236e2a arena: router wraps 429s in 200 bodies -- unwrap so the rate-limit backoff branch fires 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
 2026-08-14 21:51:56 -07:00
ciregenzandClaude Fable 5 857a5185f9 arena: rate-limit-aware LLM backoff (20-100s on 429s, 5 attempts) -- TPM windows outwaited, not booked as infra 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
 2026-08-14 21:51:41 -07:00
ciregenzandClaude Fable 5 4b76028e90 arena: final-step answer turn -- Q&A goals get told the budget is up so they answer via chat (1/17 chore episodes had answered) 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
 2026-08-14 18:13:13 -07:00
ciregenzandClaude Fable 5 442f2e75ee arena: CompWoB champion v34 = 81.1% (delta rerun confirms flips) 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
 2026-08-14 16:08:25 -07:00
ciregenzandClaude Fable 5 02347b20cf arena: MiniWoB 3-seed on v32 stack = 89.0 (fix validated on social-media; readonly-picker named); net stack ledger booked 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
 2026-08-14 15:34:46 -07:00
ciregenzandClaude Fable 5 fa3a0fdae3 arena: v34 verdict -- named 1-step losses flipped as predicted; v34 champion config 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
 2026-08-14 13:29:05 -07:00
ciregenzandClaude Fable 5 4cdda1ccc0 arena: v33 verdict (fastpath is the reverse-task killer) + v34 inversion gate (pre-registered) 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
 2026-08-14 12:57:30 -07:00
ciregenzandClaude Fable 5 95fdc5fb1d arena: MiniWoB regression root-caused -- structural ctx labels displaced discriminating card text; uniqueness post-pass (labels must separate twins) + weak-name sibling filter 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
 2026-08-14 12:25:24 -07:00
ciregenzandClaude Fable 5 7985f741c7 arena: v33 opportunistic ordering (deferral doctrine + defer-nudge), pre-registered vs the 12-task bu-win cluster 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
 2026-08-14 10:45:58 -07:00
ciregenzandClaude Fable 5 8781b8079a arena: 2026 benchmark roadmap (REAL added; exclusions reasoned) 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
 2026-08-14 10:16:45 -07:00
ciregenzandClaude Fable 5 0993fd5d9f arena: CompWoB FINAL v32 = 77.9% (>=75 goal clause MET; was 65.3) + head-to-head vs browser-use + v33 staging 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
 2026-08-14 10:14:26 -07:00
ciregenzandClaude Fable 5 aee265abfe arena: v32 verdict -- first >=5-part win ever; CompWoB mechanism ladder closed at the page-validator boundary 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
 2026-08-14 07:38:19 -07:00
ciregenzandClaude Fable 5 82b02317fe arena: v31 rerun verdict (model won't self-remediate occlusion) + v32 forced dispatch on blocked clicks (pre-registered) 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
 2026-08-14 06:34:45 -07:00
ciregenzandClaude Fable 5 eeb62798a9 arena: v31 first run void -- blocker message truncated + stale bbox + empty-completion retries; all fixed and verified 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
 2026-08-14 06:02:37 -07:00
ciregenzandClaude Fable 5 b2c0d2be08 arena: v30 verdict (trap wrapper rows) + v31 wrapper suppression (pre-registered) + raw-reply capture 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
 2026-08-14 05:28:33 -07:00
ciregenzandClaude Fable 5 e87fd7545b arena: book v29 verdict + v30 pre-registration text 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
 2026-08-14 04:54:58 -07:00
ciregenzandClaude Fable 5 7f10e100a9 arena: v29 verdict (controls clean, targets 0/8 -- model now picks RIGHT element, click blocked by overlay) + v30 blocked-click intelligence (pre-registered) 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
 2026-08-14 04:54:30 -07:00
ciregenzandClaude Fable 5 60318f2c1f arena: WebChoreArena reddit partition (deterministic evals, their max_steps=50) 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
 2026-08-14 04:20:19 -07:00
ciregenzandClaude Fable 5 2aade59eb5 arena: v29 discriminative row context (DOM group labels + sibling groups) -- pre-registered pilot 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
 2026-08-14 04:19:52 -07:00
ciregenzandClaude Fable 5 6328e78772 arena: root cause of long-composition losses -- perception blindness (identical row contexts), not reasoning 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
 2026-08-14 03:42:10 -07:00
ciregenzandClaude Fable 5 bf3b78fecc arena: v28 verdict (fail; both loop-diff halves dead) + StepRecord.target instrumentation 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
 2026-08-14 03:38:01 -07:00
ciregenzandClaude Fable 5 331fc738d9 arena: webchorearena routing (isolated chore-venv, their evaluator; canary green) 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
 2026-08-14 03:05:24 -07:00
ciregenzandClaude Fable 5 3eac8e8bcd arena: v27 verdict (fail, harmful; plan-state scaffolds 0-for-2) + v28 per-step self-eval (pre-registered) 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
 2026-08-14 02:59:17 -07:00
ciregenzandClaude Fable 5 bf39eaf3f6 arena: WebArena paired final -- both stacks 0 strict on hard partition; ours leads partials 3.3x, speed 1.9x, tokens 10x, honesty 0 vs 49 false claims 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
 2026-08-14 02:26:38 -07:00
ciregenzandClaude Fable 5 03e062d059 arena: v27 fix -- CLAUSE marker moves AFTER the action line (was colliding with action-first replies, empty actions) 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
 2026-08-13 23:29:52 -07:00
ciregenzandClaude Fable 5 1a61bfc50e arena: v27 active sub-goal ledger (pre-registered pilot) -- focused current-clause anchor, gated >=3 clauses 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
 2026-08-13 23:18:07 -07:00
ciregenzandClaude Fable 5 88291d1930 arena: v26 pilot verdict -- prediction failed (0/8 targets, controls clean); stale-plan hypothesis disconfirmed 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
 2026-08-13 23:12:36 -07:00
ciregenzandClaude Fable 5 aa5a19e51d arena: v26 serial_multi -- queued actions re-resolve targets per-observation (pre-registered pilot) 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
 2026-08-13 22:41:10 -07:00
ciregenzandClaude Fable 5 b3d69cdd0f arena: WebArena-ours final (0 strict, 6 partials) + strict-field fix and written correction 
The 6 'strict wins' previously reported were fractional checkpoint credits (5x0.667, 1x0.5)
mislabeled by the MiniWoB success predicate (reward>0). ARENA.md books the correction;
EpisodeRecord now carries an explicit strict bit so the conflation is unrepresentable.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
 2026-08-13 22:06:19 -07:00
ciregenzandClaude Fable 5 60e6e77880 arena: bu_real route for webarena_verified ids 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
 2026-08-13 19:25:26 -07:00