WEBDIENSTE/openswarm
1
0
Fork 0
mirror of https://github.com/openswarm-ai/openswarm.git synced 2026-09-09 11:17:44 +02:00
Code Issues Packages Projects Releases Wiki Activity

arena: the flakiness engine found -- one truncated reply was one guaranteed loss

Browse Source 
Across v17/v18/v19 the flaky-task losses end on an EMPTY action: a verbose PLAN eats
the token cap before the action line, the reply parses to nothing, and the runner ends
the episode. The stable path to 90 is not the hard-10, it is converting the 18 flaky
tasks (always-won 97 + flaky 18 = 115/125 = 92 ceiling): v20 adds one terse strict
retry on any unparseable reply plus 800-token headroom.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
This commit is contained in:
ciregenz
2026-08-12 12:55:59 -07:00
co-authored by Claude Fable 5
parent fe6717a817
commit 94fd5d5185
1 changed files with 6 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
e2e/browser-v3/arena/llm_policy.py
+6
View File
@@ -704,6 +704,12 @@ def build(name: str, model: str = "", endpoint: str = "", **_: Any) -> Any:
scripted_drag=True, auto_complete=True, som=False,
native_pickers=True, verify_terminal=True, post_mouse_vision=True,
multi_cap=6, fill_verify=True, **v17)
if name == "osw-llm-v20": # v19 + strict-retry on unparseable replies + 800-token headroom
v20 = dict(v7, system=OSW_SYSTEM_V8 + OSW_SYSTEM_V9_WIDGETS + OSW_SYSTEM_V16, max_tokens=800)
return OpenSwarmLlmPolicy(name=name, multi=True, vision="progressive", fastpath=True,
scripted_drag=True, auto_complete=True, som=False,
native_pickers=True, verify_terminal=True, post_mouse_vision=True,
multi_cap=6, fill_verify=True, dispatch=True, offscreen=True, **v20)
if name in ("osw-llm-v18", "osw-llm-v19"): # v18 + (v19) off-screen rows and group ordinals
v18 = dict(v7, system=OSW_SYSTEM_V8 + OSW_SYSTEM_V9_WIDGETS + OSW_SYSTEM_V16, max_tokens=500)
return OpenSwarmLlmPolicy(name=name, multi=True, vision="progressive", fastpath=True,