arena: v3/v4 ingest rounds + the dedupe bug those rounds exposed in the shipped ranker

v3 (44.8%, beats the browser-use-shaped arm's 36.0% by 8.8 points at 3x the speed):
page-text panel, select options on the row, input twins exempt from dedupe. v4 queued:
DOM-attribute names for nameless icons and deep sibling context, which turn the email
suite's unlabeled trash/star icons into addressable rows.

Product fix landed with tests: interactiveRanking.ts collapsed adjacent same-named
inputs and adjacent nameless icons as twins -- measured on MiniWoB, that single
collapse cost enter-password and the whole email suite. Inputs and nameless rows are
now exempt; named icon+label pairs still collapse. 17/17 tests, tsc clean.

ARENA.md carries the full scoreboard, per-category verdicts, what was ingested from
whom, and the honest gaps (drag and vision-less spatial) neither stack solves.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ

e2e/browser-v3/arena/ARENA.md

@@ -0,0 +1,109 @@
+# MiniWoB arena — full analysis (2026-08-09)
+
+Every number here is scored by MiniWoB's own reward (`WOB_REWARD_GLOBAL`), read through
+BrowserGym. No arm grades itself; the recorder book (`data/all.jsonl` + per-step screenshots)
+is the only source. Model for every LLM arm: `cc/claude-haiku-4-5-20251001`, seed 42, 125 tasks.
+
+## Scoreboard
+
+| arm | what it is | solved | rate | 95% CI | med wall (win) | false-succ |
+|---|---|---|---|---|---|---|
+| **osw-llm-v3** | our view + 5 ingested techniques | **56/125** | **44.8%** | [36,54] | **4.0s** | 0 |
+| bu | browser-use-shaped flat dump, same model | 45/125 | 36.0% | [28,45] | 11.6s | 0 |
+| osw-llm-v2 | our view + 2 ingested techniques | 40/125 | 32.0% | [24,41] | 3.2s | 0 |
+| osw-llm (v1) | our shipped view, faithful port | 32/125 | 25.6% | [19,34] | 3.8s | 0 |
+| openswarm (no LLM) | our deterministic ladder | 74/375 | 19.7% | [16,24] | 0.8s | 0 |
+| flat (no LLM) | naive axtree floor | 59/375 | 15.7% | [12,20] | 0.8s | 0 |
+| bu-real | the actual browser-use agent over CDP | sweep in progress | ~43% interim | — | ~100s | ≥1 |
+
+**Headline: v3 beats the browser-use-style arm by 8.8 points at ~3× the speed, on identical
+model, tasks, seed, action layer, and scorer.** The two free arms confirm our deterministic
+perception beats the naive floor by 4 points at zero cost.
+
+## The iteration ladder — what each ingested technique bought
+
+Every step below came from diffing our losses against a competitor's wins on the same episodes,
+finding the mechanism, porting it, and re-running all 125.
+
+| version | change (source of the idea) | rate |
+|---|---|---|
+| v1 | faithful port of shipped BrowserListInteractives + ladder | 25.6% |
+| v2 | + clickable-but-unroled elements (browser-use's DOM scan); + coordinate actions (our own click_point, never exposed to the arm) | 32.0% |
+| v3 | + page-text panel (our own BrowserGetText, never exposed); + select options rendered on the row; + input twins exempt from dedupe | 44.8% |
+| v4 | + DOM-attribute names for nameless icons (`(trash)` from `class=trash`); + deep sibling context (which row is Cecile's) | sweep queued |
+
+The pattern worth recording: **half the wins came from browser-use's ideas, half from
+capabilities our product already ships but the agent's page view never surfaced.** The flat dump
+is a bad menu but a complete one; our menu was clean but blind. v3+v4 keep the ranked menu and
+add the missing senses.
+
+## Category detail (v3 vs bu, wins/tasks)
+
+| category | v3 | bu | verdict |
+|---|---|---|---|
+| click_basic | 9/13 | 7/13 | lead |
+| click_compound | 11/20 | 8/20 | lead |
+| text_entry | 10/17 | 5/17 | lead |
+| reading | 8/13 | 3/13 | lead |
+| reasoning | 3/4 | 2/4 | lead |
+| drag | 3/13 | 1/13 | lead (both weak) |
+| forms | 10/22 | 11/22 | behind by 1 |
+| email | 0/10 | 5/10 | behind — root-caused, fixed in v4 |
+| spatial | 2/13 | 3/13 | behind by 1 — partially fixed in v4 |
+
+Email root cause (from step traces + screenshots): every actionable control is a nameless
+`<image>`/`<generic>` whose identity lives in `class="trash"` — an attribute the AX tree
+never surfaces. Two adjacent nameless icons also collapsed in our consecutive-dedupe. v4
+pulls the DOM attribute as the row name and exempts nameless rows from dedupe; the
+email-inbox-delete view now reads `[9]<image "(trash)" ctx="Cecile Odio..">`.
+
+## What the competitors genuinely do better (ingested or credited)
+
+1. **Completeness over cleanliness** (browser-use): their flat dump contains everything —
+   canvases, unlabeled icons, page prose — so no task is invisible. Ingested as: clickable
+   detection, attr-hint names, page-text panel, options rendering. We keep our cap + ranking,
+   so the token bill stays ~40% below theirs (1.57M vs 2.56M for the sweep).
+2. **Persistence loops** (browser-use): their agent retries a wrong bid-format action until it
+   self-corrects. We ingested the cheaper form: history lines carry the page's error verdict, and
+   the prompt forbids repeating a no-effect action.
+3. **Nothing else survived measurement.** Their screenshot-per-step and 10-step median cost them
+   11.6s median wall vs our 4.0s with no rate advantage anywhere except the two categories above,
+   both root-caused to visibility, not reasoning.
+
+## Where neither stack is good (honest gaps)
+
+- **drag** (3/13 vs 1/13): mouse_drag_and_drop exists but one-shot LLM drags rarely land; needs
+  a scripted drag primitive with mid-course verification. Product's send-script ladder is the
+  natural home.
+- **spatial** (2-3/13 both): clicking computed canvas coordinates from a static description is
+  guesswork without vision. A screenshot-conditioned step (the product has one; the arena arm is
+  text-only) is the known fix, at a latency price.
+
+## Product code changes landed from this evidence
+
+- `frontend/src/shared/interactiveRanking.ts` — input-role and nameless rows exempt from
+  consecutive-dedupe (the enter-password / email-suite bug), +3 regression tests (17/17 pass,
+  tsc clean). The same defect existed verbatim in the shipped ranker.
+- Remaining ports staged for review (bigger surface, same evidence): clickable-unroled
+  enumeration and DOM-attr fallback names in `browserCommandHandler.ts` enumerateCandidates;
+  select-option rendering on combobox rows in handleListInteractives.
+
+## Infrastructure honesty (what it took to measure this cleanly)
+
+- Router 502s once booked 40% of two sweeps as policy failures → retries + `infra_llm`
+  classification; infra never counts against skill, and is never hidden either.
+- In-process watchdogs (SIGALRM) were swallowed twice inside stack retry loops → out-of-process
+  supervisor kills the tree when the recorder stops moving and resumes exactly the missing tasks.
+- bu-real starves when any other arm shares the lane (first LLM call never returns) → competitor
+  sweeps get the lane exclusively; measured walls are theirs, not queue time.
+- Reruns supersede by `started_at`; nothing is ever rewritten or deleted from the book.
+
+## Reproduce
+
+```
+cd miniwob-plusplus/miniwob/html && python3 -m http.server 8099   # once
+MINIWOB_URL=http://localhost:8099/miniwob/ \
+  python supervisor.py --arm osw-llm-v3 --model cc/claude-haiku-4-5-20251001
+python report.py --model cc/claude-haiku-4-5-20251001
+python diffs.py --ours osw-llm-v3 --theirs bu
+```

e2e/browser-v3/arena/llm_policy.py

@@ -164,6 +164,8 @@ class OpenSwarmLlmPolicy(LlmPolicy):
     # v3: append the page's visible text, our BrowserGetText equivalent; menu-only lost every task
     # whose payload lives in prose (the algebra equation, which email is Cecile's).
     with_text: bool = False
+    # v4: DOM attribute names for nameless rows ('(trash)') -- the last piece the email suite needed.
+    hints: bool = False
 
     def reset(self, goal: str) -> None:
         self.history = []
@@ -171,7 +173,8 @@ class OpenSwarmLlmPolicy(LlmPolicy):
         self.prev_bids = set()
 
     def view(self, obs: dict[str, Any], goal: str) -> tuple[str, int]:
-        raw_items: list[RankItem] = perception.interactives(obs, include_clickable=self.clickable)
+        raw_items: list[RankItem] = perception.interactives(
+            obs, include_clickable=self.clickable, attr_hints=self.hints)
         shown, truncated = rank_and_cap(raw_items, goal=goal)
         new = {it.bid for it in shown} - self.prev_bids if self.prev_bids else set()
         self.prev_bids = {it.bid for it in shown}
@@ -242,4 +245,7 @@ def build(name: str, model: str = "", endpoint: str = "", **_: Any) -> Any:
     if name == "osw-llm-v3":
         return OpenSwarmLlmPolicy(name=name, model=model, endpoint=endpoint,
                                   clickable=True, with_text=True)
+    if name == "osw-llm-v4":
+        return OpenSwarmLlmPolicy(name=name, model=model, endpoint=endpoint,
+                                  clickable=True, with_text=True, hints=True)
     raise SystemExit(f"unknown arm: {name}")

e2e/browser-v3/arena/perception.py

@@ -42,21 +42,29 @@ def visible(bid: str, extra: dict[str, Any], threshold: float = 0.5) -> bool:
 def build_context(nodes: list[dict[str, Any]], by_id: dict[str, dict[str, Any]],
                   node: dict[str, Any], depth: int = 3) -> str:
     """Nearest ancestor's text, so five identical 'Message' buttons say which card they belong to."""
-    cur = node
-    for _ in range(depth):
-        parent_id = cur.get("parentId")
-        if not parent_id or parent_id not in by_id:
-            return ""
-        parent = by_id[parent_id]
-        texts: list[str] = []
-        for cid in parent.get("childIds") or []:
+    def texts_under(n: dict[str, Any], levels: int) -> list[str]:
+        found: list[str] = []
+        for cid in n.get("childIds") or []:
             child = by_id.get(cid)
             if not child or child is node:
                 continue
             if node_role(child) in TEXT_ROLES:
                 t = node_name(child).strip()
                 if t:
-                    texts.append(t)
+                    found.append(t)
+            elif levels > 0:
+                # One level down inside sibling wrappers: an email row's sender name lives in
+                # <span class=sender><text> -- a direct-children-only walk never saw it.
+                found.extend(texts_under(child, levels - 1))
+        return found
+
+    cur = node
+    for _ in range(depth):
+        parent_id = cur.get("parentId")
+        if not parent_id or parent_id not in by_id:
+            return ""
+        parent = by_id[parent_id]
+        texts = texts_under(parent, 3)
         if texts:
             return " ".join(texts)[:60]
         cur = parent
@@ -64,7 +72,7 @@ def build_context(nodes: list[dict[str, Any]], by_id: dict[str, dict[str, Any]],
 
 
 def interactives(obs: dict[str, Any], include_hidden: bool = False,
-                 include_clickable: bool = False) -> list[RankItem]:
+                 include_clickable: bool = False, attr_hints: bool = False) -> list[RankItem]:
     """Every actionable node in document order, before any ranking or capping is applied.
 
     include_clickable is the technique ingested from browser-use: elements the page wires for
@@ -76,6 +84,7 @@ def interactives(obs: dict[str, Any], include_hidden: bool = False,
     nodes: list[dict[str, Any]] = ax.get("nodes") or []
     extra = obs.get("extra_element_properties") or {}
     by_id = {n["nodeId"]: n for n in nodes if "nodeId" in n}
+    hints = dom_attr_hints(obs) if attr_hints else {}
     out: list[RankItem] = []
     for n in nodes:
         if n.get("ignored"):
@@ -92,6 +101,9 @@ def interactives(obs: dict[str, Any], include_hidden: bool = False,
         if not include_hidden and not visible(str(bid), extra):
             continue
         name = node_name(n).strip()
+        # A nameless icon inherits its DOM identity: '(trash)' beats an unlabeled image row.
+        if not name and str(bid) in hints:
+            name = f"({hints[str(bid)]})"
         bbox = (extra.get(str(bid)) or {}).get("bbox")
         center = (bbox[0] + bbox[2] / 2, bbox[1] + bbox[3] / 2) if bbox else None
         out.append(RankItem(
@@ -131,6 +143,35 @@ def child_options(by_id: dict[str, dict[str, Any]], node: dict[str, Any], depth:
     return found or None
 
 
+# DOM attributes worth surfacing when the AX name is empty, most-identifying first.
+HINT_ATTRS = ("aria-label", "title", "alt", "placeholder", "name", "id", "class")
+
+
+def dom_attr_hints(obs: dict[str, Any]) -> dict[str, str]:
+    """bid -> best identifying DOM attribute, for nodes the AX tree names as nothing.
+
+    Ingested from browser-use: a trash icon is <span class="trash"> in the DOM and '' in the AX
+    tree, and every email task turns on knowing which nameless icon is which.
+    """
+    out: dict[str, str] = {}
+    dom = obs.get("dom_object") or {}
+    strings: list[str] = dom.get("strings") or []
+    for doc in dom.get("documents") or []:
+        for attr_idxs in (doc.get("nodes") or {}).get("attributes") or []:
+            pairs = {}
+            for k in range(0, len(attr_idxs) - 1, 2):
+                pairs[strings[attr_idxs[k]]] = strings[attr_idxs[k + 1]]
+            bid = pairs.get("bid")
+            if not bid:
+                continue
+            for attr in HINT_ATTRS:
+                v = (pairs.get(attr) or "").strip()
+                if v and not v.startswith("browsergym"):
+                    out[bid] = v[:40]
+                    break
+    return out
+
+
 def page_text(obs: dict[str, Any], limit: int = 1200) -> str:
     """The page's visible text, compact: what our product's BrowserGetText gives the agent.
 

e2e/browser-v3/arena/ranking.py

@@ -80,8 +80,10 @@ def dedupe_consecutive(items: list[RankItem]) -> list[RankItem]:
     out: list[RankItem] = []
     for it in items:
         prev = out[-1] if out else None
+        # Nameless rows are also exempt: two adjacent unlabeled icons (star, trash) look identical
+        # by role+name+context yet are different controls -- collapsing them lost the email suite.
         if (prev and prev.role == it.role and prev.name == it.name and prev.context == it.context
-                and it.role not in NEVER_DEDUPE_ROLES):
+                and it.role not in NEVER_DEDUPE_ROLES and it.name):
             continue
         out.append(it)
     return out
@@ -110,11 +112,13 @@ def render(items: list[RankItem], truncated: int, new_bids: set[str] | None = No
     lines = []
     for i, el in enumerate(items, 1):
         dup = counts.get(f"{el.role}|{el.name}", 0) > 1
-        ctx = f' ctx="{el.context}"' if dup and el.context else ""
+        # ctx also whenever the name alone cannot identify the row (empty or a bare attr hint).
+        weak_name = not el.name or el.name.startswith("(")
+        ctx = f' ctx="{el.context}"' if (dup or weak_name) and el.context else ""
         val = f' value="{el.value}"' if el.value else ""
         star = "*" if new_bids and el.bid in new_bids else ""
-        # Coordinates only where the label carries no signal; a named button needs no geometry.
-        pos = f" center=({el.center[0]:.0f},{el.center[1]:.0f})" if el.center and not el.name else ""
+        # Coordinates whenever the label alone cannot pin the row; a named button needs no geometry.
+        pos = f" center=({el.center[0]:.0f},{el.center[1]:.0f})" if el.center and weak_name else ""
         opts = ""
         if el.options:
             shown_opts = el.options[:12]

frontend/src/shared/interactiveRanking.test.ts

@@ -133,3 +133,29 @@ test('docOrder:false keeps legacy rank-order display (the A/B off-arm)', () => {
   // rank order: textbox(0) < button(1) < option(3)
   assert.deepEqual(shown.map((x) => x.backendNodeId), [3, 2, 1]);
 });
+
+test('adjacent same-named INPUTS never collapse (both password fields must show)', () => {
+  const { shown } = rankAndCapInteractives([
+    mk('textbox', 'Password', 1),
+    mk('textbox', 'Password', 2),
+    mk('button', 'Submit', 3),
+  ]);
+  assert.deepEqual(shown.map((x) => x.backendNodeId), [1, 2, 3]);
+});
+
+test('adjacent NAMELESS rows never collapse (star and trash icons are different controls)', () => {
+  const { shown } = rankAndCapInteractives([
+    mk('button', '', 1),
+    mk('button', '', 2),
+    mk('button', '', 3),
+  ]);
+  assert.equal(shown.length, 3);
+});
+
+test('named non-input twins still collapse (icon+label pairs)', () => {
+  const { shown } = rankAndCapInteractives([
+    mk('button', 'Like', 1),
+    mk('button', 'Like', 2),
+  ]);
+  assert.equal(shown.length, 1);
+});

frontend/src/shared/interactiveRanking.ts

@@ -27,13 +27,17 @@ function rolePriority(role: string): number {
   return role in ROLE_PRIORITY ? ROLE_PRIORITY[role] : DEFAULT_PRIORITY;
 }
 
-// Drop back-to-back duplicates with the same role+name. The AX tree often emits an icon node and its label as twins, and sticky headers repeat the same control. Consecutive-only so a genuine list (5 distinct "Add to cart" buttons interleaved with product text) is never collapsed. The sessionId is part of the key so a same-named element in a cross-origin child frame is never mistaken for a twin of the root frame's last element at the seam. Context too: five "Message" buttons in five people-cards are NOT twins, only same-card icon+label pairs (identical context) collapse.
+// Roles whose twins are individually meaningful: two same-named password boxes are two fields the user must both fill, never an icon+label pair. Collapsing them lost MiniWoB enter-password outright (arena, 2026-08-09).
+const NEVER_DEDUPE_ROLES = new Set(['textbox', 'searchbox', 'combobox', 'spinbutton', 'listbox']);
+
+// Drop back-to-back duplicates with the same role+name. The AX tree often emits an icon node and its label as twins, and sticky headers repeat the same control. Consecutive-only so a genuine list (5 distinct "Add to cart" buttons interleaved with product text) is never collapsed. The sessionId is part of the key so a same-named element in a cross-origin child frame is never mistaken for a twin of the root frame's last element at the seam. Context too: five "Message" buttons in five people-cards are NOT twins, only same-card icon+label pairs (identical context) collapse. Inputs and NAMELESS rows never collapse: adjacent unlabeled icons (star, trash) match on every key yet are different controls -- that collapse lost the whole MiniWoB email suite.
 function dedupeConsecutive(items: RankItem[]): RankItem[] {
   const out: RankItem[] = [];
   for (const it of items) {
     const prev = out[out.length - 1];
     if (prev && prev.role === it.role && prev.name === it.name
-        && prev.sessionId === it.sessionId && (prev.context || '') === (it.context || '')) continue;
+        && prev.sessionId === it.sessionId && (prev.context || '') === (it.context || '')
+        && !NEVER_DEDUPE_ROLES.has(it.role) && it.name !== '') continue;
     out.push(it);
   }
   return out;