The 6 'strict wins' previously reported were fractional checkpoint credits (5x0.667, 1x0.5)
mislabeled by the MiniWoB success predicate (reward>0). ARENA.md books the correction;
EpisodeRecord now carries an explicit strict bit so the conflation is unrepresentable.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
Single-step turns + full compressed history did not move one long-composition target
(controls unharmed). The 20-minute pilot killed the hypothesis before a 2-hour sweep
could -- exactly what the post-v24 rules were for. Next hypothesis, pre-registered:
inter-section settle pacing.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ac/code
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
Full-coverage fair pair after two harness retractions: their patient per-step
eval/memory loop owns long compositions (5-7-part 4/6 vs our 0/8) at 48s/8-false-claims
cost against our 17s/0. v25 ingests the mechanism pre-registered and gated (single-step
turns + compressed full history on >=4-clause goals only); pilot with controls running
per the new discipline.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
Ground truth: one CompWoB page (enter-text-second family) is broken upstream (clears
an input before creating it); isolate mode took remaining[:1] every round, so the
supervisor retried that one impossible page 201 straight times and 72 healthy tasks
starved unseen behind it. browser-use never received them. Supervisor now rotates
through remaining tasks and blacklists any that fail setup three times; their fair
completion run is live. Score one more for verify-the-verifier: both prior narratives
about their CompWoB failures were wrong, and the record now says so.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
Symmetric audit: the composed pages' own genProblem loses an asset race and crashes
reset -- 1/101 times on our launch path, 73/101 on the CDP-port path, same pages,
same seed. Blaming their instrumentation was wrong and is withdrawn; both harness
paths now retry the reset (same courtesy both ways) and their CompWoB completion
sweep is rerunning for a genuine full-coverage number.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
Docker Desktop proved uninitializable without GUI on this box (three engine rebuilds,
factory reset included); colima on macOS's Virtualization.framework started first try
and swallowed the 131GB of site images Docker Desktop could not load. Bring-up script
now targets the colima socket; shopping/admin/forum booting while gitlab downloads.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
60.4% (-5): short tasks paid for the scaffolding, >=5-part stayed 0/8 through a third
mechanism. The cluster's invariance across runway/ordering/checklist localizes the
real need -- mid-episode replanning against current page state -- product-primitive
territory, same frontier as MiniWoB's hard-10. Champion for CompWoB remains v22/v23
at 65.3.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
Long-chain traces showed clean execution collapsing into scroll-flailing once the
agent lost its place. The goal is decomposed once into numbered clauses, rendered
every turn with the model's own last-reported position echoed back; content-agnostic
scaffolding, no task knowledge. Sweeping CompWoB against the 65.3 baseline.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
browser-use cannot load 73 of 101 composed pages (instrumentation crash: 28-task
coverage ceiling, 2 more false claims on what loads). Our ordering fix won its exact
predicted tasks; long-chain traces isolate the true frontier as lost mid-chain
bookkeeping, to be attacked with a rendered clause checklist (v24).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
macOS silently deletes /tmp files by age: pyvenv.cfg vanished mid-sweep and 84 spawns
died import-less while site-packages survived -- the exact silent-error class the goal
names. Venv, serving trees and compwob discovery now live in ~/.cache/arena (durable);
servers rebound; registration re-verified (101). Episode data was never at risk: the
book lives in the repo.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
CompWoB's simple-pair autopsy: with "Ok" ambiguous, the fastpath clicked the SECOND
quoted target first, and order-enforcing pages fail terminally on that. It now acts
only on the first unsatisfied quoted target and otherwise defers to the model --
sequence is part of the task, structurally respected. Lands in v23's queued sweep.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsbS5x2rYsMDxP2kW3qqmQ
