ciregenz
fe02b4ce91
[eric] browser: log write-tier failures (adapter bug=WARNING, site-reject=INFO) so a systemically-broken fast path isn't silently invisible to ops
2026-07-15 17:10:57 -07:00
ciregenz
449f66057d
[eric] browser: scope broadened submit vocab to the receipt-gated send-script; keep the always-on Send hint tight (broadening it regressed the default config with the send-script off)
2026-07-15 16:22:10 -07:00
ciregenz
5196ba77e3
[eric] browser: fire the 'from a previous visit' recall line only on a LEARNED playbook, not a shipped seed (else a first visit to a seeded site lies)
2026-07-15 15:46:35 -07:00
ciregenz
efb84daefc
[eric] browser: unknown-model fallback resolves a capable (sonnet) tier, not haiku (browser is agentic; haiku 50.7% vs sonnet 72.5% OSWorld)
2026-07-15 14:26:05 -07:00
ciregenz
9b7b972fef
[eric] browser: retract haiku loop-pin recommendation (OSWorld 50.7% vs sonnet 72.5%; per-turn speed is the wrong metric; fewer-turns is the real, provider-agnostic cold lever)
2026-07-15 14:18:11 -07:00
ciregenz
78fd61475a
[eric] browser: fix weather.com seed URL (404; keys on internal location code, use search box)
2026-07-15 14:09:45 -07:00
ciregenz
e877afd3c8
[eric] browser: seed 117 popular sites (search/shop/food/stream/productivity/health/gov/AI); money sites read-only-never-transact; split data into seed_playbooks.py
2026-07-15 14:05:21 -07:00
ciregenz
06744e39c4
[eric] browser: +12 seed sites & write-mechanics; correct loop-tier comment (Haiku is the ~2x lever, not Sonnet, measured n=10)
2026-07-15 13:49:58 -07:00
ciregenz
1e68aa5d7b
[eric] browser: send-script completes on X/IG/FB (submit-button vocab beyond LinkedIn Send)
2026-07-15 11:54:49 -07:00
ciregenz
1f74cc13de
[eric] browser: general capture-replay write tier (site's own captured route, same-origin + gated, default off)
2026-07-14 23:47:17 -07:00
ciregenz
d445543430
[eric] browser: wire BrowserApiWrite agent tool + robust Reddit write receipt
2026-07-14 23:27:39 -07:00
ciregenz
8bc7e479e8
[eric] browser: unified API-first write registry (site's own write API via borrowed session) + Reddit adapter
...
The general agent writes via UI puppeteering + the model loop (fire-rate ~50%, 13-52s); the deterministic alternative (cookie-borrow -> the site's OWN write API -> typed receipt) lived only in isolated MCP shims, and only Reddit's was genuinely API-first (X's shim is UI-driving). site_write_registry unifies it: has_api_write(domain,action) + api_write() route a write to a per-domain adapter (Reddit reuses reddit_writes, no fork), time it, and return a typed WriteResult (ok + the site's id/permalink receipt + latency); any failure becomes a typed ok=False so the agent falls back to the UI path, never a crash. A site with no adapter is a clean miss; adding one (X GraphQL, etc.) is one row. LIVE-VALIDATED end to end on Reddit through the registry: comment ok=True + reversible delete ok=True 196ms, verified [deleted] (deterministic, no captcha/DOM/selector, ~50-190x faster than UI). 6 unit tests (routing/receipt/fail-safe), linter clean, no import cycles. NOT yet wired as a browser-agent MCP tool (that path isn't live-testable on the wedged dev bench); receipt-string extraction returned 'ok' on the live run (write succeeded + verified reversible; id/permalink parse is a follow-up).
2026-07-14 19:51:53 -07:00
ciregenz
42a3249976
[eric] browser: broaden compose-trigger vocabulary (create/new-thread/start-a-thread variants)
...
The reveal opener regex only matched 'create a post', missing bare 'Create' and 'New thread' / 'Start a thread' buttons that real compose surfaces use. Adds those; safe by construction (a broadened trigger can never produce a false success, a fill against a real composer is still required, so a trigger that opens a signup/join upsell instead of a composer yields found:false). Validated CDP-direct: LinkedIn-post and YouTube-comment still flip (no regression), and Threads (which gates posting behind a 'Join Threads' upsell for this session) correctly does NOT false-fill
2026-07-13 14:59:21 -07:00
ciregenz
937bb35dca
[eric] browser: evalInPage runs via CDP Runtime.evaluate, not webContents.executeJavaScript
...
The executeJavaScript path is SUSPENDED by Electron until a page stops loading, so on a page whose trackers never let it settle (Reddit, LinkedIn) every in-page command queues into the 15s backend timeout = the mount-poll wedge that has blocked every agent-loop run. CDP Runtime.evaluate runs in the browser process, out of the renderer's load-blocked event loop, so it does not suspend. This reuses the already-in-production CDP bridge (sendCdp -> send-cdp-command -> wc.debugger.sendCommand, keyed by getWebContentsId, with attach + per-wc serialization + timeout already built) that the AX/click commands use; only the eval path was left on executeJavaScript. No-regression by construction: on any CDP INFRA failure (debugger can't attach because DevTools or a remote-debugging port holds the webContents, or the bridge errors) it falls back to the original executeJavaScript path, and a real page-side exception still surfaces as a throw; window.__OSW_CDP_EVAL__ = false forces the old path. Unwrap logic extracted to cdpEval.ts + 6 unit tests. NOTE not live-demonstrable on a --remote-debugging-port bench: that port globally blocks wc.debugger.attach, so the whole app CDP path (this AND the existing list_interactives) times out there; production and normal dev do not use the port, so attach works and the existing CDP commands prove it
2026-07-12 21:55:59 -07:00
ciregenz
82fe16767c
[eric] browser: handleType reads back its fill and falls back to real keystrokes = the main type action now detects when execCommand insertText did not commit (editors that reject synthetic input) and retypes via wv.sendInputEvent, same lever as the composer finder's keystrokeFill
2026-07-12 21:26:49 -07:00
ciregenz
19a3e8ad1a
[eric] browser: keystroke-fill fallback + cross-nav reveal retry (both flag-gated, default off)
...
Item 1: when the finder's execCommand insertText read-back fails (editors that reject synthetic input), retype the same text as real OS-level key events via wv.sendInputEvent, then verify from the marked element OR document.activeElement (editors like Reddit swap the node on activation, staling the selector). Item 3: when an open-first reveal navigates to a new document (Reddit thread, TikTok video, GitHub issue), the send-script re-perceives after a load beat and calls the finder once more on the destination, bounded to 2 tries. 4 new tests (cross-nav retry fires / no wasted retry) + structural tests split into test_browser_reveal.py to stay under the 300-line cap. Live real-path validation of the sendInputEvent leg is owed on a healthy rig: the eric/browser agent loop wedges at fresh-card webview mount (15.25s cap, renderer starvation, predates the suspend fix) at low load too
2026-07-12 20:52:15 -07:00
ciregenz
a3099f035c
[eric] browser: reveal-and-find gains activation-click + shadow-safe collection = a placeholder composer that fills empty (YouTube 'Add a comment') gets clicked, rescanned, and the real editor filled; deepMatch collects only selector matches (few hundred) not every node (the old all-node 4000 cap truncated before a heavy SPA's late-in-DOM composer, which hid X's OWN box); openFirstItem broadened to chat/message lists (data-testid=conversation, message/chat links) and pierces shadow DOM; CDP-direct n=2 reproduced: reveal converts LinkedIn post (modal) + YouTube comment (activation), controls stay filled
2026-07-12 20:03:05 -07:00
ciregenz
64675115f8
[eric] browser: reveal-and-find reach primitive (OSW_COMPOSER_REVEAL, default off) = when no composer is painted the finder takes ONE reversible reveal action (click a scored compose-trigger, open the first list item, or scroll) and rescans, bounded; SPA route changes keep the JS context alive so X/LinkedIn/Reddit/YouTube stay drivable in one call; never clicks an irreversible control (hard-block send/submit/pay/delete); send-script forwards reveal only under the flag; 2 tests (reveal forwarded when on, false by default)
2026-07-12 19:29:07 -07:00
ciregenz
2d49687ae0
[eric] browser: composer finder pierces shadow DOM (Reddit shreddit, Telegram, WhatsApp, Slack build their composer inside web-component shadow roots a light-DOM querySelectorAll can't see); bounded recursive walk (depth 8, 400-node cap). Live finder-probe on real DOMs: correctly picks X's contenteditable 'Post text' + excludes the search combobox; the remaining wall is that lazy/dialog/chat-gated composers aren't rendered at scan time (detection is downstream of render)
2026-07-10 00:56:05 -07:00
ciregenz
4c85dd6314
[eric] browser: structural composer finder (OSW_COMPOSER_STRUCT, default off) = an in-page primitive that ranks editable regions (contenteditable/textarea/text-input near a Send/Post/Reply control, excluding search boxes) instead of matching AX names, marks the winner + fills+reads-back in-page (the only reliable commit-check for a React contenteditable whose text never reaches the AX tree); send-script uses it as a fallback when the name detector misses, unblocking unnamed/non-standard composers (Reddit Lexical, X DM); 3 tests, all safety gates + dry-run stop unchanged
2026-07-10 00:13:44 -07:00
ciregenz
5ebe554a4c
[eric] browser: OSW_PRESTAGE_WARMUP (default off) = before the aux loop, poll perception up to a wall-clock-bounded 8s so a cold heavy SPA (Gmail/LinkedIn) that just opened is perceivable instead of read-empty-then-give-up; wall-clock (not iteration) bound so a slow/wedged card can't overrun the outer prestage timeout; a page that never settles still fails fast (honest wedge, not hidden)
2026-07-09 17:25:45 -07:00
ciregenz
31c4817146
[eric] browser: prestage opener-aware block rule (OSW_PRESTAGE_OPENER, default off) = a compose-entry word (post/comment/reply/tweet/write) is allowed to REVEAL a composer while none is present but REFUSED the instant a composer textbox exists (then it's the real submit); hard-irreversible words (send/submit/pay/buy/delete/subscribe) refused always; +1 deeper reach step + composer-open system prompt; flag-off is byte-identical to the legacy blanket gate; 6 tests incl the submit-still-refused safety invariant
2026-07-09 16:42:55 -07:00
ciregenz
329209259b
[eric] browser: dry-run terminates the browser run with a grep-stable [dryrun-report] gate line (a dry-run decline falling through to the model loop could perform the REAL send the flag exists to avoid, and a loop rescue would contaminate gate attribution); inert when the flag is off; +2 tests
2026-07-09 16:26:15 -07:00
ciregenz
db9aea3d2b
[eric] browser: composer regex catches X's 'Post text' + 'add a comment' (live coverage sweep: X tweet composer IS in the AX list but its name didn't match; safely excludes 'postal code'). Proves the deeper point: composer names vary per site, name-regex is inherently chasing them
2026-07-09 00:21:22 -07:00
ciregenz
5cff552f0d
[eric] browser: send-script dry-run probe (OSW_SENDSCRIPT_DRYRUN) = fills + verifies readiness but stops before the irreversible click, so generalization to a new site can be proven live with zero outward send
2026-07-09 00:01:00 -07:00
ciregenz
4eb1f909d2
[eric] browser: generalize the send-script surface gate from a LinkedIn-URL regex to a STRUCTURAL 'composer or opener present in the perception' check, so the same ~14s send fast-lane fires on ANY messaging site (X/Slack/Discord/Instagram/Gmail), no per-site code; broadened composer+opener name regexes to general messaging shapes; all safety gates unchanged (site-agnostic already); +2 tests incl a non-LinkedIn X composer firing
2026-07-08 23:53:11 -07:00
ciregenz
7de724c930
[eric] browser: fix the 2 warm-marriage replay bugs = (1) 'Send a message to X' opener no longer trips the replay send-boundary (it opens the composer, reversible), (2) marriage replays NAV+opener only and hands composer->send to the send-script (which polls the lazy overlay), instead of replaying a composer-textbox click that races the render; 4 tests
2026-07-08 23:39:19 -07:00
ciregenz
8dcf4b4ae0
[eric] browser: skip prestage when a learned skill exists for the host+task (the replayed prefix owns the nav; prestage would aux-drive the same path for ~8-12s first); skill key hoisted above the gate so skip + replay share one derivation
2026-07-08 23:22:25 -07:00
ciregenz
3257caafe4
[eric] browser: warm-write marriage (OSW_REPLAY_SENDTAIL, default off) = a learned send-skill's replayed prefix hands its post-nav state to the verified send-script tail, so a warm write completes with ZERO model turns; also fixes prefix steps missing from action_log (warm re-record was clobbering the skill with a tail-only version); zero-LLM e2e test
2026-07-08 23:20:07 -07:00
ciregenz
eae31e744f
[eric] browser: fold page text into the auto-attached post-action state (OSW_FOLD_TEXT, default off) = after any action the model has BOTH clickable elements AND readable text, so it stops double-perceiving (measured: 52% of tool calls are perception, ~half redundant list+GetText pairs, the biggest untouched turn-cost)
2026-07-08 17:08:16 -07:00
ciregenz
e0162779a7
[eric] browser: gate document-order display behind OSW_DOC_ORDER (default on) so the A/B can measure it against legacy rank-order; 14 ranking tests
2026-07-08 14:57:24 -07:00
ciregenz
2234ae46de
[eric] browser: interactives render in DOCUMENT order (rank still picks who survives the cap, but display is top-to-bottom page order); the scrambled list forced the model to burn orientation turns reading page text to recover position on ordinal tasks ('the 4th story' landed at [48]), the real cause of the slow HN planning
2026-07-08 14:00:02 -07:00
ciregenz
a308444487
[eric] browser: planner maps ordinals to the Nth matching row (live fire declined 'the 4th story's comments' because it wasn't told it may count)
2026-07-07 23:26:43 -07:00
ciregenz
f238fbfce9
[eric] browser: code-side plan dispatch (OSW_PLAN_DISPATCH, default off) = one aux call compiles the task's mechanical prefix into verified steps, code executes them resolve-late+verified, the big model starts with that work done; irreversibles refused in the PARSER, fail-open everywhere; the turn-collapser that doesn't wait for model tool adoption (0/3 A/Bs), 4 tests
2026-07-07 23:13:41 -07:00
ciregenz
e67b22df66
[eric] browser: reframe BrowserBatch schema copy (was 'your standard way to ACT', steering the model away from the verified tool; now points dependent named click/fill sequences at BrowserActVerified)
2026-07-07 23:04:30 -07:00
ciregenz
02fee0983e
[eric] browser: evict only AGENT-SPAWNED cards (a wedged USER card must never be deleted out from under the user; unverifiable ownership = no delete, reuse-skip is the remedy); caught in the downstream-risk review, 3 tests
2026-07-07 23:04:09 -07:00
ciregenz
aa752303cd
[eric] browser: verified-action executor = resolve-late resolver + act/verify/re-aim step loop + BrowserActVerified tool (model emits 2-4 dependent click/fill steps in ONE turn, code resolves each by name against the live page, verifies the specific effect, re-aims once; irreversible targets refused in code, solo-send rule unchanged); 16 unit + 1 loop-interception test
2026-07-07 22:42:39 -07:00
ciregenz
1e6dd84495
[eric] browser: verified-action executor foundation = generic site-agnostic effect verification (did the action produce the SPECIFIC expected effect: url_changed/appeared/gone/filled/cleared), zero per-site code; wired into the send-script's receipt (proves it on the real flow, behavior-identical to the old inline check via an equivalence test) so it's the executor's verification core, not dead scaffolding
2026-07-07 18:00:35 -07:00
ciregenz
c423517a9f
[eric] browser: close the evict->remount race (wait for the dead webview's teardown to land before recovery spawns its replacement; live-observed the recovery card still capped once when it mounted next to a still-freeing dead one); failure-path-only wait, invisible next to the 15s cap it prevents
2026-07-07 17:56:38 -07:00
ciregenz
3123e9a1b4
[eric] browser: evict a dead card's webview BEFORE recovery spawns a fresh one (the recovery-card wedge: two heavy pages starve the renderer event loop; unmounting the wedged one leaves the recovery card the only heavy neighbor); immediate removeBrowserCard (no fade/keep-pill) + layout drop, gated by OSW_DEADCARD_EVICT, 2 tests
2026-07-07 17:20:57 -07:00
ciregenz
b60f0b94aa
[eric] browser: wrong-target detectability probe = the click hit-test now reports WHAT element was actually under the click point, so a click that lands on the wrong element (occluded/stale/moved, and still changes the page) is finally measurable, the failure a page-change metric can't see
2026-07-07 16:47:47 -07:00
ciregenz
2b63953650
[eric] browser: click-effect probe wraps ALL clickBackendNode exits (was only on the coordinate branch; real-site clicks exit via the covered-element JS path, so the metric captured ~none)
2026-07-07 15:48:37 -07:00
ciregenz
8f2565e027
[eric] browser: thread self-heal + click-effect metric into BATCH sub-clicks too (most clicks are batched; gating only top-level click_index measured ~none of them) + scan batch sub-results for the telemetry
2026-07-07 15:41:07 -07:00
ciregenz
f54f474de7
[eric] browser: click-effect metric (measure before fixing) = fingerprint the page before/after a click to catch the INVISIBLE failure the tool-error counter misses (a click that succeeds but hits the wrong/dead element = nothing changes); flag-gated OSW_CLICK_EFFECT_PROBE, zero cost off, 3 unit tests
2026-07-07 15:33:33 -07:00
ciregenz
522b55f2c8
[eric] browser: log self-heal click recoveries for the A/B (backend can't see the renderer console.log)
2026-07-07 15:07:12 -07:00
ciregenz
5ca7308fa5
[eric] browser: self-healing click (Phase 1) = a failed cached-index click auto-escalates to fresh by-name resolution in-code before the model ever sees the error (stale index = ~half of all runs' tool-errors, each costs a ~3s re-strategize turn); error-triggered so no double-act, OSW_SELFHEAL_CLICK=0 kill switch, 5 unit tests
2026-07-07 15:05:41 -07:00
ciregenz
6bdd75d978
[eric] browser: strip the OSW_SENDSCRIPT_PROBE diagnostic scaffolding (its job is done, it captured the overlay ground truth)
2026-07-07 14:05:08 -07:00
ciregenz
2894ec3ca6
[eric] browser: send-script polls a short window for the lazy overlay composer before declining (r269: prestage's LAST step was the Message click, overlay renders 1-2s later so one immediate read still missed it)
2026-07-07 12:16:34 -07:00
ciregenz
b03cb53795
[eric] browser: send-script receipt=False must NOT set send_confirmed (r264 set it unconditionally, so an unverified send let the model FALSELY claim delivery); only a cleared-composer receipt confirms
2026-07-07 12:06:48 -07:00
ciregenz
9128fd19fb
[eric] browser: send-script takes one fresh interactives read before declining (prestage's staged snapshot is frozen at the Message click; the overlay composer lazy-renders a beat later, r263 declined on exactly that stale-snapshot miss)
2026-07-07 11:59:58 -07:00
ciregenz