Commit Graph
1697 Commits
Author SHA1 Message Date
ciregenz d7fe6e35f0 [eric] browser: salvage a max_tokens-truncated plan array + 1000-token headroom (verbose pretty-print cut the closing bracket live) 2026-07-16 16:05:14 -07:00
ciregenz 706a93e0e9 [eric] browser: prefill [ so the plan aux can't waffle in prose (caught live) + read-script skips the settle-retry on results lists 2026-07-16 16:02:41 -07:00
ciregenz d786d7fdea [eric] browser: perceive-only waits for page substance (aux asks were accidental settle time; thin list starved plan-dispatch) + log empty plan replies 2026-07-16 15:54:40 -07:00
ciregenz 3fa98e73a3 [eric] browser: verified-step smart-wait polls the expectation early instead of sleeping the flat settle 2026-07-16 15:51:47 -07:00
ciregenz e6b6d1a1a0 [eric] browser: prestage runs perceive-only for read tasks when plan-dispatch owns the click-through (its aux asks measured ~2s of nothing there) 2026-07-16 15:51:47 -07:00
ciregenz 15edeee9fe [eric] browser: prestage overrules READY once on a results-list URL (aux stalled at search 2/4 cold runs; deterministic, fail-open) 2026-07-16 15:35:55 -07:00
ciregenz 3e2b779c34 [eric] browser: read-script demands complete answers + one settle-retry on INSUFFICIENT (partial hydration answered with the title missing) 2026-07-16 15:27:35 -07:00
ciregenz f1bf6c78af [eric] browser: read-script waits out SPA hydration on a thin page (184-char LinkedIn profile mid-render, measured) 2026-07-16 15:24:07 -07:00
ciregenz e4938044a4 [eric] browser: prestage clicks through to the person's own page for READ tasks too (search results are not the staged page) 2026-07-16 15:22:13 -07:00
ciregenz 77181665c5 [eric] browser: read-script answers a read task from the staged authed page in one aux call (answer-or-INSUFFICIENT, fail-open) 2026-07-16 15:06:49 -07:00
ciregenz 59edd9db2d [eric] browser: 1st-degree outranks all cues when picking a person to message (run 2 chose a 2nd-degree stranger first) 2026-07-16 14:49:38 -07:00
ciregenz 80ccdfc76e [eric] browser: skip plan-dispatch when the send composer is already staged (4.7s of aux poking for nothing) 2026-07-16 14:35:08 -07:00
ciregenz 493b58eca5 [eric] browser: plan-dispatch compiles cross-page chains + marked chosen picks (disambiguation turn-collapser); irreversible wall unchanged 2026-07-16 14:30:15 -07:00
ciregenz f7472b3dbf [eric] browser: recover a card that opens straight onto a JSON URL (initial-load path + homepage fallback) 2026-07-16 10:37:32 -07:00
ciregenz 6bec61eda4 [eric] browser: steer the model to page URLs, not raw JSON/API endpoints (read APIs via BrowserReplayRoute) 2026-07-15 23:07:41 -07:00
ciregenz de3261d2d9 [eric] browser: a navigate landing on a raw JSON/API URL hands data to the agent and recovers the card off the wall 2026-07-15 23:06:02 -07:00
ciregenz 72ddb07b59 [eric] browser: mid-loop post-fill code-send (model types, code finishes the send + verifies), so un-quoted sends skip the ~3-4 stale-Send-button turns 2026-07-15 20:02:57 -07:00
ciregenz fd1d312a91 [eric] browser: extract the send-script's send tail into shared complete_send (find Send + click + two-sided receipt) 2026-07-15 19:57:30 -07:00
ciregenz 2357111ac8 [eric] browser: skip the blocking candidate scan on send tasks (it never ranks candidates for a message/reply, so it was ~4s of dead first-turn latency) 2026-07-15 19:30:17 -07:00
ciregenz 424ff22ce3 [eric] test: fix compose-confirmation test (asyncio.run, no leading-underscore attr) 2026-07-15 19:15:02 -07:00
ciregenz 32215136d2 [eric] browser: model-written send confirmation (cheap aux call, validated + fail-open to template) instead of a hardcoded line 2026-07-15 19:14:31 -07:00
ciregenz 7edb81a6cc [eric] test: warm-send summary assert case-insensitive for the friendly confirmation 2026-07-15 19:09:18 -07:00
ciregenz 37d4cf23f5 [eric] browser: friendly non-dev send confirmation ('Done, I sent "X" for you' vs 'the send registered and the composer cleared') 2026-07-15 19:07:15 -07:00
ciregenz 3a9474e77d [eric] browser: cap+timeout the candidate scan (kill the 8s empty-scan idle) + send-script enables prestage opener (fixes ~50% fire-rate miss on search-page landings) 2026-07-15 18:26:52 -07:00
ciregenz 363124e0d5 [eric] browser: goodrx seed is drug-name lookup not free-text search (caught by live URL validation: /coffee 404s) 2026-07-15 17:36:04 -07:00
ciregenz fe02b4ce91 [eric] browser: log write-tier failures (adapter bug=WARNING, site-reject=INFO) so a systemically-broken fast path isn't silently invisible to ops 2026-07-15 17:10:57 -07:00
ciregenz 449f66057d [eric] browser: scope broadened submit vocab to the receipt-gated send-script; keep the always-on Send hint tight (broadening it regressed the default config with the send-script off) 2026-07-15 16:22:10 -07:00
ciregenz 5196ba77e3 [eric] browser: fire the 'from a previous visit' recall line only on a LEARNED playbook, not a shipped seed (else a first visit to a seeded site lies) 2026-07-15 15:46:35 -07:00
ciregenz efb84daefc [eric] browser: unknown-model fallback resolves a capable (sonnet) tier, not haiku (browser is agentic; haiku 50.7% vs sonnet 72.5% OSWorld) 2026-07-15 14:26:05 -07:00
ciregenz 9b7b972fef [eric] browser: retract haiku loop-pin recommendation (OSWorld 50.7% vs sonnet 72.5%; per-turn speed is the wrong metric; fewer-turns is the real, provider-agnostic cold lever) 2026-07-15 14:18:11 -07:00
ciregenz 78fd61475a [eric] browser: fix weather.com seed URL (404; keys on internal location code, use search box) 2026-07-15 14:09:45 -07:00
ciregenz e877afd3c8 [eric] browser: seed 117 popular sites (search/shop/food/stream/productivity/health/gov/AI); money sites read-only-never-transact; split data into seed_playbooks.py 2026-07-15 14:05:21 -07:00
ciregenz 06744e39c4 [eric] browser: +12 seed sites & write-mechanics; correct loop-tier comment (Haiku is the ~2x lever, not Sonnet, measured n=10) 2026-07-15 13:49:58 -07:00
ciregenz 1e68aa5d7b [eric] browser: send-script completes on X/IG/FB (submit-button vocab beyond LinkedIn Send) 2026-07-15 11:54:49 -07:00
ciregenz 1f74cc13de [eric] browser: general capture-replay write tier (site's own captured route, same-origin + gated, default off) 2026-07-14 23:47:17 -07:00
ciregenz d445543430 [eric] browser: wire BrowserApiWrite agent tool + robust Reddit write receipt 2026-07-14 23:27:39 -07:00
ciregenz 8bc7e479e8 [eric] browser: unified API-first write registry (site's own write API via borrowed session) + Reddit adapter
The general agent writes via UI puppeteering + the model loop (fire-rate ~50%, 13-52s); the deterministic alternative (cookie-borrow -> the site's OWN write API -> typed receipt) lived only in isolated MCP shims, and only Reddit's was genuinely API-first (X's shim is UI-driving). site_write_registry unifies it: has_api_write(domain,action) + api_write() route a write to a per-domain adapter (Reddit reuses reddit_writes, no fork), time it, and return a typed WriteResult (ok + the site's id/permalink receipt + latency); any failure becomes a typed ok=False so the agent falls back to the UI path, never a crash. A site with no adapter is a clean miss; adding one (X GraphQL, etc.) is one row. LIVE-VALIDATED end to end on Reddit through the registry: comment ok=True + reversible delete ok=True 196ms, verified [deleted] (deterministic, no captcha/DOM/selector, ~50-190x faster than UI). 6 unit tests (routing/receipt/fail-safe), linter clean, no import cycles. NOT yet wired as a browser-agent MCP tool (that path isn't live-testable on the wedged dev bench); receipt-string extraction returned 'ok' on the live run (write succeeded + verified reversible; id/permalink parse is a follow-up).
2026-07-14 19:51:53 -07:00
ciregenz 42a3249976 [eric] browser: broaden compose-trigger vocabulary (create/new-thread/start-a-thread variants)
The reveal opener regex only matched 'create a post', missing bare 'Create' and 'New thread' / 'Start a thread' buttons that real compose surfaces use. Adds those; safe by construction (a broadened trigger can never produce a false success, a fill against a real composer is still required, so a trigger that opens a signup/join upsell instead of a composer yields found:false). Validated CDP-direct: LinkedIn-post and YouTube-comment still flip (no regression), and Threads (which gates posting behind a 'Join Threads' upsell for this session) correctly does NOT false-fill
2026-07-13 14:59:21 -07:00
ciregenz 937bb35dca [eric] browser: evalInPage runs via CDP Runtime.evaluate, not webContents.executeJavaScript
The executeJavaScript path is SUSPENDED by Electron until a page stops loading, so on a page whose trackers never let it settle (Reddit, LinkedIn) every in-page command queues into the 15s backend timeout = the mount-poll wedge that has blocked every agent-loop run. CDP Runtime.evaluate runs in the browser process, out of the renderer's load-blocked event loop, so it does not suspend. This reuses the already-in-production CDP bridge (sendCdp -> send-cdp-command -> wc.debugger.sendCommand, keyed by getWebContentsId, with attach + per-wc serialization + timeout already built) that the AX/click commands use; only the eval path was left on executeJavaScript. No-regression by construction: on any CDP INFRA failure (debugger can't attach because DevTools or a remote-debugging port holds the webContents, or the bridge errors) it falls back to the original executeJavaScript path, and a real page-side exception still surfaces as a throw; window.__OSW_CDP_EVAL__ = false forces the old path. Unwrap logic extracted to cdpEval.ts + 6 unit tests. NOTE not live-demonstrable on a --remote-debugging-port bench: that port globally blocks wc.debugger.attach, so the whole app CDP path (this AND the existing list_interactives) times out there; production and normal dev do not use the port, so attach works and the existing CDP commands prove it
2026-07-12 21:55:59 -07:00
ciregenz 82fe16767c [eric] browser: handleType reads back its fill and falls back to real keystrokes = the main type action now detects when execCommand insertText did not commit (editors that reject synthetic input) and retypes via wv.sendInputEvent, same lever as the composer finder's keystrokeFill 2026-07-12 21:26:49 -07:00
ciregenz 19a3e8ad1a [eric] browser: keystroke-fill fallback + cross-nav reveal retry (both flag-gated, default off)
Item 1: when the finder's execCommand insertText read-back fails (editors that reject synthetic input), retype the same text as real OS-level key events via wv.sendInputEvent, then verify from the marked element OR document.activeElement (editors like Reddit swap the node on activation, staling the selector). Item 3: when an open-first reveal navigates to a new document (Reddit thread, TikTok video, GitHub issue), the send-script re-perceives after a load beat and calls the finder once more on the destination, bounded to 2 tries. 4 new tests (cross-nav retry fires / no wasted retry) + structural tests split into test_browser_reveal.py to stay under the 300-line cap. Live real-path validation of the sendInputEvent leg is owed on a healthy rig: the eric/browser agent loop wedges at fresh-card webview mount (15.25s cap, renderer starvation, predates the suspend fix) at low load too
2026-07-12 20:52:15 -07:00
ciregenz a3099f035c [eric] browser: reveal-and-find gains activation-click + shadow-safe collection = a placeholder composer that fills empty (YouTube 'Add a comment') gets clicked, rescanned, and the real editor filled; deepMatch collects only selector matches (few hundred) not every node (the old all-node 4000 cap truncated before a heavy SPA's late-in-DOM composer, which hid X's OWN box); openFirstItem broadened to chat/message lists (data-testid=conversation, message/chat links) and pierces shadow DOM; CDP-direct n=2 reproduced: reveal converts LinkedIn post (modal) + YouTube comment (activation), controls stay filled 2026-07-12 20:03:05 -07:00
ciregenz 64675115f8 [eric] browser: reveal-and-find reach primitive (OSW_COMPOSER_REVEAL, default off) = when no composer is painted the finder takes ONE reversible reveal action (click a scored compose-trigger, open the first list item, or scroll) and rescans, bounded; SPA route changes keep the JS context alive so X/LinkedIn/Reddit/YouTube stay drivable in one call; never clicks an irreversible control (hard-block send/submit/pay/delete); send-script forwards reveal only under the flag; 2 tests (reveal forwarded when on, false by default) 2026-07-12 19:29:07 -07:00
ciregenz 2d49687ae0 [eric] browser: composer finder pierces shadow DOM (Reddit shreddit, Telegram, WhatsApp, Slack build their composer inside web-component shadow roots a light-DOM querySelectorAll can't see); bounded recursive walk (depth 8, 400-node cap). Live finder-probe on real DOMs: correctly picks X's contenteditable 'Post text' + excludes the search combobox; the remaining wall is that lazy/dialog/chat-gated composers aren't rendered at scan time (detection is downstream of render) 2026-07-10 00:56:05 -07:00
ciregenz 4c85dd6314 [eric] browser: structural composer finder (OSW_COMPOSER_STRUCT, default off) = an in-page primitive that ranks editable regions (contenteditable/textarea/text-input near a Send/Post/Reply control, excluding search boxes) instead of matching AX names, marks the winner + fills+reads-back in-page (the only reliable commit-check for a React contenteditable whose text never reaches the AX tree); send-script uses it as a fallback when the name detector misses, unblocking unnamed/non-standard composers (Reddit Lexical, X DM); 3 tests, all safety gates + dry-run stop unchanged 2026-07-10 00:13:44 -07:00
ciregenz 5ebe554a4c [eric] browser: OSW_PRESTAGE_WARMUP (default off) = before the aux loop, poll perception up to a wall-clock-bounded 8s so a cold heavy SPA (Gmail/LinkedIn) that just opened is perceivable instead of read-empty-then-give-up; wall-clock (not iteration) bound so a slow/wedged card can't overrun the outer prestage timeout; a page that never settles still fails fast (honest wedge, not hidden) 2026-07-09 17:25:45 -07:00
ciregenz 31c4817146 [eric] browser: prestage opener-aware block rule (OSW_PRESTAGE_OPENER, default off) = a compose-entry word (post/comment/reply/tweet/write) is allowed to REVEAL a composer while none is present but REFUSED the instant a composer textbox exists (then it's the real submit); hard-irreversible words (send/submit/pay/buy/delete/subscribe) refused always; +1 deeper reach step + composer-open system prompt; flag-off is byte-identical to the legacy blanket gate; 6 tests incl the submit-still-refused safety invariant 2026-07-09 16:42:55 -07:00
ciregenz 329209259b [eric] browser: dry-run terminates the browser run with a grep-stable [dryrun-report] gate line (a dry-run decline falling through to the model loop could perform the REAL send the flag exists to avoid, and a loop rescue would contaminate gate attribution); inert when the flag is off; +2 tests 2026-07-09 16:26:15 -07:00
ciregenz db9aea3d2b [eric] browser: composer regex catches X's 'Post text' + 'add a comment' (live coverage sweep: X tweet composer IS in the AX list but its name didn't match; safely excludes 'postal code'). Proves the deeper point: composer names vary per site, name-regex is inherently chasing them 2026-07-09 00:21:22 -07:00
ciregenz 5cff552f0d [eric] browser: send-script dry-run probe (OSW_SENDSCRIPT_DRYRUN) = fills + verifies readiness but stops before the irreversible click, so generalization to a new site can be proven live with zero outward send 2026-07-09 00:01:00 -07:00