ciregenz
1f74cc13de
[eric] browser: general capture-replay write tier (site's own captured route, same-origin + gated, default off)
2026-07-14 23:47:17 -07:00
ciregenz
42a3249976
[eric] browser: broaden compose-trigger vocabulary (create/new-thread/start-a-thread variants)
...
The reveal opener regex only matched 'create a post', missing bare 'Create' and 'New thread' / 'Start a thread' buttons that real compose surfaces use. Adds those; safe by construction (a broadened trigger can never produce a false success, a fill against a real composer is still required, so a trigger that opens a signup/join upsell instead of a composer yields found:false). Validated CDP-direct: LinkedIn-post and YouTube-comment still flip (no regression), and Threads (which gates posting behind a 'Join Threads' upsell for this session) correctly does NOT false-fill
2026-07-13 14:59:21 -07:00
ciregenz
937bb35dca
[eric] browser: evalInPage runs via CDP Runtime.evaluate, not webContents.executeJavaScript
...
The executeJavaScript path is SUSPENDED by Electron until a page stops loading, so on a page whose trackers never let it settle (Reddit, LinkedIn) every in-page command queues into the 15s backend timeout = the mount-poll wedge that has blocked every agent-loop run. CDP Runtime.evaluate runs in the browser process, out of the renderer's load-blocked event loop, so it does not suspend. This reuses the already-in-production CDP bridge (sendCdp -> send-cdp-command -> wc.debugger.sendCommand, keyed by getWebContentsId, with attach + per-wc serialization + timeout already built) that the AX/click commands use; only the eval path was left on executeJavaScript. No-regression by construction: on any CDP INFRA failure (debugger can't attach because DevTools or a remote-debugging port holds the webContents, or the bridge errors) it falls back to the original executeJavaScript path, and a real page-side exception still surfaces as a throw; window.__OSW_CDP_EVAL__ = false forces the old path. Unwrap logic extracted to cdpEval.ts + 6 unit tests. NOTE not live-demonstrable on a --remote-debugging-port bench: that port globally blocks wc.debugger.attach, so the whole app CDP path (this AND the existing list_interactives) times out there; production and normal dev do not use the port, so attach works and the existing CDP commands prove it
2026-07-12 21:55:59 -07:00
ciregenz
82fe16767c
[eric] browser: handleType reads back its fill and falls back to real keystrokes = the main type action now detects when execCommand insertText did not commit (editors that reject synthetic input) and retypes via wv.sendInputEvent, same lever as the composer finder's keystrokeFill
2026-07-12 21:26:49 -07:00
ciregenz
19a3e8ad1a
[eric] browser: keystroke-fill fallback + cross-nav reveal retry (both flag-gated, default off)
...
Item 1: when the finder's execCommand insertText read-back fails (editors that reject synthetic input), retype the same text as real OS-level key events via wv.sendInputEvent, then verify from the marked element OR document.activeElement (editors like Reddit swap the node on activation, staling the selector). Item 3: when an open-first reveal navigates to a new document (Reddit thread, TikTok video, GitHub issue), the send-script re-perceives after a load beat and calls the finder once more on the destination, bounded to 2 tries. 4 new tests (cross-nav retry fires / no wasted retry) + structural tests split into test_browser_reveal.py to stay under the 300-line cap. Live real-path validation of the sendInputEvent leg is owed on a healthy rig: the eric/browser agent loop wedges at fresh-card webview mount (15.25s cap, renderer starvation, predates the suspend fix) at low load too
2026-07-12 20:52:15 -07:00
ciregenz
a3099f035c
[eric] browser: reveal-and-find gains activation-click + shadow-safe collection = a placeholder composer that fills empty (YouTube 'Add a comment') gets clicked, rescanned, and the real editor filled; deepMatch collects only selector matches (few hundred) not every node (the old all-node 4000 cap truncated before a heavy SPA's late-in-DOM composer, which hid X's OWN box); openFirstItem broadened to chat/message lists (data-testid=conversation, message/chat links) and pierces shadow DOM; CDP-direct n=2 reproduced: reveal converts LinkedIn post (modal) + YouTube comment (activation), controls stay filled
2026-07-12 20:03:05 -07:00
ciregenz
64675115f8
[eric] browser: reveal-and-find reach primitive (OSW_COMPOSER_REVEAL, default off) = when no composer is painted the finder takes ONE reversible reveal action (click a scored compose-trigger, open the first list item, or scroll) and rescans, bounded; SPA route changes keep the JS context alive so X/LinkedIn/Reddit/YouTube stay drivable in one call; never clicks an irreversible control (hard-block send/submit/pay/delete); send-script forwards reveal only under the flag; 2 tests (reveal forwarded when on, false by default)
2026-07-12 19:29:07 -07:00
ciregenz
2d49687ae0
[eric] browser: composer finder pierces shadow DOM (Reddit shreddit, Telegram, WhatsApp, Slack build their composer inside web-component shadow roots a light-DOM querySelectorAll can't see); bounded recursive walk (depth 8, 400-node cap). Live finder-probe on real DOMs: correctly picks X's contenteditable 'Post text' + excludes the search combobox; the remaining wall is that lazy/dialog/chat-gated composers aren't rendered at scan time (detection is downstream of render)
2026-07-10 00:56:05 -07:00
ciregenz
4c85dd6314
[eric] browser: structural composer finder (OSW_COMPOSER_STRUCT, default off) = an in-page primitive that ranks editable regions (contenteditable/textarea/text-input near a Send/Post/Reply control, excluding search boxes) instead of matching AX names, marks the winner + fills+reads-back in-page (the only reliable commit-check for a React contenteditable whose text never reaches the AX tree); send-script uses it as a fallback when the name detector misses, unblocking unnamed/non-standard composers (Reddit Lexical, X DM); 3 tests, all safety gates + dry-run stop unchanged
2026-07-10 00:13:44 -07:00
ciregenz
e0162779a7
[eric] browser: gate document-order display behind OSW_DOC_ORDER (default on) so the A/B can measure it against legacy rank-order; 14 ranking tests
2026-07-08 14:57:24 -07:00
ciregenz
2234ae46de
[eric] browser: interactives render in DOCUMENT order (rank still picks who survives the cap, but display is top-to-bottom page order); the scrambled list forced the model to burn orientation turns reading page text to recover position on ordinal tasks ('the 4th story' landed at [48]), the real cause of the slow HN planning
2026-07-08 14:00:02 -07:00
ciregenz
3123e9a1b4
[eric] browser: evict a dead card's webview BEFORE recovery spawns a fresh one (the recovery-card wedge: two heavy pages starve the renderer event loop; unmounting the wedged one leaves the recovery card the only heavy neighbor); immediate removeBrowserCard (no fade/keep-pill) + layout drop, gated by OSW_DEADCARD_EVICT, 2 tests
2026-07-07 17:20:57 -07:00
ciregenz
b60f0b94aa
[eric] browser: wrong-target detectability probe = the click hit-test now reports WHAT element was actually under the click point, so a click that lands on the wrong element (occluded/stale/moved, and still changes the page) is finally measurable, the failure a page-change metric can't see
2026-07-07 16:47:47 -07:00
ciregenz
2b63953650
[eric] browser: click-effect probe wraps ALL clickBackendNode exits (was only on the coordinate branch; real-site clicks exit via the covered-element JS path, so the metric captured ~none)
2026-07-07 15:48:37 -07:00
ciregenz
8f2565e027
[eric] browser: thread self-heal + click-effect metric into BATCH sub-clicks too (most clicks are batched; gating only top-level click_index measured ~none of them) + scan batch sub-results for the telemetry
2026-07-07 15:41:07 -07:00
ciregenz
f54f474de7
[eric] browser: click-effect metric (measure before fixing) = fingerprint the page before/after a click to catch the INVISIBLE failure the tool-error counter misses (a click that succeeds but hits the wrong/dead element = nothing changes); flag-gated OSW_CLICK_EFFECT_PROBE, zero cost off, 3 unit tests
2026-07-07 15:33:33 -07:00
ciregenz
5ca7308fa5
[eric] browser: self-healing click (Phase 1) = a failed cached-index click auto-escalates to fresh by-name resolution in-code before the model ever sees the error (stale index = ~half of all runs' tool-errors, each costs a ~3s re-strategize turn); error-triggered so no double-act, OSW_SELFHEAL_CLICK=0 kill switch, 5 unit tests
2026-07-07 15:05:41 -07:00
ciregenz
46c300e4ee
[eric] browser: unwedge stuck-loading webviews (executeJavaScript queues until load stops; recaptcha/tracker stragglers held isLoading for minutes = every command timed out); guarded eval stops stragglers only after dom-ready
2026-07-07 02:18:13 -07:00
ciregenz
53e3a4462a
Merge remote-tracking branch 'origin/eric/dev' into eric/browser
...
# Conflicts:
# backend/apps/agents/providers/registry.py
# backend/tests/test_v2_invariants.py
2026-07-06 18:37:26 -07:00
ciregenz
1b96bc597c
[eric] context: lossless tool-report spill (browser+gws caps point at the full file) + visible recovery pill
2026-07-06 14:16:31 -07:00
ciregenz
ead27dabcc
[eric] paste-dialog: uncontrolled editor (large pastes typed like molasses through controlled MUI autosize)
2026-07-05 16:19:12 -07:00
ciregenz
452b232637
[eric] merge #119 : pasted-text cards editable with autosave
2026-07-05 16:14:57 -07:00
ciregenz
f3b7df3f4d
[eric] palette: Go to Skills/Actions opens Settings tabs; drop Go to Modes (routes died in #118 + modes-removal)
2026-07-03 15:53:55 -07:00
ciregenz
45084403a3
[eric] merge #118 : move customization into settings (rebased over modes-removal + apps fold-in)
2026-07-03 15:52:05 -07:00
ciregenz
85e0273191
[eric] models: drop Gemini 3 Flash rows (flaky, 429-throttled direct-API lane); keep ag/gemini-3-flash aux
2026-07-03 14:56:08 -07:00
ciregenz
3fe8599a22
[eric] chat: restore the model-picker trigger (was inside the removed mode pill)
2026-07-03 03:12:26 -07:00
ciregenz
25c944cc70
[eric] modes: hide the mode system from the UI (composer picker + sidebar); default agent
2026-07-02 21:39:59 -07:00
ciregenz
dfd7844f44
[eric] apps: card header gets open-another (+) and collapse-to-nothing toolbar toggle
2026-07-02 21:39:59 -07:00
ciregenz
40d975c702
[eric] apps: fix 2nd instance on old workspaces (port shim) + dedupe header keys
2026-07-02 21:39:59 -07:00
ciregenz
9603912237
[eric] merge eric/app-ux: app builder UX + browser tab drag-out + CreateApp fold-in
2026-07-02 16:10:56 -07:00
ciregenz
227bd1badc
[eric] apps: fold App Builder into agents via CreateApp; delete standalone /apps page + editor
2026-07-02 03:20:52 -07:00
abccodes
2b806242f2
[aidan] ux/large-text-paste: make pasted-text cards editable with autosave
2026-07-02 02:45:52 -07:00
ciregenz
df603a0b75
[eric] browser: drag a tab out of the strip (drop on a card absorbs it, drop on canvas opens a new browser)
2026-07-02 01:29:32 -07:00
ciregenz
85d77ca552
[eric] apps: multiple independent instances per app (own runtime + ports; alt-click or paste to open)
2026-07-02 01:24:22 -07:00
ciregenz
a4e81a8f2f
[eric] hitl: Approve All persists always-allow so mid-run identical commands stop re-prompting
2026-07-02 00:41:54 -07:00
ciregenz
7076af1e6b
[eric] canvas: cmd+scroll zooms over app cards (interactive webviews + srcdoc iframes)
2026-07-02 00:38:06 -07:00
ciregenz
f19284a456
[eric] apps: card Preview/Code/Terminal switcher, terminal hard-reload, agent-readable terminal.log
2026-07-02 00:35:13 -07:00
ciregenz
f8620d62d0
[eric] actions: sign-in button + live signed-in status for reddit/x/tiktok browser-login MCPs
2026-06-30 19:41:13 -07:00
ciregenz
f24e6ecc52
[eric] mcp: wire X + TikTok into catalog, preflight, env injection + linter exceptions
2026-06-30 17:19:58 -07:00
ciregenz
c04a663996
[eric] browser-bridge: /api/browser-session/action + renderer perform_action + shim client
2026-06-30 17:19:58 -07:00
abccodes
9cda4a8721
[aidan] ux/sidebar-settings: move customization into settings
2026-06-30 17:00:27 -07:00
ciregenz
63c32554ae
[eric] mcp: reddit session-borrow MCP (free, full human action set; WIP)
2026-06-29 23:05:15 -07:00
ciregenz
8eb48dfc42
[eric] app-use: strip pierre's transient [app-agent] trace scaffolding + write-only controls cache (no speculative scaffolding)
2026-06-29 23:03:45 -07:00
ciregenz
1b6d747326
[eric] merge #107 (pierre): app-use OPENSWARM_APP bridge + AppAgent
...
# Conflicts:
# electron/main.js
# frontend/src/app/pages/Views/ViewPreview.tsx
2026-06-29 21:34:11 -07:00
ciregenz
197fe258ab
[eric] merge #113 (aidan): skills agent-discovery + multi-file install + versioning
2026-06-29 21:26:00 -07:00
ciregenz
bd8cdbfb0b
[eric] browser: drive the loading bar off main-frame navigation only, so ad/sub-frame loads don't re-sweep it
2026-06-29 20:55:05 -07:00
ciregenz
9f2041ca58
[eric] browser: loading bar is an overlay so it doesn't jitter the webview up/down on busy pages
2026-06-29 20:54:54 -07:00
ciregenz
8784ca937c
[eric] browser/apps: drop the iframe fallback for a 'run in the desktop app' message (keeps srcdoc + Windows-crash iframe)
2026-06-29 16:38:03 -07:00
ciregenz
6afbc5b849
[eric] browser: throttle in-page URL mirroring so map-churn pages don't storm dashboard saves
2026-06-29 16:37:51 -07:00
abccodes
07a6c9ab70
[aidan] feat/skill-registry: multi-file install, tree cache, and skill versioning
2026-06-29 02:13:13 -07:00
ciregenz