WEBDIENSTE/openswarm
1
0
Fork 0
mirror of https://github.com/openswarm-ai/openswarm.git synced 2026-09-22 09:34:53 +02:00
Code Issues Packages Projects Releases Wiki Activity

[eric] browser: steer paginated gathers to sweep in batch (RepeatFlow + sessionStorage / scroll-then-read), not one page per turn

Browse Source
This commit is contained in:
ciregenz
2026-06-09 02:39:31 -07:00
parent bfcad98b76
commit e24a63c13c
1 changed files with 9 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files
backend/apps/agents/browser/browser_schema.py
+9 -5
View File
@@ -859,11 +859,15 @@ SYSTEM_PROMPT = (
"comes back empty or shaped wrong, STOP, that is your signal to switch to BrowserExtract, "
"not to debug another selector. The answer must end up IN your Done message, so gather it "
"for real; never report done on a gather task you couldn't actually read.\n"
"- Spanning MANY pages (get all N, every result)? Confirm the page shape ONCE, then COMMIT "
"to the sweep: don't re-verify each page works. The site usually exposes far fewer than a "
"round number asks (a '1000' is often ~15 pages); gather every page it does expose, then "
"Done with the full set and a one-line note on the real ceiling. Accumulate into a page "
"variable as you go (e.g. window.__rows) so nothing is lost between pages.\n"
"- Getting ALL N across MANY pages? Sweep in BATCH, never one page per turn (that is how 40 "
"turns yields only 2 pages, the slowest possible way). Two shapes: (a) ONE page that loads "
"more as you SCROLL (a feed): scroll to the very end, THEN read every row into a page "
"variable in a single pass. (b) SEPARATE pages you NAVIGATE between (page-2 / cursor URLs): "
"build the list of page URLs, then use BrowserRepeatFlow over them with a [navigate, evaluate] "
"step whose evaluate MERGES that page's rows into sessionStorage (it survives same-origin "
"navigation; a plain window var is wiped on every navigate). Either way you finish with the "
"whole set in ONE place. Sites expose far fewer than round numbers ask (a '1000' is often "
"~15 pages); take everything it exposes and note the real ceiling in your Done.\n"
"- A BIG result set (hundreds of rows) does NOT fit in your reply, it truncates. Do NOT "
"chunk it back through your messages 100 at a time. Once it's gathered into a page variable, "
"call BrowserSaveData('JSON.stringify(window.__rows)', 'results.json') ONCE: it writes the "