WEBDIENSTE/openswarm
1
0
Fork 0
mirror of https://github.com/openswarm-ai/openswarm.git synced 2026-09-11 12:17:45 +02:00
Code Issues Packages Projects Releases Wiki Activity

[shawn] chore: merge origin/dev into mcp-integrations, resolve conflicts

Browse Source 
Brings the branch current with dev (was 18 behind) so the PR
squash-merges into dev with zero conflicts. Resolved 6 conflicts:

- backend/main.py: keep both workflows and telegram_bot SubApps
- backend/auth.py: keep the Spotify OAuth callback path exemptions
- backend/apps/agents/agent_manager.py: take the deletions of
  _build_connected_tools_context, _approx_tokens, and
  _summarize_message_block (confirmed dead, zero callers on dev)
- backend/apps/tools_lib/tools_lib.py: keep both dev's
  google_oauth_token_proxy and the MCP credential endpoints
- electron/main.js: keep both the Instagram MCP CLI helpers and
  dev's one-line waitForBackend comment
- frontend/src/app/pages/Tools/Tools.tsx: keep the credential dialogs

Also widens the Tools.tsx snackbar severity union to include
'warning', a value its own code already passes (a pre-existing type
error surfaced once dev was merged and tsc was run).

Verified: 832 backend tests pass, webpack build succeeds, tsc clean
except one pre-existing allowpopups error in dev's BrowserCard.tsx.
This commit is contained in:
TheAchiever6823
2026-05-20 03:15:04 -07:00
parent 49130d9351 4eab66fc66
commit fe0c3af0f2
265 changed files with 15290 additions and 8324 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/workflows/gitleaks.yml
+8 -1
View File
@@ -59,7 +59,14 @@ jobs:
fi
if [ -n "$BEFORE" ] && [ "$BEFORE" != "$AFTER" ]; then
gitleaks detect --source . --redact --verbose --no-banner --log-opts="${BEFORE}..${AFTER}"
elif [ "$BEFORE" = "$AFTER" ]; then
# New-branch push pointing at an existing main commit: range
# is empty, nothing new to scan. Don't fall through to a full
# repo scan — that would re-flag every historical secret a
# past PR already cleared. Pass-through.
echo "No new commits on this branch vs main; skipping scan."
else
# Couldn't establish a range — full scan as fallback.
# Truly couldn't establish a range (e.g. orphan branch with
# no shared history). Full scan is the only safe option.
gitleaks detect --source . --redact --verbose --no-banner
fi
.github/workflows/release-windows.yml
+10
View File
@@ -110,6 +110,16 @@ jobs:
$ErrorActionPreference = 'Stop'
$shouldPublish = ($env:GITHUB_EVENT_NAME -eq 'push') -or `
($env:GITHUB_EVENT_NAME -eq 'workflow_dispatch' -and $env:PUBLISH_INPUT -eq 'true')
# electron-builder auto-detects prerelease from semver suffix in electron/package.json,
# but EP_PRE_RELEASE forces the GitHub Releases publisher to mark it Pre-release even
# when the runner's environment differs from local. Set it whenever the version has a "-" suffix.
$version = (Get-Content electron/package.json | ConvertFrom-Json).version
if ($version -match '-') {
$env:EP_PRE_RELEASE = 'true'
Write-Host "Version $version is EXPERIMENTAL; setting EP_PRE_RELEASE=true"
} else {
Write-Host "Version $version is STABLE"
}
if ($shouldPublish) {
Write-Host "Build mode: PUBLISH"
pwsh -NoProfile -File scripts\build-app-win.ps1 -Publish
CONTRIBUTING.md
+191
View File
@@ -0,0 +1,191 @@
# Contributing to OpenSwarm
A guide for all OpenSwarm contributors.
## Branches
There are two protected branches:
- **`main`**: the stable, production-ready branch. Every merge to `main` represents a versioned release. Never commit directly to it from any branch that is not **`dev`**.
- **`dev`**: the active development branch. All feature branches merge here first. This is where work-in-progress code lives and gets tested before release.
Never commit directly to either branch. Every change, no matter how small, gets its own branch and pull request.
### Naming format
```
yourname/type/short-description
```
All lowercase, hyphens between words. Keep it short but descriptive.
| Prefix | When to use | Example |
| --- | --- | --- |
| `feat/` | New feature | `haik/feat/add-dark-mode` |
| `fix/` | Bug fix | `arnav/fix/login-crash` |
| `refactor/` | Restructuring code without changing behavior | `cire/refactor/cleanup-auth` |
| `docs/` | Documentation only | `haik/docs/update-readme` |
| `chore/` | Build scripts, CI, dependencies, tooling | `arnav/chore/update-deps` |
### Creating a branch
```bash
git checkout dev
git pull
git checkout -b yourname/feat/my-feature
```
Always branch off of the latest `dev`.
## Commits
### Format
```
[yourname] type: short description in imperative mood
```
### Examples
```
[bob] feat: add user profile page
[bob] fix: prevent crash when token expires
[bob] refactor: split auth into separate module
[bob] docs: add setup instructions to README
[bob] chore: upgrade node to v22
```
### Rules
- Start with `[name] type:` prefix (same list as branches above).
- Use imperative mood. "add" not "added", "fix" not "fixed".
- One commit = one logical unit of work.
## The Workflow
### For day-to-day development
1. **Pull latest dev**
```bash
git checkout dev && git pull
```
2. **Create a branch**
```bash
git checkout -b yourname/feat/my-feature
```
3. **Do your work, commit as you go**
```bash
git add .
git commit -m "[yourname] feat: whatever you did"
```
4. **Push your branch**
```bash
git push
```
5. **Open a Pull Request on GitHub**
base: `dev`, compare: `yourname/feat/my-feature`.
6. **Wait for review and approval.**
7. **The maintainer merges it into `dev`** (branches are deleted automatically after merge).
### For outside contributors (people not on the core team)
1. Fork the repo (creates your own copy).
2. Clone your fork.
3. Create a branch off of `dev` and do your work (same naming conventions).
4. Push to your fork.
5. Open a Pull Request from your fork to the main repo's `dev` branch.
6. Wait for review and approval.
## Pull Requests
### Title
Use the same format as commits:
```
[yourname] feat: add dark mode toggle
[yourname] fix: resolve crash on empty input
```
### Description
Write a short explanation of what the change does and why. Two to three sentences is enough. If the change is visual, include a screenshot.
### Scope
One logical change per PR. Do not bundle unrelated work. A bug fix and a new feature should be separate PRs, even if you noticed the bug while building the feature.
## Merging
All PRs into `dev` are merged using **squash and merge**. This takes all the commits in your PR and combines them into one clean commit on `dev`. This keeps the history readable even if your branch had many small or messy commits.
Only the maintainer (i.e. Eric) merges PRs. Do not merge your own work (unless ur Eric).
### Squash and Merge
When you have a branch with, say, 5 commits:
```
feat: start building login page
fix: typo in login form
feat: add password validation
fix: forgot to import useState
feat: finish login page styling
```
**Squash and merge** takes all 5 of those and combines them into a single commit when merging the PR:
```
feat: add login page (#12)
```
So `dev` gets one clean commit instead of messy work-in-progress history. The full commit history still exists on the PR page if anyone ever needs to look at it.
**How it works:** You don't do anything special. When you click the green "Merge pull request" button on a PR, there's a dropdown arrow next to it. Pick "Squash and merge" from that dropdown. GitHub then asks you to write the final squashed commit message before confirming.
**Does it happen by default?** No. GitHub defaults to a regular merge commit. But you can change this in repo settings:
1. Go to repo **Settings > General**.
2. Scroll to **Pull Requests**.
3. Uncheck "Allow merge commits".
4. Uncheck "Allow rebase merging".
5. Keep only **"Allow squash merging"** checked.
After that, squash and merge is the only option anyone sees. No dropdown to pick from, no way to accidentally do a regular merge.
*Note: this has already been set up in our repo settings, so we're good to go. If this ever needs to be modified, call Haik.*
## Releases
When `dev` has accumulated enough changes and is stable, the maintainer merges `dev` into `main` via a PR. Every merge to `main` represents a versioned release.
### Flow
```
feature branches -> PR into dev -> test and stabilize -> PR from dev into main -> tag a release
```
### Versioning
Releases use semantic versioning:
| Change type | Version bump | Example |
| --- | --- | --- |
| Bug fixes, plus modifications or additions to existing features | Patch | `v1.0.0` -> `v1.0.1` |
| Completely new features (backwards compatible) | Minor | `v1.0.0` -> `v1.1.0` |
| Breaking changes | Major | `v1.1.0` -> `v2.0.0` |
## Quick Reference
| Action | Command |
| --- | --- |
| Update your local dev | `git checkout dev && git pull` |
| Create a new branch | `git checkout -b yourname/type/description` |
| Stage all files | `git add .` |
| Commit | `git commit -m "[yourname] type: description"` |
| Push a new branch | `git push -u origin yourname/type/description` |
| Push subsequent commits | `git push` |
| See who wrote a line | `git blame filename` |
| See commit history | `git log --oneline` |
| See your current branch | `git branch` |
| Switch to an existing branch | `git checkout branch-name` |
backend/apps/agents/9router_gpt5_patch.js
+14 -71
View File
@@ -1,34 +1,6 @@
// Node-runtime patch loaded via `node --require <this>` before 9router boots.
//
// Why this exists
// ---------------
// OpenAI's GPT-5 family (gpt-5.4, gpt-5.4-mini, gpt-5.5, gpt-5.3-codex, …)
// rejects the legacy `max_tokens` parameter with HTTP 400, requiring
// `max_completion_tokens`. 9router (every released version, including 0.4.20)
// blindly forwards `max_tokens` in its Anthropic→OpenAI translator. We can't
// fix 9router from outside (env vars are ignored, baseUrl on the openai
// provider is hardcoded, prefix routing falls back). Instead we intercept
// the HTTPS write at the Node syscall layer — the actual boundary OpenAI
// sees — and rename the field on the way out.
//
// Safety contract
// ---------------
// • Scope: only requests whose hostname is `api.openai.com`. Every other
// outbound HTTP/HTTPS call passes through unmodified.
// • Model gate: only requests whose body parses as JSON with
// `model.startsWith("gpt-5")` (after stripping common prefixes 9router
// adds). GPT-4 / Claude / etc. unaffected.
// • Failure mode: every step is wrapped in try/catch and falls back to the
// unmodified original on any error. Worst case is "request behaves
// exactly as it would without this patch" — never worse than baseline.
// • Idempotency: the patch self-flags so re-loading via multiple --require
// doesn't double-wrap.
//
// Verification
// ------------
// Set OPENSWARM_DEBUG_GPT5_PATCH=1 in the env to log "[openswarm] 9router-
// gpt5-patch installed" on stderr and "rewrote max_tokens → max_completion_tokens"
// on each rewrite.
// Rewrites `max_tokens` to `max_completion_tokens` for GPT-5 calls (which 9router still emits) and floors completion tokens at 32K for reasoning headroom.
// Hostname-gated to api.openai.com; every step is try/catch so failure falls back to baseline behavior.
'use strict';
@@ -40,7 +12,7 @@ const DEBUG = process.env.OPENSWARM_DEBUG_GPT5_PATCH === '1';
function _log(msg) {
if (DEBUG) {
try { process.stderr.write('[openswarm-gpt5-patch] ' + msg + '\n'); } catch (_) { /* ignore */ }
try { process.stderr.write('[openswarm-gpt5-patch] ' + msg + '\n'); } catch (_) {}
}
}
@@ -48,9 +20,7 @@ function isGpt5Model(model) {
if (typeof model !== 'string') return false;
let m = model.trim().toLowerCase();
if (!m) return false;
// Strip routing prefixes 9router may have added: cp-openai/, openai/,
// cx/, openrouter/, or:openai/. Don't strip cp- (custom-provider) blindly
// because cp-anything could match a non-OpenAI custom node.
// Strip 9router prefixes; don't blindly strip cp- (could be a non-OpenAI custom node).
const prefixes = ['cp-openai/', 'openai/', 'cx/', 'openrouter/', 'or:openai/'];
for (const p of prefixes) {
if (m.startsWith(p)) { m = m.slice(p.length); break; }
@@ -58,19 +28,7 @@ function isGpt5Model(model) {
return m.startsWith('gpt-5');
}
// Minimum completion-token budget for GPT-5 reasoning models.
// GPT-5 burns 8-30K tokens on internal reasoning BEFORE producing any
// user-visible output. The Anthropic CLI's default max_tokens (~4096) is
// way under that floor — OpenAI accepts the request, runs reasoning until
// it hits the cap, then returns "Could not finish the message because
// max_tokens or model output limit was reached" with zero user-visible
// content. Floor at 32K so reasoning has room AND the user gets an
// actual response. Cost is unaffected because OpenAI bills for
// tokens-consumed, not max_completion_tokens (which is just a cap).
//
// We use max(requestedValue, 32K) — never lower the user's value, only
// raise it. If the user explicitly sets a high value (e.g. 100K) we
// honor it untouched.
// GPT-5 burns 8-30K reasoning tokens before any output; the CLI's default 4096 caps before content lands. Floor at 32K and only raise, never lower.
const GPT5_MIN_COMPLETION_TOKENS = 32768;
function maybeRewriteBody(bodyStr) {
@@ -80,8 +38,7 @@ function maybeRewriteBody(bodyStr) {
if (!parsed || typeof parsed !== 'object' || Array.isArray(parsed)) return bodyStr;
if (!isGpt5Model(parsed.model)) return bodyStr;
let mutated = false;
// Both fields present (unlikely but possible): drop the legacy one so
// OpenAI doesn't reject for "both specified".
// Drop legacy field if both present, else OpenAI 400s on "both specified".
if ('max_tokens' in parsed && 'max_completion_tokens' in parsed) {
delete parsed.max_tokens;
mutated = true;
@@ -90,15 +47,13 @@ function maybeRewriteBody(bodyStr) {
parsed.max_completion_tokens = parsed.max_tokens;
delete parsed.max_tokens;
mutated = true;
_log('rewrote max_tokens → max_completion_tokens for ' + parsed.model);
_log('rewrote max_tokens to max_completion_tokens for ' + parsed.model);
}
// Floor max_completion_tokens at 32K for reasoning headroom. Only raise,
// never lower — if the user explicitly set 100K, keep 100K.
if (typeof parsed.max_completion_tokens === 'number' && parsed.max_completion_tokens < GPT5_MIN_COMPLETION_TOKENS) {
const orig = parsed.max_completion_tokens;
parsed.max_completion_tokens = GPT5_MIN_COMPLETION_TOKENS;
mutated = true;
_log('raised max_completion_tokens ' + orig + ' → ' + GPT5_MIN_COMPLETION_TOKENS + ' for ' + parsed.model + ' (reasoning headroom)');
_log('raised max_completion_tokens ' + orig + ' to ' + GPT5_MIN_COMPLETION_TOKENS + ' for ' + parsed.model);
}
return mutated ? JSON.stringify(parsed) : bodyStr;
}
@@ -112,7 +67,6 @@ function _hostFromOpts(opts) {
function patchHttpRequest(orig) {
return function patchedRequest() {
const args = Array.prototype.slice.call(arguments);
// First arg may be a URL string, URL object, or options object.
let opts = args[0];
let host = '';
try {
@@ -125,33 +79,28 @@ function patchHttpRequest(orig) {
return orig.apply(this, args);
}
// Outbound request to OpenAI: intercept body. The Anthropic SDK and
// 9router both call .write(body) then .end(), or .end(body) directly.
let req;
try { req = orig.apply(this, args); } catch (e) { throw e; }
const origWrite = req.write.bind(req);
const origEnd = req.end.bind(req);
const chunks = [];
let isStringMode = null; // null until first chunk; then true=string, false=buffer
let isStringMode = null;
function recordChunk(chunk) {
if (chunk == null) return;
if (typeof chunk === 'string') {
if (isStringMode === false) {
// Mixed mode — fall back: convert prior buffers to string
for (let i = 0; i < chunks.length; i++) chunks[i] = chunks[i].toString('utf8');
}
isStringMode = true;
chunks.push(chunk);
} else if (Buffer.isBuffer(chunk)) {
if (isStringMode === true) {
// Mixed: convert prior strings to buffers
for (let i = 0; i < chunks.length; i++) chunks[i] = Buffer.from(chunks[i], 'utf8');
}
isStringMode = false;
chunks.push(chunk);
} else {
// Unknown shape — abandon interception
throw new Error('unknown-chunk-shape');
}
}
@@ -162,12 +111,10 @@ function patchHttpRequest(orig) {
recordChunk(chunk);
return true;
} catch (_) {
// Abandon interception — pass through immediately and disable buffering.
// Flush anything we'd buffered so far.
try {
for (const c of chunks) origWrite(c);
chunks.length = 0;
} catch (_) { /* ignore */ }
} catch (_) {}
return origWrite.apply(req, [chunk].concat(restArgs));
}
};
@@ -186,19 +133,17 @@ function patchHttpRequest(orig) {
if (req.getHeader && typeof req.getHeader === 'function' && req.getHeader('content-length')) {
req.setHeader('Content-Length', newBuf.length);
}
} catch (_) { /* ignore */ }
} catch (_) {}
return origEnd.call(req, newBuf);
}
// No rewrite — send original body intact
if (chunks.length === 0) return origEnd.apply(req, restArgs);
if (isStringMode === true) return origEnd.call(req, chunks.join(''));
return origEnd.call(req, Buffer.concat(chunks));
} catch (_) {
// Abandon — flush any buffered content + tail chunk
try {
for (const c of chunks) origWrite(c);
chunks.length = 0;
} catch (_) { /* ignore */ }
} catch (_) {}
if (chunk != null) return origEnd.apply(req, [chunk].concat(restArgs));
return origEnd.apply(req, restArgs);
}
@@ -216,13 +161,11 @@ if (!_https.__openswarm_gpt5_patched) {
_http.__openswarm_gpt5_patched = true;
_log('installed https.request + http.request interceptors');
} catch (e) {
// Patch failed — log and continue. 9router will work as normal,
// GPT-5 calls will fail with the same 400 they did before. Never worse.
_log('install failed: ' + (e && e.message ? e.message : String(e)));
}
}
// Also patch global fetch (Node 18+). 9router uses fetch in some paths.
// Node 18+ fetch path; 9router uses fetch in some routes.
if (typeof globalThis.fetch === 'function' && !globalThis.fetch.__openswarm_gpt5_patched) {
try {
const origFetch = globalThis.fetch;
@@ -249,7 +192,7 @@ if (typeof globalThis.fetch === 'function' && !globalThis.fetch.__openswarm_gpt5
if (k.toLowerCase() === 'content-length') newInit.headers[k] = newLen;
}
}
} catch (_) { /* ignore */ }
} catch (_) {}
}
return origFetch.call(this, input, newInit);
}
backend/apps/agents/agent_manager.py
+521 -146
View File
File diff suppressed because it is too large Load Diff
backend/apps/agents/agents.py
+34 -78
View File
@@ -11,12 +11,7 @@ import logging
logger = logging.getLogger(__name__)
# In-flight dedup map for generate-group-meta. Keyed by (session_id, group_id).
# When the frontend issues N concurrent requests for the same group (which it
# can during heavy streaming), we only fire ONE upstream Anthropic call and
# return the same Future to all callers. Eliminates the 429 thundering herd
# without changing retry/fallback semantics — each unique (session, group)
# still gets its full retry budget, just not multiplied by N callers.
# Dedup concurrent generate-group-meta calls; collapses the 429 thundering herd by sharing one upstream Future per (session, group).
_group_meta_inflight: dict[tuple[str, str], asyncio.Future] = {}
@asynccontextmanager
@@ -32,7 +27,6 @@ async def agents_lifespan():
agents = SubApp("agents", agents_lifespan)
# REST Endpoints
@agents.router.get("/sessions")
async def list_sessions(dashboard_id: str = ""):
@@ -41,9 +35,23 @@ async def list_sessions(dashboard_id: str = ""):
@agents.router.get("/sessions/{session_id}")
async def get_session(session_id: str):
"""Returns the session by id.
Falls back to a disk load when the session isn't in the in-memory
dict. Without this, any surface that queries a session before the
dashboard has restored it (Apps editor opened cold, deep link to a
chat, a workflow step inspecting an old session) hits a 404 even
though the JSON file is sitting on disk. The disk-load path is
O(1) memory hit after the first call: resume_session moves the
session into agent_manager.sessions and the next GET short-circuits
on the in-memory check.
"""
session = agent_manager.get_session(session_id)
if not session:
raise HTTPException(status_code=404, detail="Session not found")
try:
session = await agent_manager.resume_session(session_id)
except ValueError:
raise HTTPException(status_code=404, detail="Session not found")
return session.model_dump(mode="json")
@agents.router.post("/launch")
@@ -57,12 +65,7 @@ async def send_message(session_id: str, body: dict):
if not prompt:
raise HTTPException(status_code=400, detail="prompt is required")
# Pre-flight MCP suggestion (Phase 3, Layer N). Runs in parallel with
# the agent launch path — if it produces suggestions, they're
# surfaced inline in the chat via agent:mcp_suggestions WS event.
# Fails open: any error from the classifier is swallowed and the
# agent proceeds normally. The classifier is short-circuited for
# obviously-local prompts (greetings, shell commands, file paths).
# Run MCP-suggestion classifier in parallel with the agent launch; fails open.
try:
from backend.apps.agents.mcp_preflight import run_preflight
from backend.apps.agents.ws_manager import ws_manager as _ws
@@ -79,7 +82,6 @@ async def send_message(session_id: str, body: dict):
except Exception:
pass
# Non-blocking — don't gate the agent on the classifier.
import asyncio as _asyncio
_asyncio.create_task(_emit_preflight())
except Exception:
@@ -111,6 +113,7 @@ async def handle_approval(response: ApprovalResponse):
"behavior": response.behavior,
"message": response.message,
"updated_input": response.updated_input,
"trust_pattern": response.trust_pattern,
})
return {"ok": True}
@@ -146,11 +149,7 @@ async def generate_group_meta(session_id: str, body: dict):
if not group_id or not tool_calls:
raise HTTPException(status_code=400, detail="group_id and tool_calls are required")
# In-flight dedup. If an identical request is already running, await its
# result instead of firing another Anthropic call. This is the entire fix
# for the 429 storm we were seeing — N concurrent identical requests
# collapse to 1 upstream call. Refinement requests bypass dedup since
# they may legitimately want fresh results with different inputs.
# Dedup: share an in-flight Future across callers; refinement requests bypass since they may want fresh results.
is_refinement = body.get("is_refinement", False)
key = (session_id, group_id)
if not is_refinement:
@@ -159,8 +158,7 @@ async def generate_group_meta(session_id: str, body: dict):
try:
return await existing
except Exception:
# If the in-flight call failed, fall through and try again
# ourselves rather than propagating someone else's error.
# In-flight call failed; retry ourselves rather than propagate someone else's error.
pass
future: asyncio.Future = asyncio.get_event_loop().create_future()
@@ -182,7 +180,6 @@ async def generate_group_meta(session_id: str, body: dict):
future.set_exception(e)
raise
finally:
# Always clear our slot if we own it, so the next request runs fresh.
if not is_refinement and _group_meta_inflight.get(key) is future:
_group_meta_inflight.pop(key, None)
@@ -252,12 +249,7 @@ async def resume_session(session_id: str):
@agents.router.post("/sessions/{session_id}/warm-cache")
async def warm_session_cache(session_id: str):
"""Fire a max_tokens=1 dummy request through the agent path so
Anthropic processes the system+tools prefix and writes the prompt
cache. The next real user turn lands a cache hit instead of paying
cold-start TTFT. Non-blocking, fire-and-forget on the frontend.
Returns 200 even on failure (best-effort).
"""
"""Fire a max_tokens=1 dummy request to prime the Anthropic prompt cache; best-effort."""
try:
await agent_manager.warm_prompt_cache(session_id)
except Exception:
@@ -265,10 +257,6 @@ async def warm_session_cache(session_id: str):
return {"ok": True}
# ---------------------------------------------------------------------------
# 9Router / Subscription endpoints
# ---------------------------------------------------------------------------
@agents.router.get("/subscriptions/status")
async def subscriptions_status():
"""Check if 9Router is running and list connected providers."""
@@ -277,8 +265,7 @@ async def subscriptions_status():
return {"running": False, "providers": [], "models": []}
connections = await get_providers()
models = await get_models()
# Frontend consumers (OnboardingModal, Settings) read
# `data.providers.connections` — preserve that envelope here.
# Frontend reads data.providers.connections; preserve the envelope.
return {"running": True, "providers": {"connections": connections}, "models": models}
@@ -295,11 +282,7 @@ async def subscriptions_connect(body: dict):
if not is_running():
raise HTTPException(status_code=503, detail="9Router not available. Please install Node.js.")
# If reconnecting a primary lane (e.g. gemini-cli), drop its cascade
# siblings first. The registry prefers antigravity over gemini-cli
# when both are present, so a stale antigravity token would keep
# 400ing even after gemini-cli refreshes. Wiping the sibling forces
# the registry onto the freshly reconnected lane.
# Reconnecting gemini-cli must wipe antigravity; registry prefers AG and a stale AG token would 400 after gemini-cli refreshes.
cascade = _PROVIDER_CASCADE_REMOVES.get(provider, [])
if cascade:
try:
@@ -310,7 +293,6 @@ async def subscriptions_connect(body: dict):
try:
result = await start_oauth(provider)
# For auth_code flows, store pending state so the callback can exchange
if result.get("flow") == "authorization_code" and result.get("state"):
from backend.main import _pending_oauth
_pending_oauth[result["state"]] = {
@@ -384,8 +366,7 @@ async def subscriptions_models():
@agents.router.post("/probe-model")
async def probe_model(body: dict):
"""1-token health probe. Returns {ok, latency_ms} or {ok:false, error}
or {ok:true, skipped:true} when the route's ambiguous (silent beats wrong)."""
"""1-token health probe; returns latency or skipped when the route is ambiguous (silent beats wrong)."""
import time as _time
short_name = (body or {}).get("model") or ""
if not short_name:
@@ -444,8 +425,7 @@ async def probe_model(body: dict):
except Exception as e:
msg = str(e).splitlines()[0] if str(e) else type(e).__name__
low = msg.lower()
# Suppress transients — chat will retry naturally and probe-time aliasing
# 404s often differ from how the chat path resolves the same id.
# Suppress transients: chat retries naturally and probe-time alias 404s often differ from chat resolution.
if any(s in low for s in (
"timeout", "timed out",
"connection reset", "connection aborted",
@@ -474,7 +454,7 @@ async def list_models():
try:
conns = await _9r_providers()
raw_providers = {c.get("provider", "") for c in conns if c.get("isActive") or c.get("testStatus") == "active"}
# 9Router uses "claude"; our models use api="anthropic" — map across.
# 9Router uses "claude"; our models use api="anthropic". Map across.
_9R_TO_API = {
"claude": "anthropic",
"codex": "codex",
@@ -486,8 +466,7 @@ async def list_models():
logger.debug(f"Failed to fetch 9Router providers: {e}")
def _serialize(models: list[dict]) -> list[dict]:
# Native models. Tiers describe the model itself; billing_kind
# describes the user's wallet for it. Pricing is shown only for paid.
# Tiers describe the model; billing_kind describes the wallet. Pricing shown only for paid.
from backend.apps.agents.providers.registry import (
COST_PER_1M_TOKENS,
compute_tiers,
@@ -518,7 +497,7 @@ async def list_models():
"reasoning": bool(m.get("reasoning", False)),
"input_cost_per_1m": input_cost,
"output_cost_per_1m": output_cost,
# Strict — subscription doesn't count. Pickerside uses Subscription chip.
# Strict free; subscriptions show via the picker's Subscription chip.
"is_free": billing_kind == "free",
"billing_kind": billing_kind,
"tiers": list(tiers),
@@ -539,8 +518,7 @@ async def list_models():
cc_variants = [m for m in anthropic_models if m.get("route") == "cc"]
api_variants = [m for m in anthropic_models if m.get("route") == "api"]
# Pro mode shows two groups (Pro proxy + Anthropic alternates via cc/api);
# own-key mode collapses to one Anthropic group using adaptive routing.
# Pro mode splits into Pro proxy + Anthropic alternates; own-key collapses to one adaptive group.
notes: list[dict] = []
if is_openswarm_pro:
result["OpenSwarm Pro"] = _serialize(adaptive)
@@ -605,8 +583,7 @@ async def list_models():
if visible:
result[provider_name] = visible
# OR catalog fetched straight from openrouter.ai (independent of 9Router
# boot state) so picker populates the moment a key lands.
# Fetch OpenRouter catalog directly (independent of 9Router) so picker fills the moment a key lands.
if has_openrouter_key:
try:
from backend.apps.agents.providers.registry import fetch_openrouter_models
@@ -654,10 +631,7 @@ async def list_models():
entries = sorted(by_vendor[vendor], key=lambda x: x["label"].lower())
result[f"OpenRouter · {pretty}"] = entries
# User-configured custom OpenAI-compatible providers (Ollama Cloud, Together, etc).
# Each provider becomes its own group in the picker; each model is addressed via
# the `custom/<slug>/<model_id>` value, which `_find_builtin_model` synthesises
# into a route='api' / api='custom' entry at request time.
# Custom OpenAI-compatible providers (Ollama Cloud, Together, etc); addressed via custom/<slug>/<model_id>.
from backend.apps.agents.providers.registry import _custom_provider_slug_for_lookup
for cp in (getattr(settings, "custom_providers", None) or []):
cp_name = (getattr(cp, "name", "") or "").strip()
@@ -692,27 +666,14 @@ async def list_models():
return {"models": result, "notes": notes}
# Google's two OAuth lanes (gemini-cli and antigravity) share user-facing
# meaning (both = "Google subscription") but 9Router treats them as
# separate connections with independent token lifecycles. The registry
# prefers `ag/` over `gc/` whenever AG is active because AG bypasses the
# thoughtSignature validator that breaks multi-step tool turns. That
# preference becomes a footgun when AG's token expires silently: the
# user reconnects "Google", only gemini-cli refreshes, and every request
# still routes through the stale AG token -> 400 Invalid argument.
#
# Cascade is one-directional. gemini-cli is the primary lane the UI
# exposes; operations on it sweep antigravity too. Direct operations on
# antigravity (e.g. an explicit AG opt-in/out path) MUST NOT cascade
# back to gemini-cli or we'd nuke the user's main Google connection.
# gemini-cli and antigravity are two Google OAuth lanes; registry prefers AG, so we cascade-wipe AG when reconnecting gemini-cli to avoid stale-AG 400s. One-directional: AG operations MUST NOT cascade back.
_PROVIDER_CASCADE_REMOVES: dict[str, list[str]] = {
"gemini-cli": ["antigravity"],
}
async def _delete_provider_connections(providers: list[str]) -> int:
"""Delete all 9Router connections whose provider is in the given list.
Returns the count actually removed. Silent if 9Router is unreachable."""
"""Delete 9Router connections in `providers`; returns count removed, silent on 9Router unreachable."""
import httpx
from backend.apps.nine_router import NINE_ROUTER_API, get_providers
try:
@@ -733,12 +694,7 @@ async def _delete_provider_connections(providers: list[str]) -> int:
@agents.router.post("/subscriptions/disconnect")
async def subscriptions_disconnect(body: dict):
"""Disconnect a subscription provider via 9Router.
For Google's paired lanes (gemini-cli + antigravity), wipe BOTH so a
subsequent reconnect lands on a clean slate instead of resurrecting
a stale sibling.
"""
"""Disconnect a subscription provider via 9Router; cascades-wipe Google's paired lanes."""
provider = body.get("provider", "")
if not provider:
raise HTTPException(status_code=400, detail="provider required")
backend/apps/agents/anthropic_proxy.py
+20 -81
View File
@@ -1,20 +1,4 @@
"""Lightweight Anthropic-format HTTP proxy.
When a user is on openswarm-pro with a non-Claude primary (GPT/Gemini/etc.),
the Claude Code CLI needs a single `ANTHROPIC_BASE_URL` that can serve BOTH:
1. the primary model calls (e.g. `cx/gpt-5` → must go to 9Router)
2. auxiliary Claude calls for subagents, WebSearch delegation
(e.g. `claude-haiku-4-5` → must go to OpenSwarm Pro's cloud proxy)
9Router doesn't know about OpenSwarm Pro, and we don't want to maintain a
custom 9Router provider-node for that. This proxy splits requests by the
`model` field in the body and forwards each to the correct upstream.
Mounted at `/api/anthropic-proxy`. Set `ANTHROPIC_BASE_URL` to
`http://127.0.0.1:<backend-port>/api/anthropic-proxy` in the CLI env for
Pro users with non-Claude primaries.
"""
"""Anthropic-format HTTP proxy splitting requests by model field; primary to 9Router, aux Claude to Pro proxy."""
import json
import logging
@@ -48,42 +32,27 @@ _CLAUDE_MODEL_PREFIXES = (
_GEMINI_MODEL_PREFIXES = ("gemini/", "gc/", "ag/")
# Bare-model patterns that resolve to Gemini's native API (gemini-3-flash-api,
# gemini-3.1-pro-api, gemini-3.1-flash-lite-api, etc. — when user supplies own
# Google API key in Settings → Models). These bypass our `gemini/` prefix so
# the prefix-only check above misses them; we match on the bare-name shape
# here too so $schema scrubbing fires for own-key Gemini sessions.
# Pre-fix: 8/8 own-key Gemini sessions in production failed with 400 because
# JSON Schema's $schema field leaked into Google's tools[].function_declarations
# payload. (See raw_payloads where status=error on every gemini-*-api session.)
# Own-key Gemini ("gemini-3-flash-api" etc.) skips the gemini/ prefix; match bare names so $schema scrub still fires.
_GEMINI_BARE_MODEL_PATTERNS = ("gemini-",)
# Fields Gemini's function_declarations validator rejects. 9Router 0.3.60's
# translator strips allOf/anyOf/oneOf/const-toplevel/required but misses
# these. Each one we've seen Gemini 400 on in production with "Unknown
# name 'X' at request.tools[N].function_declarations[N].parameters.…"
# Keys 9Router 0.3.60 misses that Gemini's function_declarations validator 400s on. Each was caught in prod.
_GEMINI_FORBIDDEN_SCHEMA_KEYS = {
# JSON-Schema metadata fields Gemini's stricter validator doesn't accept.
"$schema",
"$id", # ag/gemini-3.1-pro-high session, 2026-05-08
"$ref", # JSON-Schema reference; Gemini wants inlined types
"$defs", # ditto
"definitions", # legacy alias for $defs
# Constraint fields Gemini doesn't implement.
"$id",
"$ref",
"$defs",
"definitions",
"additionalProperties",
"propertyNames",
"patternProperties",
"exclusiveMinimum",
"exclusiveMaximum",
"const", # nested const leaks through 9Router's top-level-only strip.
# Anthropic-specific tool-call hints not part of vanilla JSON Schema.
# Anthropic's CLI emits these on tools that benefit from response
# priming; Gemini's validator rejects all unknown keys.
"prefill", # ag/gemini-3.1-pro-high session, 2026-05-08
"enumTitles", # human-readable enum labels; OpenAI-only convention
"title", # safe to keep usually but Gemini sometimes rejects under nested arrays
"examples", # JSON-Schema 2019-09 keyword Gemini doesn't honor
"default", # often allowed but rejected in nested array.items
"const",
"prefill",
"enumTitles",
"title",
"examples",
"default",
"readOnly",
"writeOnly",
"deprecated",
@@ -106,28 +75,15 @@ def _scrub_gemini_schema(node):
return node
# Models that REQUIRE max_completion_tokens instead of max_tokens.
# OpenAI's GPT-5.x family (gpt-5.4, gpt-5.4-mini, gpt-5.5, gpt-5.3-codex,
# etc.) introduced this in late 2025 — the legacy `max_tokens` field returns
# a 400 "Unsupported parameter: 'max_tokens' is not supported with this
# model. Use 'max_completion_tokens' instead." Anthropic's CLI / SDK still
# emits `max_tokens` because that's the Anthropic-format wire shape; we
# rename it on the way out for OpenAI-routed GPT-5 models.
# GPT-5.x rejects max_tokens; needs max_completion_tokens. Anthropic-format wire still emits max_tokens; we rename on the way out.
_OPENAI_MAX_COMPLETION_TOKENS_MODELS = ("gpt-5",)
def _is_openai_max_completion_tokens_model(model: str) -> bool:
"""Match every shape a GPT-5 model name might arrive in. Includes:
- bare: "gpt-5", "gpt-5.5", "gpt-5.4-mini"
- api-suffixed: "gpt-5.5-api" (desktop's pinned-api naming)
- 9router-prefixed: "openai/gpt-5.5" (post-translation name)
- codex-routed: "cx/gpt-5.3-codex" (CLI subscription)
Anything WITHOUT "gpt-5" in the (lowercased) string is rejected.
"""
"""Match every shape a GPT-5 name might arrive in (bare, api-suffixed, openai/-prefixed, cx/-routed)."""
m = (model or "").strip().lower()
if not m:
return False
# Strip common routing prefixes so we can match the bare model body.
for prefix in ("openai/", "cx/", "openrouter/", "or:openai/", "cp/", "cp-"):
if m.startswith(prefix):
m = m[len(prefix):]
@@ -136,12 +92,7 @@ def _is_openai_max_completion_tokens_model(model: str) -> bool:
def _scrub_request_for_openai_gpt5(body: bytes) -> bytes:
"""Rename `max_tokens` → `max_completion_tokens` for GPT-5 models.
Bytes-in/out, never raises. No-op if the body isn't JSON or doesn't
contain `max_tokens`. Drops the legacy field if BOTH are present so
the API doesn't reject for "both fields specified".
"""
"""Rename max_tokens to max_completion_tokens for GPT-5; bytes in/out, never raises."""
if not body:
return body
try:
@@ -179,8 +130,7 @@ def _scrub_request_for_gemini(body: bytes) -> bytes:
return json.dumps(parsed).encode("utf-8")
# Headers we strip before forwarding — these change hop-by-hop or we
# replace them with upstream-specific auth.
# Hop-by-hop headers or auth we replace with the upstream-specific value.
_HOP_HEADERS = {
"host",
"content-length",
@@ -206,9 +156,7 @@ def _is_gemini_model(model: str) -> bool:
m = (model or "").strip().lower()
if m.startswith(_GEMINI_MODEL_PREFIXES):
return True
# Bare-name match: "gemini-3-flash-api", "gemini-3.1-pro-api", etc.
# Excludes anthropic-routed gemini models (those carry "/" or other
# routing prefixes via the registry).
# Bare-name match for own-key Gemini; excludes anthropic-routed gemini (those carry "/").
if "/" in m:
return False
return any(m.startswith(p) for p in _GEMINI_BARE_MODEL_PATTERNS)
@@ -220,15 +168,12 @@ def _pick_upstream(model: str) -> tuple[str, dict[str, str]]:
s = load_settings()
if _is_claude_model(model):
# Prefer Pro cloud proxy when configured.
if getattr(s, "connection_mode", "own_key") == "openswarm-pro":
bearer = getattr(s, "openswarm_bearer_token", "") or ""
proxy = (getattr(s, "openswarm_proxy_url", "") or "https://api.openswarm.com").rstrip("/")
if bearer and proxy:
return (proxy, {"Authorization": f"Bearer {bearer}"})
# Fall through — let 9Router handle it (maybe user has a real Claude sub).
# Default: 9Router for everything else (cx/, gc/, gh/, apikey-routed models).
return ("http://127.0.0.1:20128", {"x-api-key": "9router"})
@@ -243,7 +188,7 @@ def _pick_upstream(model: str) -> tuple[str, dict[str, str]]:
include_in_schema=False,
)
async def _healthcheck():
"""CLI healthchecks the proxy root — return 200 so it doesn't 404."""
"""CLI healthchecks the proxy root; return 200 so it doesn't 404."""
return {"ok": True}
@@ -272,13 +217,7 @@ async def proxy(rest: str, request: Request):
for k, v in request.headers.items():
if k.lower() in _HOP_HEADERS:
continue
# The CLI we spawn carries our per-install auth token via
# `x-api-key` (we set `ANTHROPIC_API_KEY=<our_token>` on the
# spawn env, and the CLI forwards that value as x-api-key). We
# must NOT forward that header to the real upstream — it would
# leak our local token to api.openswarm.com / 9Router, AND it
# would shadow the real upstream auth (bearer or `9router`
# literal) that `_pick_upstream` wants to set. Strip it here.
# CLI carries our install token as x-api-key; never forward (leak + shadows real upstream auth).
if k.lower() == "x-api-key":
continue
forward_headers[k] = v
backend/apps/agents/browser_agent.py
+43 -43
View File
@@ -73,7 +73,7 @@ def _hash_tool_call(tool_name: str, tool_input: dict, result: dict) -> tuple[str
"""Build a stable hash key for a tool call, including its result.
Including the result hash means that legitimate progress (same input,
different output — e.g. BrowserScroll on a long feed) does NOT count
different output; e.g. BrowserScroll on a long feed) does NOT count
as a loop. Only same-input + same-output is treated as stuck.
"""
try:
@@ -108,7 +108,7 @@ def _detect_loop(
_LOOP_WARNING_TEXT = (
"LOOP DETECTED: You have called this tool with these exact parameters and "
"gotten the same result {count} times in a row. STOP retrying this approach "
"— it is not working. Try a fundamentally different strategy: "
", it is not working. Try a fundamentally different strategy: "
"(1) check the page state with BrowserScreenshot or BrowserGetText, "
"(2) try a different selector or a different tool, "
"(3) use BrowserPressKey for keyboard shortcuts if the site supports them, "
@@ -121,7 +121,7 @@ def _validate_message_pairing(messages: list[dict]) -> bool:
message in the same list. Returns False if there's an orphan, which means
the cached history would 400 if sent to the API.
This is the last line of defense against cache corruption — if it ever
This is the last line of defense against cache corruption; if it ever
returns False on a resume, we drop the cache and start fresh rather than
crash on the next API call.
"""
@@ -145,7 +145,7 @@ def _validate_message_pairing(messages: list[dict]) -> bool:
def _is_fresh_user_message(msg: dict) -> bool:
"""A 'fresh' user message starts a new turn — string content or a list
"""A 'fresh' user message starts a new turn; string content or a list
that contains no tool_result blocks. These are the only safe cut points
because they don't reference any prior assistant tool_use blocks."""
if msg.get("role") != "user":
@@ -165,7 +165,7 @@ def _summarize_messages(messages: list[dict]) -> str:
Extracts the original user task, a count of tool calls by name with their
key parameters, the last few ReportProgress brain states, and the most
recent assistant text. No LLM call required — this is purely structural
recent assistant text. No LLM call required; this is purely structural
extraction from the existing message history.
"""
if not messages:
@@ -256,7 +256,7 @@ def _trim_history_by_turns(messages: list[dict], max_messages: int) -> list[dict
function avoids that by:
1. Walking forward to find a clean turn boundary (a fresh user-text
message that starts a new turn — no tool_result content).
message that starts a new turn; no tool_result content).
2. Summarizing everything BEFORE that boundary into a single user-text
message and prepending it to the kept tail.
3. If no clean boundary exists at all, returning the original history
@@ -285,7 +285,7 @@ def _trim_history_by_turns(messages: list[dict], max_messages: int) -> list[dict
# Second pass: if no cut point gets us under the cap (e.g. the current
# turn alone is bigger than max_messages), use the LATEST clean cut point
# available. The tail will still exceed the cap, but it's the smallest
# safe history we can produce — and any compaction is better than none.
# safe history we can produce; and any compaction is better than none.
if cut_index is None:
for i in range(len(messages) - 1, 0, -1):
if _is_fresh_user_message(messages[i]):
@@ -293,7 +293,7 @@ def _trim_history_by_turns(messages: list[dict], max_messages: int) -> list[dict
break
if cut_index is None:
# No clean cut anywhere in the history. Return original — better to
# No clean cut anywhere in the history. Return original; better to
# exceed the cap than to corrupt the conversation.
return list(messages)
@@ -326,7 +326,7 @@ BROWSER_TOOLS_SCHEMA = [
"working_memory": {
"type": "string",
"description": (
"Short notes about what you've learned about this site so far — "
"Short notes about what you've learned about this site so far; "
"selectors that work, keyboard shortcuts, layout quirks, what "
"you've tried that failed. Carry this forward across turns."
),
@@ -459,7 +459,7 @@ BROWSER_TOOLS_SCHEMA = [
"[2]<link \"Settings\">, etc. Use this BEFORE BrowserClickIndex. This is "
"the PREFERRED way to discover clickable elements on hostile sites "
"(Tinder, Instagram, TikTok) where CSS selectors fail because the page "
"uses unlabeled <div>s — the accessibility tree sees roles and names "
"uses unlabeled <div>s; the accessibility tree sees roles and names "
"even when raw HTML doesn't expose them. Much more reliable than "
"BrowserGetElements (which uses CSS selectors)."
),
@@ -476,7 +476,7 @@ BROWSER_TOOLS_SCHEMA = [
"Uses native OS-level mouse events (event.isTrusted=true) so it works "
"on sites that filter out synthetic JS events. Always call "
"BrowserListInteractives first to get a fresh index list. If the click "
"returns 'index no longer valid', the page changed — re-list and retry."
"returns 'index no longer valid', the page changed; re-list and retry."
),
"input_schema": {
"type": "object",
@@ -496,7 +496,7 @@ BROWSER_TOOLS_SCHEMA = [
"is executed in order, with the URL captured before/after each one. "
"If the URL changes mid-batch (the page navigated), the rest of the "
"batch is aborted and you get a partial result. Use this when you "
"have a known sequence — typing then pressing Enter, swiping multiple "
"have a known sequence; typing then pressing Enter, swiping multiple "
"times, clicking through pagination. Max 5 actions per batch.\n\n"
"Sub-action types and their params:\n"
"- click_index: { index: int }\n"
@@ -537,7 +537,7 @@ BROWSER_TOOLS_SCHEMA = [
"description": (
"Press a keyboard key (or key combination) on the page using a real native "
"input event. Use this for keyboard shortcuts when JS-dispatched events get "
"ignored — sites like Tinder, Slack, Notion, Gmail listen for trusted key "
"ignored; sites like Tinder, Slack, Notion, Gmail listen for trusted key "
"events. Examples: 'ArrowLeft', 'ArrowRight', 'Enter', 'Escape', 'Tab', "
"'Space', single letters like 'a'. Prefer this over BrowserEvaluate with "
"dispatchEvent for keyboard shortcuts."
@@ -579,7 +579,7 @@ BROWSER_TOOLS_SCHEMA = [
"name": "RequestHumanIntervention",
"description": (
"Request the user's help when you encounter an obstacle you cannot solve "
"programmatically — captchas, login prompts, cookie consent walls, "
"programmatically; captchas, login prompts, cookie consent walls, "
"two-factor authentication, or any blocking popup. The agent will pause "
"until the user resolves the issue and clicks Continue."
),
@@ -590,7 +590,7 @@ BROWSER_TOOLS_SCHEMA = [
"type": "string",
"description": (
"One short sentence describing the obstacle. Keep it under "
"15 words. Example: 'Login required — please sign in to X/Twitter.'"
"15 words. Example: 'Login required; please sign in to X/Twitter.'"
),
},
"instruction": {
@@ -624,7 +624,7 @@ ACTION_MAP = {
SYSTEM_PROMPT = (
"You are a website-agnostic browser automation agent. You can operate on ANY "
"website the user is signed into — social media, dating apps, email, productivity "
"website the user is signed into; social media, dating apps, email, productivity "
"tools, dashboards, ecommerce, anything. Assume the user has already logged in.\n\n"
"## Required output structure: ReportProgress before every action\n"
@@ -654,36 +654,36 @@ SYSTEM_PROMPT = (
"If this is a continuation of an earlier conversation on the same browser, the "
"messages above already contain everything you've tried, what worked, what failed, "
"and the page state. READ THAT HISTORY before acting. Do NOT take a fresh screenshot "
"or re-explore the DOM if you already know what's on screen — just act. Only re-orient "
"or re-explore the DOM if you already know what's on screen; just act. Only re-orient "
"if the page has clearly changed (after navigation, after a multi-second wait, or if "
"your last action mutated the page in unexpected ways).\n\n"
"## Try multiple strategies, learn from failures\n"
"Sites vary wildly. When one approach fails, switch tactics — don't retry the same "
"Sites vary wildly. When one approach fails, switch tactics; don't retry the same "
"thing. The escalation ladder, fastest to slowest:\n"
"1. **Keyboard shortcuts via BrowserPressKey** — fastest and most reliable on sites "
"1. **Keyboard shortcuts via BrowserPressKey**; fastest and most reliable on sites "
"that support them (Tinder swipes, Gmail navigation, Slack message jump, etc.). "
"Always check if the site shows keyboard hints in the UI before falling back to clicks. "
"BrowserPressKey sends real native events that pass the `event.isTrusted` check, so "
"it works where dispatchEvent in BrowserEvaluate silently fails.\n"
"2. **Accessibility tree via BrowserListInteractives + BrowserClickIndex** — the "
"2. **Accessibility tree via BrowserListInteractives + BrowserClickIndex**; the "
"accessibility tree sees roles and names that the raw DOM doesn't, even on sites "
"like Tinder, Instagram, and TikTok that use unlabeled <div>s with click handlers. "
"Call BrowserListInteractives to get a numbered list (`[1]<button \"Like\">`, "
"`[2]<link \"Settings\">`), then BrowserClickIndex with the number. The click uses "
"native OS-level mouse events so it works where DOM .click() doesn't. THIS IS YOUR "
"GO-TO STRATEGY for unlabeled or hostile sites — try this BEFORE BrowserGetElements.\n"
"3. **Semantic CSS selectors** — `button[aria-label='X']`, `[role='button']`, "
"GO-TO STRATEGY for unlabeled or hostile sites; try this BEFORE BrowserGetElements.\n"
"3. **Semantic CSS selectors**; `button[aria-label='X']`, `[role='button']`, "
"`a[href*='...']`. Try these via BrowserGetElements + BrowserClick when the site "
"actually has semantic HTML.\n"
"4. **Text-based JS query** — when both of the above fail, use BrowserEvaluate to "
"4. **Text-based JS query**; when both of the above fail, use BrowserEvaluate to "
"find elements by visible text: `Array.from(document.querySelectorAll('*')).find(el => el.textContent.trim() === 'Like')`.\n"
"5. **Coordinate-based fallback** — last resort: take a screenshot, identify the "
"5. **Coordinate-based fallback**; last resort: take a screenshot, identify the "
"button visually, then click by approximate coords.\n\n"
"## Batch known sequences with BrowserBatch\n"
"When you have a known sequence of actions — typing then pressing Enter, "
"swiping multiple times, clicking through pagination — emit them all in a "
"When you have a known sequence of actions; typing then pressing Enter, "
"swiping multiple times, clicking through pagination; emit them all in a "
"single BrowserBatch call instead of one tool per turn. The batch executes "
"sub-actions sequentially and aborts if the URL changes mid-batch (so you "
"won't operate on stale state). Max 5 sub-actions per batch.\n"
@@ -703,21 +703,21 @@ SYSTEM_PROMPT = (
"- Do NOT call the same failing tool twice with identical parameters. If selector "
"X failed, try a DIFFERENT selector or a DIFFERENT strategy.\n"
"- For repeated actions (swiping through profiles, going through inbox messages), "
"use BrowserPressKey if available — it's an order of magnitude faster than DOM clicks.\n\n"
"use BrowserPressKey if available; it's an order of magnitude faster than DOM clicks.\n\n"
"## When you genuinely cannot proceed\n"
"Use RequestHumanIntervention for:\n"
"- Login walls (the user thinks they're logged in but the session expired)\n"
"- Captchas, 2FA prompts, age verification gates\n"
"- Anything genuinely ambiguous about user intent\n"
"Don't use it for normal tool failures — try a different approach first.\n\n"
"Don't use it for normal tool failures; try a different approach first.\n\n"
"## Tool reference\n"
"- BrowserScreenshot: visual snapshot. Use sparingly, not after every action.\n"
"- BrowserGetText: returns up to 15000 chars of visible text. Useful for reading "
"content without an image.\n"
"- BrowserScroll: handles nested scroll containers (Notion, Gmail). Returns "
"atTop/atBottom — stop looping when scroll delta is 0.\n"
"atTop/atBottom; stop looping when scroll delta is 0.\n"
"- BrowserGetElements: enumerate interactive elements with selectors.\n"
"- BrowserClick / BrowserType: standard DOM interaction.\n"
"- BrowserPressKey: native key events (preferred for shortcuts).\n"
@@ -730,7 +730,7 @@ SYSTEM_PROMPT = (
MAX_TURNS = 40
# Tools that count as "action tools" — calling any of these in a turn requires
# Tools that count as "action tools"; calling any of these in a turn requires
# the model to also call ReportProgress in the same turn (after the first
# turn). Read-only tools and meta tools are exempt.
_ACTION_TOOLS_REQUIRING_REPORT = {
@@ -906,17 +906,17 @@ async def run_browser_agent(
# When the parent session is running on a non-Claude model (e.g. gpt-5.4),
# the browser agent inherits it and we route through 9Router's prefix.
# Tool-use fidelity for browser-specific tools (BrowserNavigate, click,
# type, etc.) through 9Router's claude→openai translator is UNVERIFIED —
# type, etc.) through 9Router's claude→openai translator is UNVERIFIED ,
# if translation is poor, the user should manually switch this session
# back to Claude in the model picker.
if _find_builtin_model(model) is not None:
api_model = resolve_model_id_for_sdk(model, browser_settings)
else:
# Unknown model string — fall back to whatever aux model is available
# Unknown model string; fall back to whatever aux model is available
try:
api_model, _ = await resolve_aux_model(browser_settings, preferred_tier="haiku")
except ValueError:
# Nothing connected at all — surface a clear error so the caller
# Nothing connected at all; surface a clear error so the caller
# (parent agent) sees it in the tool result instead of crashing
# on a 400 from 9Router.
session.status = "error"
@@ -953,13 +953,13 @@ async def run_browser_agent(
# Resume prior conversation on this browser if we have one cached. This
# lets the sub-agent skip the "take a screenshot to figure out where I am"
# cycle every time the parent issues a new task. Defensively validate
# the cache — if it's somehow corrupted (orphaned tool_use_ids), drop
# the cache; if it's somehow corrupted (orphaned tool_use_ids), drop
# it and start fresh rather than crash on the next API call.
prior_messages = _browser_history.get(browser_id) or []
if prior_messages and not _validate_message_pairing(prior_messages):
logger.warning(
f"[browser-agent {session_id}] cached history for {browser_id} has "
f"orphaned tool_use_ids — dropping cache and starting fresh"
f"orphaned tool_use_ids; dropping cache and starting fresh"
)
_browser_history.pop(browser_id, None)
prior_messages = []
@@ -967,7 +967,7 @@ async def run_browser_agent(
action_log: list[dict] = []
final_screenshot: str | None = None
# Loop detection state — sliding window of recent state-mutating tool calls
# Loop detection state; sliding window of recent state-mutating tool calls
recent_tool_calls: list[tuple[str, str, str]] = []
loop_trigger_count = 0
@@ -1094,7 +1094,7 @@ async def run_browser_agent(
logger.error(
f"[browser-agent {session_id}] hit "
f"{MAX_CONSECUTIVE_VIOLATIONS} consecutive ReportProgress "
f"violations — aborting to prevent runaway loop"
f"violations; aborting to prevent runaway loop"
)
# Surface a user-visible error message so the frontend
# shows something coherent instead of just stopping.
@@ -1113,7 +1113,7 @@ async def run_browser_agent(
})
break
else:
# Reset on a clean turn — only CONSECUTIVE violations
# Reset on a clean turn; only CONSECUTIVE violations
# count toward the limit. A single bad turn followed by
# a good one shouldn't kill the agent.
consecutive_violations = 0
@@ -1128,7 +1128,7 @@ async def run_browser_agent(
cancelled = True
break
# Handle ReportProgress — no-op execution that just records the
# Handle ReportProgress; no-op execution that just records the
# model's brain state and streams it to the dashboard.
if tu.name == "ReportProgress":
eval_prev = tu.input.get("evaluation_previous", "")
@@ -1163,7 +1163,7 @@ async def run_browser_agent(
rejection_text = (
"REJECTED: You called an action tool without first calling "
"ReportProgress in the same turn. ReportProgress is REQUIRED "
"before every batch of action tools — it's how you reflect "
"before every batch of action tools; it's how you reflect "
"on what just happened and articulate your next goal. Try "
"again: emit ReportProgress and your action tool(s) in the "
"same response."
@@ -1189,7 +1189,7 @@ async def run_browser_agent(
})
continue
# Handle RequestHumanIntervention — pause and wait for user
# Handle RequestHumanIntervention; pause and wait for user
if tu.name == "RequestHumanIntervention":
problem = tu.input.get("problem", "")
instruction = tu.input.get("instruction", "")
@@ -1341,7 +1341,7 @@ async def run_browser_agent(
if loop_trigger_count >= _LOOP_HARD_CAP:
logger.warning(
f"[browser-agent {session_id}] hit loop hard cap "
f"({_LOOP_HARD_CAP}) — force-exiting"
f"({_LOOP_HARD_CAP}); force-exiting"
)
break
@@ -1376,7 +1376,7 @@ async def run_browser_agent(
# Persist conversation history so the next BrowserAgent call on this
# browser can resume rather than re-orient. Trim to the most recent
# _MAX_HISTORY_MESSAGES turns to keep token usage bounded — but
# _MAX_HISTORY_MESSAGES turns to keep token usage bounded; but
# never split a tool_use ↔ tool_result pair across the cut, or the
# next API request will 400.
_browser_history[browser_id] = _trim_history_by_turns(
backend/apps/agents/browser_agent_mcp_server.py
+1 -6
View File
@@ -1,10 +1,5 @@
#!/usr/bin/env python3
"""
Stdio MCP server that exposes BrowserAgent and BrowserAgents delegation tools.
Launched as a subprocess by the Claude Agent SDK. Proxies task delegation
to the OpenSwarm backend via HTTP, which runs browser sub-agents.
"""
"""Stdio MCP server exposing BrowserAgent/BrowserAgents delegation tools."""
import base64
import json
backend/apps/agents/invoke_agent_mcp_server.py
+2 -8
View File
@@ -1,11 +1,5 @@
#!/usr/bin/env python3
"""
Stdio MCP server that exposes the InvokeAgent tool.
Launched as a subprocess by the Claude Agent SDK. Proxies invocation
requests to the OpenSwarm backend via HTTP, which forks the target
agent session and runs it with the new message.
"""
"""Stdio MCP server exposing the InvokeAgent tool; proxies to /api/invoke-agent/run."""
import json
import sys
@@ -113,7 +107,7 @@ def handle_tool_call(tool_name: str, arguments: dict) -> dict:
lines = [f"**Invoked Agent Result** (forked session: {forked_id})"]
if source_name:
lines[0] = f"**Invoked Agent Result** — {source_name} (forked session: {forked_id})"
lines[0] = f"**Invoked Agent Result**; {source_name} (forked session: {forked_id})"
if cost > 0:
lines.append(f"*Cost: ${cost:.4f}*")
lines.append("")
backend/apps/agents/mcp_meta_server.py
+9 -25
View File
@@ -1,23 +1,5 @@
#!/usr/bin/env python3
"""Stdio MCP server exposing the MCP activation gate.
Tools:
- MCPList: enumerate installed MCP servers (active + available).
- MCPSearch(query): rank servers by relevance to a free-form query.
- MCPActivate(server_name): activate a server for the rest of the session.
The activation gate is the dispatch-layer enforcement of the product invariant
"all MCP actions only via ToolSearch": the model can only reach an MCP server's
tools if the user has approved MCPActivate for that server, which appends to
session.active_mcps. _build_mcp_servers in agent_manager.py intersects connected
MCPs with that list before handing them to the SDK, so unactivated servers are
literally unreachable — the gate cannot be bypassed by ignoring prompt rules.
HITL: the model's invocation of MCPActivate goes through agent_manager's pre-
tool approval hook just like any other tool call — the user is prompted to
approve activation in the standard ApprovalBar UI. No separate HITL inside this
server.
"""
"""Stdio MCP server exposing the MCP activation gate (MCPList/MCPSearch/MCPActivate)."""
import json
import os
@@ -69,7 +51,7 @@ TOOLS = [
"description": (
"Request activation of an MCP server for this session. Triggers a "
"user approval prompt; on approve the server's tools become callable "
"next turn. Always confirm the server name via MCPList/MCPSearch first — "
"next turn. Always confirm the server name via MCPList/MCPSearch first; "
"invalid names return alternatives instead of activating."
),
"inputSchema": {
@@ -81,7 +63,7 @@ TOOLS = [
},
"reason": {
"type": "string",
"description": "Why you need it — shown to the user in the approval prompt.",
"description": "Why you need it; shown to the user in the approval prompt.",
},
},
"required": ["server_name"],
@@ -133,7 +115,7 @@ def format_servers(servers: list[dict], heading: str = "") -> str:
name = s.get("name", "")
desc = s.get("description") or f"{name} integration"
status = s.get("status", "available")
lines.append(f"- `{name}` [{status}] — {desc}")
lines.append(f"- `{name}` [{status}]; {desc}")
return "\n".join(lines)
@@ -189,15 +171,17 @@ def handle_tool_call(tool_name: str, arguments: dict) -> dict:
"isError": True,
}
if result.get("status") == "already_active":
return {"content": [{"type": "text", "text": f"`{server_name}` is already active for this session — its tools should be callable now."}]}
return {"content": [{"type": "text", "text": f"`{server_name}` is already active for this session; its tools should be callable now."}]}
if result.get("status") == "activated":
return {
"content": [{
"type": "text",
"text": (
f"Activated `{server_name}`. Its tools (`mcp__{server_name}__*`) "
f"will be callable on the NEXT turn. End this turn now and the user's "
f"next message will see the new tools."
f"are NOT callable in this turn; the transport snapshot is "
f"already locked. This turn will end automatically and a "
f"hidden continuation turn will fire with the new tools "
f"loaded. Do not attempt any other tool call now."
),
}],
}
backend/apps/agents/mcp_preflight.py
+21 -86
View File
@@ -1,20 +1,4 @@
"""Pre-flight MCP suggestion classifier.
Runs before a new agent launches. Given the user's initial prompt, decides:
1. Is this prompt vague or information-gathering (is_vague) — used to
conditionally inject the discovery scaffolding into the system prompt.
2. Does it suggest a not-yet-connected MCP that would dramatically
improve the outcome — surfaced to the user as a one-click
"Connect X" modal before the agent runs.
Only the curated shortlist of MCPs we ship and have vetted is considered.
The full community MCP registry is NOT mined for suggestions — flaky/
unvetted entries would make the "magic" moment feel broken.
Provider-agnostic: calls whatever cheap-tier aux model the user has wired
via `resolve_aux_model` (Haiku / GPT-5.4-mini / Gemini-2.5-flash / etc.).
If no provider is connected, fails open (no suggestions, no scaffolding).
"""
"""Pre-flight classifier; decides is_vague (scaffolding inject) + suggests an MCP to connect. Fails open."""
from __future__ import annotations
@@ -32,56 +16,45 @@ from backend.apps.tools_lib.tools_lib import _load_all as load_all_tools
logger = logging.getLogger(__name__)
# ---------------------------------------------------------------------------
# Tool-agnostic discovery scaffolding — appended to the agent's system prompt
# only when preflight flags the prompt as vague/information-gathering.
# ---------------------------------------------------------------------------
# Tool-agnostic discovery scaffolding; appended only when preflight flags the prompt as vague/info-gathering.
DISCOVERY_SCAFFOLDING = (
"# Discovery before action\n"
"When a request is vague or could be grounded in user context, do not "
"guess generic defaults. First silently enumerate what would change the "
"output — voice, tone, audience, prior context, recent precedent, facts "
"output; voice, tone, audience, prior context, recent precedent, facts "
"that only live in the user's data. Then look at your available tools and "
"pick the ones that could answer those unknowns. Read a few examples "
"(usually 3–10 is enough), summarize what you found into a few bullets, "
"(usually 3, 10 is enough), summarize what you found into a few bullets, "
"then act confidently.\n\n"
"Tool-selection hierarchy for information gathering:\n"
" 1. Direct local access (filesystem reads, code search, shell) — "
" 1. Direct local access (filesystem reads, code search, shell); "
"cheapest and fastest.\n"
" 2. Connected services / MCP tools — for user data that lives in a "
" 2. Connected services / MCP tools; for user data that lives in a "
"linked account (email, calendar, notes, tickets, etc.).\n"
" 3. Web search / fetch — for public information that isn't in your "
" 3. Web search / fetch; for public information that isn't in your "
"training cutoff.\n"
" 4. Browser automation — only when a real interactive session or "
" 4. Browser automation; only when a real interactive session or "
"login is required.\n"
" 5. Sub-agents — only for parallelizable subtasks or to isolate heavy "
" 5. Sub-agents; only for parallelizable subtasks or to isolate heavy "
"context. Not for serial steps.\n\n"
"Asking the user is a fallback, not a first move. Never fabricate. If "
"no tool can ground a critical unknown, ask one concise question."
)
# ---------------------------------------------------------------------------
# Curated MCP shortlist. These `id` values MUST match the exact `name` field
# on ToolDefinition entries that OpenSwarm ships as defaults (see
# `backend/data/tools/*.json` — one file per tool, `name` is the canonical
# key used everywhere else in the app). Mismatches would cause the
# enabled/disabled filter to no-op and the frontend modal to render nothing.
#
# Keep in sync with the Custom Action Sets list in Settings → Tools.
# ---------------------------------------------------------------------------
# Curated shortlist; `id` MUST match ToolDefinition.name exactly or the enabled/dismissed filter no-ops and the modal renders nothing.
CuratedEntry = dict[str, Any]
CURATED_SHORTLIST: list[CuratedEntry] = [
{
"id": "Google Workspace",
"title": "Google Workspace",
"description": "Gmail, Calendar, Drive, Docs, Sheets, Slides — for reading/sending email, checking the user's schedule, and pulling context from their documents.",
"description": "Gmail, Calendar, Drive, Docs, Sheets, Slides; for reading/sending email, checking the user's schedule, and pulling context from their documents.",
},
{
"id": "Microsoft 365",
"title": "Microsoft 365",
"description": "Outlook email, Calendar, OneDrive, Teams, Excel, OneNote — Microsoft-stack equivalent of Google Workspace.",
"description": "Outlook email, Calendar, OneDrive, Teams, Excel, OneNote; Microsoft-stack equivalent of Google Workspace.",
},
{
"id": "Slack",
@@ -101,7 +74,7 @@ CURATED_SHORTLIST: list[CuratedEntry] = [
{
"id": "HubSpot",
"title": "HubSpot",
"description": "CRM contacts, deals, companies, tickets — when the user's task involves their customer relationships.",
"description": "CRM contacts, deals, companies, tickets; when the user's task involves their customer relationships.",
},
{
"id": "Airtable",
@@ -111,12 +84,12 @@ CURATED_SHORTLIST: list[CuratedEntry] = [
{
"id": "Reddit",
"title": "Reddit",
"description": "Browse subreddits, search posts, analyze users — when the task involves public Reddit content.",
"description": "Browse subreddits, search posts, analyze users; when the task involves public Reddit content.",
},
{
"id": "YouTube",
"title": "YouTube",
"description": "Video transcripts, details, comments, channel stats, search — when the task involves YouTube content.",
"description": "Video transcripts, details, comments, channel stats, search; when the task involves YouTube content.",
},
{
"id": "Instagram",
@@ -126,48 +99,25 @@ CURATED_SHORTLIST: list[CuratedEntry] = [
]
# ---------------------------------------------------------------------------
# Local skip filter — short-circuits the LLM call for obviously-local prompts
# where no MCP could add value. Saves ~200ms + ~$0.0001 per launch.
# ---------------------------------------------------------------------------
# Short-circuit for obviously-local prompts where no MCP helps. Saves ~200ms + ~$0.0001 per launch.
_PATH_LIKE = re.compile(r"^[./~]|/[\w\-]+/|\.[a-zA-Z]{1,5}\b")
_SHELL_PREFIX = re.compile(r"^\s*[\$!/]")
def _is_obviously_local(prompt: str) -> bool:
"""Heuristic: does this prompt obviously not need any MCP?
Returns True for:
- very short prompts (< 8 chars, likely greetings or acknowledgments)
- shell-command-ish prompts ("! ls", "$ git status", "/clear")
- prompts that are essentially a single file path reference
On True we skip preflight entirely; the agent launches with no
scaffolding and no suggestion modal.
"""
"""True for prompts that obviously can't benefit from MCP (very short, shell-ish, single path)."""
s = prompt.strip()
if len(s) < 8:
return True
if _SHELL_PREFIX.match(s):
return True
# Single-token path-ish prompt (e.g. "./src/foo.ts")
if " " not in s and _PATH_LIKE.search(s):
return True
return False
# ---------------------------------------------------------------------------
# Main entry point
# ---------------------------------------------------------------------------
async def run_preflight(prompt: str, timeout_s: float = 2.0) -> dict:
"""Classify the user's prompt and return suggestions + vagueness flag.
Always returns a dict of shape:
{"is_vague": bool, "suggestions": [Suggestion, ...]}
Never raises: any failure (no provider, aux model timeout, bad JSON)
fails open — returns is_vague=False and empty suggestions.
"""
"""Classify the prompt and return {is_vague, suggestions}; never raises."""
default: dict[str, Any] = {"is_vague": False, "suggestions": []}
if not prompt or not prompt.strip():
@@ -179,33 +129,20 @@ async def run_preflight(prompt: str, timeout_s: float = 2.0) -> dict:
try:
settings = load_settings()
available = _build_available_shortlist(settings)
if not available:
# Everything in the shortlist is either enabled, dismissed, or
# out-of-scope for this user. We still run the classifier (for
# is_vague) but with an empty candidate list — the model will
# only fill in is_vague and return no suggestions.
pass
result = await asyncio.wait_for(
_call_classifier(settings, prompt, available),
timeout=timeout_s,
)
# Re-validate suggestion ids against the curated shortlist so a
# hallucinated id can't reach the frontend.
# Re-validate ids against the curated shortlist so hallucinations can't reach the frontend.
valid_ids = {e["id"] for e in CURATED_SHORTLIST}
result["suggestions"] = [
_decorate(s, available) for s in result.get("suggestions", [])
if isinstance(s, dict) and s.get("id") in valid_ids
]
# Drop anything that ended up with no matching available entry
# (e.g. already enabled by the user between preflight and now).
result["suggestions"] = [s for s in result["suggestions"] if s is not None]
result["is_vague"] = bool(result.get("is_vague"))
# Suppress suggestions on concrete prompts. False-positives here
# are worse than missed positives — interrupting a user who typed
# "refactor foo.ts" to ask about GitHub MCP would feel broken.
# Vague/info-gathering prompts are where suggestions help; concrete
# tasks should just launch.
# Suppress on concrete prompts; false-positives feel broken (interrupting "refactor foo.ts" to suggest GitHub MCP).
if not result["is_vague"]:
result["suggestions"] = []
return result
@@ -251,7 +188,7 @@ async def _call_classifier(settings, prompt: str, available: list[CuratedEntry])
client = get_anthropic_client_for_model(settings, aux_model)
catalog_lines = "\n".join(
f"- id: {e['id']} | {e['title']} — {e['description']}"
f"- id: {e['id']} | {e['title']}; {e['description']}"
for e in available
) or "- (no candidate services available for this user)"
@@ -287,7 +224,6 @@ async def _call_classifier(settings, prompt: str, available: list[CuratedEntry])
messages=[{"role": "user", "content": user_turn}],
)
# Extract text content. Handle both string and content-block shapes.
text = ""
if isinstance(resp.content, list):
for block in resp.content:
@@ -298,7 +234,6 @@ async def _call_classifier(settings, prompt: str, available: list[CuratedEntry])
text = str(resp.content)
text = text.strip()
# Strip any accidental code fences
if text.startswith("```"):
text = re.sub(r"^```(?:json)?\s*", "", text)
text = re.sub(r"\s*```\s*$", "", text)
backend/apps/agents/models.py
+35 -86
View File
@@ -11,7 +11,7 @@ class AgentConfig(BaseModel):
system_prompt: Optional[str] = None
allowed_tools: list[str] = Field(default_factory=lambda: ["Read", "Edit", "Write", "Bash", "Glob", "Grep", "AskUserQuestion"])
max_turns: Optional[int] = None
target_directory: Optional[str] = None # if None, uses repo root
target_directory: Optional[str] = None
dashboard_id: Optional[str] = None
class ApprovalRequest(BaseModel):
@@ -20,12 +20,28 @@ class ApprovalRequest(BaseModel):
tool_name: str
tool_input: dict[str, Any]
created_at: datetime = Field(default_factory=datetime.now)
# Set when this approval was triggered by the sensitive-path override
# rather than the user's normal "ask" policy. Three correlated fields:
# - sensitive_pattern: the fnmatch pattern (canonical id; what we
# persist into the trusted allowlist if the user opts in).
# - sensitive_label: short human label (e.g. "SSH folder (~/.ssh)").
# - sensitive_why: plain-English risk explanation; lets the modal
# justify itself to a non-developer.
# All three None for ordinary "ask" approvals.
sensitive_pattern: Optional[str] = None
sensitive_label: Optional[str] = None
sensitive_why: Optional[str] = None
class ApprovalResponse(BaseModel):
request_id: str
behavior: Literal["allow", "deny"]
message: Optional[str] = None
updated_input: Optional[dict[str, Any]] = None
# When the user checked "Always allow files like this" on a sensitive-
# path approval, the backend persists the matched fnmatch pattern
# (from ApprovalRequest.sensitive_pattern) to disk so future writes
# against the same pattern skip the modal.
trust_pattern: bool = False
class Message(BaseModel):
id: str = Field(default_factory=lambda: uuid4().hex)
@@ -39,26 +55,15 @@ class Message(BaseModel):
forced_tools: Optional[list[str]] = None
images: Optional[list[dict]] = None
hidden: bool = False
# Optional client-generated id used by the frontend to reconcile an
# optimistic message bubble (rendered synchronously on send) with the
# server-confirmed echo. Plumbed through send_message and round-tripped
# back via the agent:message WS event so the frontend can dedupe.
# Frontend-generated id for optimistic-bubble dedup against the server echo.
client_message_id: Optional[str] = None
# Wall-clock duration in milliseconds spent producing this message's
# content. For thinking blocks: time from content_block_start →
# content_block_stop. Lets the persisted ThinkingBubble show
# "Thought for Ns" on reload instead of falling back to the static
# "Thoughts" label. Optional for back-compat with messages saved
# before this field existed.
# Wall-clock ms producing this message's content; for thinking, content_block_start -> stop. Lets reloaded bubbles show "Thought for Ns".
elapsed_ms: Optional[int] = None
# Approximate output tokens for this message's content. For thinking
# blocks we use the same char/3.6 heuristic the live UI uses so the
# number frozen on the persisted bubble matches what the user saw
# rising during the stream. Pure display, not billing.
# Approx output tokens; thinking uses char/3.6 to match the live UI's count. Display only.
tokens: Optional[int] = None
# tool_count drives the "3 tools used" segment on the thinking pill.
# Drives the "N tools used" segment on the thinking pill.
tool_count: Optional[int] = None
# combined input + output + children tokens for the turn (overloaded name).
# Combined input + output + children tokens for the turn (overloaded name).
input_tokens: Optional[int] = None
class MessageBranch(BaseModel):
@@ -85,40 +90,22 @@ class AgentSession(BaseModel):
allowed_tools: list[str] = Field(default_factory=list)
max_turns: Optional[int] = None
cwd: Optional[str] = None
# Origin remote and branch resolved at session start. Persisted so a
# resumed session reattaches to the same project even if the user has
# since `cd`'d elsewhere; also surfaced in the session list UI so the
# user can tell two sessions apart by repo.
# Resolved at session start so resume reattaches to the same repo even after the user cd's elsewhere.
repo_url: Optional[str] = None
branch: Optional[str] = None
created_at: datetime = Field(default_factory=datetime.now)
closed_at: Optional[datetime] = None
# Wall-clock of the first stream event from the agent SDK. Set once
# at the start of the first turn so resumed sessions can show "first
# response was at HH:MM" in the session list without rescanning the
# message log.
# Wall-clock of the first stream event so resumed sessions can show "first response at HH:MM" without rescan.
first_response_at: Optional[datetime] = None
# Operational log of HITL approval decisions, one entry per request:
# {tool, behavior, decision_ms}. Persisted alongside the session so a
# reload restores the full approval timeline (which calls were
# approved, denied, and how long each took).
# HITL approval log: {tool, behavior, decision_ms} per entry.
approval_decisions: list[dict] = Field(default_factory=list)
cost_usd: float = 0.0
tokens: dict[str, int] = Field(default_factory=lambda: {"input": 0, "output": 0})
# Total wall-clock ms the agent spent in `status="running"`. Accumulates
# across turns; persists across resume. Used by the session-close
# report so we can report "agent active time" alongside total session
# duration. Off by default so legacy sessions deserialize cleanly.
# Total ms in status="running", accumulated across turns/resume; powers session-close "agent active time".
agent_active_ms: int = 0
# Accumulated wall-clock ms spent on each model. Updated when the
# active model changes (model switch) or on close. Surfaced in the
# session header so the user can see "Sonnet: 45s · Haiku: 12s"
# without scanning turns by hand.
# Per-model wall-clock ms; updated on model switch or close.
time_per_model: dict[str, int] = Field(default_factory=dict)
# Per-tool latency rollup: { tool_name: { count, total_ms, max_ms } }.
# Populated as tools complete. Surfaced in the session "tools used"
# row so the user can see which tool calls were slow without
# opening every turn.
# Per-tool latency: { tool_name: { count, total_ms, max_ms } }.
tool_latencies: dict[str, dict] = Field(default_factory=dict)
browser_domains: list[str] = Field(default_factory=list)
messages: list[Message] = Field(default_factory=list)
@@ -130,58 +117,20 @@ class AgentSession(BaseModel):
browser_id: Optional[str] = None
parent_session_id: Optional[str] = None
needs_fork: bool = False
# Stronger than needs_fork: when True, the next turn drops `resume=`
# entirely and replays history into a brand-new sdk_session_id. This
# is the only way to make the bundled CLI re-read mcp_servers from
# the rebuilt options dict — `fork_session=True` only forks the
# conversation tree, it inherits the original transport's MCP server
# set. Set after MCPActivate when prior turns exist so the newly
# activated server's tools actually reach the model.
# Stronger than needs_fork: drop resume= and replay history into a fresh sdk_session_id; fork_session alone won't re-read mcp_servers.
needs_fresh_session: bool = False
# Set when MCPActivate (or analogous activation) wants the agent to
# auto-continue immediately after the current turn ends — without
# requiring the user to type another message. The agent loop reads
# this at the end of `_run_agent_loop`; if set, it clears it and
# dispatches a new hidden turn with `pending_continuation_prompt` as
# the prompt. Race-free vs. the original asyncio-task approach.
# Auto-continue: agent loop dispatches a hidden turn at end-of-loop using pending_continuation_prompt. Race-free vs background tasks.
pending_continuation: bool = False
pending_continuation_prompt: Optional[str] = None
# Sanitized server names (matching tools_lib._sanitize_server_name) of MCP
# servers the model has explicitly activated this session via the
# MCPActivate meta-tool. Empty by default — the gate in
# _build_mcp_servers intersects connected MCPs with this list, so no
# MCP tool is callable until the model searches for and activates a
# server. The product invariant is that this is non-bypassable: the
# filter lives at the dispatch layer (mcp_servers passed to the SDK),
# not the prompt layer.
# Sanitized server names model has explicitly activated this session; _build_mcp_servers intersects connected MCPs with this. Non-bypassable; dispatch-layer gate.
active_mcps: list[str] = Field(default_factory=list)
# Estimated framework preamble tokens (preset + tool defs + MCP descs +
# composed prompt). Subtracted from displayed input for honest "this turn"
# numbers. Heuristic; clamped >= 0.
# Heuristic preamble tokens (preset + tool defs + MCP descs + composed prompt); subtracted from displayed input.
framework_overhead_tokens: int = 0
# Compaction state. compact_threshold_pct is the live ctx_used ratio
# that triggers _maybe_compact at the next turn boundary — turn-based
# thresholds break under uneven workloads (one big Bash dump fills
# context fast; 30 chitchat turns barely move it). 0.65 = 130K of the
# 200K standard tier. compacted_through_msg_id is the last message id
# covered by the most recent summary so we don't re-summarize on
# every turn.
# Live ctx_used ratio triggering _maybe_compact at the next turn boundary; turn-based thresholds break under uneven workloads. 0.65 = 130K of 200K.
compact_threshold_pct: float = 0.65
compacted_through_msg_id: Optional[str] = None
# Pre-send hard guard. Fires later than the compaction threshold —
# 0.90 of 200K = 180K — to give the auto-compact path a chance to
# bring the request back under the ceiling. If still over after
# compaction, LRU-trim the oldest active_mcps. Past this we surface
# the friendly context-overflow card instead of letting a 429 hit.
# Hard pre-send guard at 0.90 (= 180K); past compaction we LRU-trim active_mcps, then surface the overflow card.
context_soft_cap_pct: float = 0.90
context_window: int = 200_000
# How much the model should "think" before answering. Provider-agnostic
# value that gets translated per-API in agent_manager:
# off — no thinking
# low — minimal thinking (fastest)
# medium — balanced
# high — extensive thinking (slowest, smartest)
# auto — let the model / provider default decide (recommended)
# Only applies to models flagged with reasoning: True in the registry.
# Existing sessions without this field will default to "auto".
# Provider-agnostic thinking level (off/low/medium/high/auto), translated per-API in agent_manager; only affects reasoning-flagged models.
thinking_level: Literal["off", "low", "medium", "high", "auto"] = "auto"
backend/apps/agents/openai_passthrough.py
+4 -37
View File
@@ -1,28 +1,4 @@
"""Tiny OpenAI-API pass-through with `max_tokens` → `max_completion_tokens`
rename for GPT-5.x models.
Why this exists
---------------
OpenAI's GPT-5 family (gpt-5.4-mini, gpt-5.5, gpt-5.3-codex, etc.)
rejects the legacy `max_tokens` parameter with HTTP 400:
"Unsupported parameter: 'max_tokens' is not supported with this model.
Use 'max_completion_tokens'."
Anthropic's CLI emits requests in Anthropic format (which uses `max_tokens`),
9Router 0.3.60 translates Anthropic→OpenAI and preserves `max_tokens`
(it doesn't know about the GPT-5 change). We can't bump 9Router because
0.3.60 is pinned to fix a separate WebSearch regression in the 0.3.x
range (see backend/apps/nine_router.py:27-36).
So we slot a thin proxy between 9Router and api.openai.com. The CLI is
unaware: it sees its OPENAI_BASE_URL pointing at this local passthrough,
not OpenAI. We rename the field for GPT-5 models and forward unchanged
otherwise. Streaming + non-streaming both work because we proxy bytes.
Mounted at `/api/openai-passthrough` and consumed by setting
OPENAI_BASE_URL to `http://127.0.0.1:<port>/api/openai-passthrough/v1`
in the CLI's spawn env (see agent_manager.py).
"""
"""Tiny OpenAI passthrough renaming max_tokens to max_completion_tokens for GPT-5; 9Router 0.3.60 is pinned and doesn't know the change."""
import json
import logging
@@ -45,8 +21,7 @@ async def openai_passthrough_lifespan():
openai_passthrough = SubApp("openai-passthrough", openai_passthrough_lifespan)
# Models that REQUIRE max_completion_tokens. Mirrors anthropic_proxy.py's
# matcher but lives here so this module doesn't depend on that one.
# Mirrors anthropic_proxy.py's GPT-5 matcher; duplicated to avoid the cross-module dep.
_GPT5_PREFIXES = ("gpt-5",)
_OPENAI_UPSTREAM = "https://api.openai.com/v1"
_HOP_HEADERS = {
@@ -60,7 +35,6 @@ def _is_gpt5(model: str) -> bool:
m = (model or "").strip().lower()
if not m:
return False
# Strip routing prefixes 9Router may have added.
for prefix in ("openai/", "cx/", "openrouter/", "or:openai/", "cp/", "cp-"):
if m.startswith(prefix):
m = m[len(prefix):]
@@ -69,12 +43,7 @@ def _is_gpt5(model: str) -> bool:
def _scrub_max_tokens(body: bytes) -> bytes:
"""Rename max_tokens → max_completion_tokens for GPT-5 models.
Bytes-in/out, never raises. No-op if body isn't JSON, model isn't GPT-5,
or max_tokens isn't present. If both fields are present (unlikely),
drops the legacy field so OpenAI doesn't 400 on the conflict.
"""
"""Rename max_tokens to max_completion_tokens for GPT-5; bytes in/out, never raises."""
if not body:
return body
try:
@@ -113,9 +82,7 @@ async def passthrough(rest: str, request: Request):
if request.url.query:
upstream_url = f"{upstream_url}?{request.url.query}"
# Stream upstream response body straight back to the caller. httpx's
# streaming context handles Server-Sent Events the CLI uses for chat
# completions without buffering the full response in memory.
# Stream upstream body back; httpx handles SSE without buffering the full response.
client = httpx.AsyncClient(timeout=httpx.Timeout(connect=10.0, read=300.0, write=60.0, pool=30.0))
try:
upstream_req = client.build_request(
backend/apps/agents/providers/registry.py
+17 -17
View File
@@ -249,7 +249,7 @@ async def fetch_openrouter_models(api_key: str | None) -> list[dict]:
continue
if isinstance(out_mods, list) and out_mods and "text" not in out_mods:
continue
# Tools required — agent loop doesn't work without function calling.
# Tools required; agent loop doesn't work without function calling.
params = m.get("supported_parameters") or []
if not isinstance(params, list) or "tools" not in params:
continue
@@ -310,7 +310,7 @@ _CUSTOM_VALUE_PREFIX = "custom/"
def _custom_provider_slug_for_lookup(name: str) -> str:
"""Mirror nine_router._custom_provider_slug — duplicated here to avoid
"""Mirror nine_router._custom_provider_slug; duplicated here to avoid
importing from nine_router (circular: nine_router imports from settings)."""
import re
s = re.sub(r"[^a-zA-Z0-9-]+", "-", (name or "").strip().lower()).strip("-")
@@ -336,7 +336,7 @@ def _find_builtin_model(short_name: str) -> dict | None:
"""Look up a model entry by its short `value`.
OpenRouter entries (prefixed `or:<vendor>/<model>`) and custom-provider
entries (prefixed `custom/<slug>/<model_id>`) aren't in BUILTIN_MODELS —
entries (prefixed `custom/<slug>/<model_id>`) aren't in BUILTIN_MODELS ,
they're synthesised on demand so the rest of the routing code can treat
them like BUILTIN_MODELS entries."""
for models in BUILTIN_MODELS.values():
@@ -501,7 +501,7 @@ async def resolve_aux_model(
return ("cx/gpt-5.4-mini", base_url)
if "gemini-cli" in connected:
return ("gc/gemini-3.1-flash-lite-preview", base_url)
# OR is metered, hence last — saves OR-only users from "Untitled session" hell.
# OR is metered, hence last; saves OR-only users from "Untitled session" hell.
if "openrouter" in connected:
return (or_aux, base_url)
@@ -519,7 +519,7 @@ def get_context_window(provider: str, model: str, settings: AppSettings | None =
if m["value"] == model:
return m.get("context_window", 128_000)
# Check custom providers — picker values are `custom/<slug>/<bare_model>`;
# Check custom providers; picker values are `custom/<slug>/<bare_model>`;
# cp.models[].value stores the bare model id the user typed. Match the
# bare-model tail against any custom provider's models list.
if settings:
@@ -538,7 +538,7 @@ def get_context_window(provider: str, model: str, settings: AppSettings | None =
# ---------------------------------------------------------------------------
# Curated model tiers — Intelligence, Speed, Cost on a 1-5 scale
# Curated model tiers; Intelligence, Speed, Cost on a 1-5 scale
# ---------------------------------------------------------------------------
#
# Hand-tuned from public benchmarks + per-token pricing (knowledge cutoff
@@ -737,7 +737,7 @@ def _heuristic_tiers(label: str, output_cost_per_1m: float, reasoning: bool) ->
import re as _re
out = output_cost_per_1m or 0.0
# Cost bucket — same 5-tier cost ladder as before.
# Cost bucket; same 5-tier cost ladder as before.
if out < 0.5:
cb = 1
elif out < 2:
@@ -774,7 +774,7 @@ def _heuristic_tiers(label: str, output_cost_per_1m: float, reasoning: bool) ->
elif param_b > 0:
size_tier = 1
else:
size_tier = 0 # unknown — fall back to cost
size_tier = 0 # unknown; fall back to cost
# Intelligence is the max of cost bucket and parsed size tier.
# Cost is high-confidence for closed-source frontier; size is
@@ -783,7 +783,7 @@ def _heuristic_tiers(label: str, output_cost_per_1m: float, reasoning: bool) ->
intel = max(cb, size_tier)
if reasoning and intel < 4:
# Reasoning is a strong intelligence signal but only for
# genuinely smaller models — frontier closed-source already
# genuinely smaller models; frontier closed-source already
# caps at 5, so don't double-count there.
intel += 1
@@ -851,15 +851,15 @@ def compute_billing_kind(
settings,
) -> str:
"""Return one of:
'subscription' — covered by an OAuth sub or Pro plan; hide cost row
'api_key' — direct API-key path (Anthropic / OpenAI / Gemini)
'free' — genuinely $0 per token (rate-limited OR :free tier)
'paid' — per-token metering through OpenRouter; show pricing
'subscription'; covered by an OAuth sub or Pro plan; hide cost row
'api_key' ; direct API-key path (Anthropic / OpenAI / Gemini)
'free' ; genuinely $0 per token (rate-limited OR :free tier)
'paid' ; per-token metering through OpenRouter; show pricing
Why 'api_key' is split from 'paid': both meter per-token, but the user
is paying a different counterparty. Letting the picker filter chips
"API key" vs "Subscription" gives users a clear way to scope to their
billing relationship — direct API key vs OAuth subscription — instead
billing relationship; direct API key vs OAuth subscription; instead
of conflating them under a generic "paid" bucket.
Subscription paths:
@@ -894,7 +894,7 @@ def compute_billing_kind(
COST_PER_1M_TOKENS: dict[tuple[str, str], tuple[float, float]] = {
# (provider, model): (input_cost_per_1M, output_cost_per_1M)
# NOTE: `calculate_cost` is currently unused in the live path — real
# NOTE: `calculate_cost` is currently unused in the live path; real
# cost numbers come from 9Router's usage stats. These entries are kept
# so the table matches BUILTIN_MODELS and can
# be used by any future native-loop path. Subscription-routed models
@@ -905,14 +905,14 @@ COST_PER_1M_TOKENS: dict[tuple[str, str], tuple[float, float]] = {
("Anthropic", "opus"): (5.0, 25.0),
("Anthropic", "opus-4-7"): (5.0, 25.0),
("Anthropic", "haiku"): (1.0, 5.0),
# OpenAI — Codex subscription path, user pays nothing per token
# OpenAI; Codex subscription path, user pays nothing per token
("OpenAI", "gpt-5.5"): (0.0, 0.0),
("OpenAI", "gpt-5.4"): (0.0, 0.0),
("OpenAI", "gpt-5.4-mini"): (0.0, 0.0),
("OpenAI", "gpt-5.3-codex"): (0.0, 0.0),
("OpenAI", "gpt-5.3-codex-high"): (0.0, 0.0),
("OpenAI", "gpt-5.3-codex-xhigh"): (0.0, 0.0),
# Google — Gemini CLI subscription path, user pays nothing per token
# Google; Gemini CLI subscription path, user pays nothing per token
("Google", "gemini-3.1-pro"): (0.0, 0.0),
("Google", "gemini-3.1-flash-lite"): (0.0, 0.0),
("Google", "gemini-3-pro"): (0.0, 0.0),
backend/apps/agents/schedule_mcp_server.py
+337
View File
@@ -0,0 +1,337 @@
#!/usr/bin/env python3
"""Stdio MCP server exposing scheduled-workflow tools to the agent.
Why this exists: the agent should be able to schedule recurring work on
the user's behalf, but ALWAYS through the native scheduler (visible,
auditable, cost-capped) rather than `crontab`. Each tool is a thin
wrapper around /api/workflows/*. The descriptions are written to nudge
the agent toward AskUserQuestion-first behavior (confirm cadence with
the user before calling ScheduleWorkflow).
"""
import json
import sys
import os
import urllib.request
import urllib.error
BACKEND_PORT = os.environ.get("OPENSWARM_PORT", "8324")
BACKEND_AUTH = os.environ.get("OPENSWARM_AUTH_TOKEN", "")
BACKEND_BASE = f"http://127.0.0.1:{BACKEND_PORT}/api/workflows"
PARENT_SESSION_ID = os.environ.get("OPENSWARM_PARENT_SESSION_ID", "")
DASHBOARD_ID = os.environ.get("OPENSWARM_DASHBOARD_ID", "")
PRESETS = {
"daily_morning": {"enabled": True, "repeat_unit": "day", "repeat_every": 1, "hour": 9, "minute": 0, "on_days": []},
"weekdays_morning": {"enabled": True, "repeat_unit": "week", "repeat_every": 1, "hour": 9, "minute": 0, "on_days": [1, 2, 3, 4, 5]},
"weekly_monday": {"enabled": True, "repeat_unit": "week", "repeat_every": 1, "hour": 9, "minute": 0, "on_days": [1]},
"weekly_friday": {"enabled": True, "repeat_unit": "week", "repeat_every": 1, "hour": 17, "minute": 0, "on_days": [5]},
"monthly_first": {"enabled": True, "repeat_unit": "month", "repeat_every": 1, "hour": 9, "minute": 0, "on_days": []},
}
TOOLS = [
{
"name": "ScheduleWorkflow",
"description": (
"Create a recurring scheduled workflow for the user. Use this "
"ONLY after confirming cadence with the user via AskUserQuestion "
"(do not assume — the user must pick or accept the time). "
"The workflow runs the listed steps on the schedule and is "
"visible in the user's Workflows hub. Never use crontab, "
"launchctl, or schtasks to schedule recurring work; always use "
"this tool so the user can see, pause, edit, or delete it. "
"After creating, briefly confirm to the user what was scheduled."
),
"inputSchema": {
"type": "object",
"properties": {
"title": {"type": "string", "description": "Short workflow name shown in the hub and on the dashboard card."},
"steps": {
"type": "array",
"items": {"type": "string"},
"description": "Ordered list of instructions for the agent to execute on each fire. Each string is one step.",
},
"preset": {
"type": "string",
"enum": ["daily_morning", "weekdays_morning", "weekly_monday", "weekly_friday", "monthly_first", "custom"],
"description": "Cadence preset. Use 'custom' to specify your own hour/minute/days.",
},
"hour": {"type": "integer", "description": "Hour 0-23 in the user's local time. Required when preset='custom'."},
"minute": {"type": "integer", "description": "Minute 0/15/30/45. Required when preset='custom'."},
"repeat_unit": {"type": "string", "enum": ["day", "week", "month"], "description": "Required when preset='custom'."},
"on_days": {
"type": "array",
"items": {"type": "integer"},
"description": "Weekdays (Sun=0..Sat=6) when preset='custom' and repeat_unit='week'.",
},
"source_session_id": {"type": "string", "description": "Optional; the chat session this workflow was created from. Inherits its tool surface."},
},
"required": ["title", "steps", "preset"],
},
},
{
"name": "ListScheduledWorkflows",
"description": "List the user's scheduled workflows. Use this to find a workflow the user is referring to before editing or deleting it.",
"inputSchema": {"type": "object", "properties": {}},
},
{
"name": "UpdateScheduledWorkflow",
"description": "Modify an existing scheduled workflow. Only pass the fields you want to change. Always confirm with the user via AskUserQuestion before making changes that meaningfully alter behavior (cadence, steps, permissions).",
"inputSchema": {
"type": "object",
"properties": {
"workflow_id": {"type": "string"},
"title": {"type": "string"},
"steps": {"type": "array", "items": {"type": "string"}},
"schedule_enabled": {"type": "boolean", "description": "Quick on/off without changing other schedule fields."},
"hour": {"type": "integer"},
"minute": {"type": "integer"},
"repeat_unit": {"type": "string", "enum": ["day", "week", "month"]},
"on_days": {"type": "array", "items": {"type": "integer"}},
},
"required": ["workflow_id"],
},
},
{
"name": "DeleteScheduledWorkflow",
"description": "Permanently delete a scheduled workflow. Cannot be undone. ALWAYS confirm via AskUserQuestion before calling this — the user should pick from a list, not have you guess.",
"inputSchema": {
"type": "object",
"properties": {"workflow_id": {"type": "string"}},
"required": ["workflow_id"],
},
},
{
"name": "PauseAllWorkflows",
"description": "Globally pause every scheduled workflow. In-flight runs finish; future runs are blocked until resumed. Use when the user wants a temporary stop (vacation, debugging) without deleting workflows.",
"inputSchema": {"type": "object", "properties": {}},
},
{
"name": "ResumeAllWorkflows",
"description": "Resume scheduled workflows after a previous PauseAllWorkflows.",
"inputSchema": {"type": "object", "properties": {}},
},
{
"name": "RunWorkflowNow",
"description": "Trigger an immediate one-off run of a scheduled workflow. The schedule continues to fire on its normal cadence in addition.",
"inputSchema": {
"type": "object",
"properties": {"workflow_id": {"type": "string"}},
"required": ["workflow_id"],
},
},
]
def send_response(id_, result=None, error=None):
msg = {"jsonrpc": "2.0", "id": id_}
if error is not None:
msg["error"] = error
else:
msg["result"] = result
sys.stdout.write(json.dumps(msg) + "\n")
sys.stdout.flush()
def _call(method: str, path: str, body=None) -> dict:
url = BACKEND_BASE + path
data = json.dumps(body).encode() if body is not None else None
headers = {"Content-Type": "application/json"}
if BACKEND_AUTH:
headers["Authorization"] = f"Bearer {BACKEND_AUTH}"
req = urllib.request.Request(url, data=data, headers=headers, method=method)
try:
with urllib.request.urlopen(req, timeout=30) as resp:
return json.loads(resp.read().decode() or "null") or {}
except urllib.error.HTTPError as e:
body_err = e.read().decode() if e.fp else str(e)
return {"_error": f"HTTP {e.code}: {body_err}"}
except Exception as e:
return {"_error": str(e)}
def _build_schedule_from_preset(preset: str, args: dict) -> dict:
base = {"timezone": "local", "on_missed": "skip", "ends_at": None, "max_runs": None, "runs_count": 0}
if preset == "custom":
return {
**base,
"enabled": True,
"repeat_unit": args.get("repeat_unit", "day"),
"repeat_every": 1,
"hour": int(args.get("hour", 9)),
"minute": int(args.get("minute", 0)),
"on_days": list(args.get("on_days") or []),
}
preset_def = PRESETS.get(preset)
if not preset_def:
return {}
return {**base, **preset_def, "repeat_every": 1}
def handle_schedule_workflow(args: dict) -> dict:
title = args.get("title") or "Scheduled workflow"
steps_in = args.get("steps") or []
preset = args.get("preset") or "daily_morning"
schedule = _build_schedule_from_preset(preset, args)
if not schedule:
return _err(f"Unknown preset: {preset}. Use one of: {list(PRESETS.keys()) + ['custom']}.")
body = {
"title": title,
"steps": [{"id": f"s{i+1}", "text": s} for i, s in enumerate(steps_in) if s],
"schedule": schedule,
"source_session_id": args.get("source_session_id") or PARENT_SESSION_ID or None,
}
r = _call("POST", "/create", body)
if "_error" in r:
return _err(r["_error"])
wid = r.get("id", "")
nxt = r.get("next_run_at") or "soon"
return _ok(f"Scheduled \"{title}\" ({preset}). Workflow id: {wid}. Next run: {nxt}. The user can view, pause, or edit it in the Workflows hub.")
def handle_list(_args: dict) -> dict:
r = _call("GET", "/list")
if "_error" in r:
return _err(r["_error"])
ws = r.get("workflows", [])
if not ws:
return _ok("No scheduled workflows yet.")
lines = ["Scheduled workflows:"]
for w in ws:
s = w.get("schedule") or {}
enabled = s.get("enabled")
unit = s.get("repeat_unit", "?")
hour = s.get("hour")
title = w.get("title", "(untitled)")
wid = w.get("id", "")
state = "ON" if enabled else "off"
lines.append(f" - {title} [{state}] {unit} at {hour:02d}:00 (id: {wid})")
return _ok("\n".join(lines))
def handle_update(args: dict) -> dict:
wid = args.get("workflow_id") or ""
if not wid:
return _err("workflow_id is required.")
cur = _call("GET", f"/{wid}")
if "_error" in cur:
return _err(cur["_error"])
sched = cur.get("schedule") or {}
patch: dict = {}
if "title" in args: patch["title"] = args["title"]
if "steps" in args:
patch["steps"] = [{"id": f"s{i+1}", "text": s} for i, s in enumerate(args["steps"] or []) if s]
sched_patch = dict(sched)
sched_dirty = False
if "schedule_enabled" in args:
sched_patch["enabled"] = bool(args["schedule_enabled"])
sched_dirty = True
for k in ("hour", "minute", "repeat_unit", "on_days"):
if k in args:
sched_patch[k] = args[k]
sched_dirty = True
if sched_dirty:
patch["schedule"] = sched_patch
if not patch:
return _ok(f"No changes requested for workflow {wid}.")
r = _call("PATCH", f"/{wid}", patch)
if "_error" in r:
return _err(r["_error"])
return _ok(f"Updated \"{r.get('title', wid)}\". Next run: {r.get('next_run_at') or 'paused/unscheduled'}.")
def handle_delete(args: dict) -> dict:
wid = args.get("workflow_id") or ""
if not wid:
return _err("workflow_id is required.")
r = _call("DELETE", f"/{wid}")
if "_error" in r:
return _err(r["_error"])
return _ok(f"Deleted workflow {wid}.")
def handle_pause_all(_args: dict) -> dict:
r = _call("POST", "/pause-all")
if "_error" in r:
return _err(r["_error"])
return _ok("All scheduled workflows are paused. In-flight runs will finish; future fires are blocked. Resume with ResumeAllWorkflows.")
def handle_resume_all(_args: dict) -> dict:
r = _call("POST", "/resume-all")
if "_error" in r:
return _err(r["_error"])
return _ok("Scheduled workflows resumed.")
def handle_run_now(args: dict) -> dict:
wid = args.get("workflow_id") or ""
if not wid:
return _err("workflow_id is required.")
r = _call("POST", f"/{wid}/run")
if "_error" in r:
return _err(r["_error"])
if r.get("status") == "skipped":
return _ok(f"Run was skipped: {r.get('error', 'unknown reason')}.")
return _ok(f"Run started (run id: {r.get('run_id', '')}). Output will appear in the workflow's History.")
def _ok(text: str) -> dict:
return {"content": [{"type": "text", "text": text}]}
def _err(text: str) -> dict:
return {"content": [{"type": "text", "text": f"Error: {text}"}], "isError": True}
HANDLERS = {
"ScheduleWorkflow": handle_schedule_workflow,
"ListScheduledWorkflows": handle_list,
"UpdateScheduledWorkflow": handle_update,
"DeleteScheduledWorkflow": handle_delete,
"PauseAllWorkflows": handle_pause_all,
"ResumeAllWorkflows": handle_resume_all,
"RunWorkflowNow": handle_run_now,
}
def main():
for line in sys.stdin:
line = line.strip()
if not line:
continue
try:
msg = json.loads(line)
except json.JSONDecodeError:
continue
method = msg.get("method")
id_ = msg.get("id")
params = msg.get("params", {})
if method == "initialize":
send_response(id_, {
"protocolVersion": "2024-11-05",
"capabilities": {"tools": {}},
"serverInfo": {"name": "openswarm-schedule", "version": "1.0.0"},
})
elif method == "notifications/initialized":
pass
elif method == "tools/list":
send_response(id_, {"tools": TOOLS})
elif method == "tools/call":
tool_name = params.get("name", "")
arguments = params.get("arguments", {})
handler = HANDLERS.get(tool_name)
if handler is None:
send_response(id_, _err(f"Unknown tool: {tool_name}"))
else:
send_response(id_, handler(arguments))
elif method == "ping":
send_response(id_, {})
elif id_ is not None:
send_response(id_, error={"code": -32601, "message": f"Method not found: {method}"})
if __name__ == "__main__":
main()
backend/apps/agents/seq_log.py
+10 -88
View File
@@ -1,49 +1,4 @@
"""Per-session WS event sequencing, ring buffer, and terminal-event persistence.
Why this exists
---------------
WS sockets die for a thousand reasons that have nothing to do with the
agent task: laptop sleep, captive portals, NAT idle timeout, VPN
renegotiation. Without this module, a transient drop is fatal —
mid-stream events are lost forever and the UI can't tell whether the
run finished or merely went quiet.
Contract
--------
Every WS event for a session goes through `stamp(...)`, which is an
async context manager that:
1. Acquires the per-session lock.
2. Bumps a monotonic `seq` integer.
3. Appends the JSON payload to a bounded ring buffer.
4. Yields (seq, payload_str) to the caller.
5. Holds the lock until the caller exits the `async with` — meaning
the caller's `ws.send_text(...)` happens *under the same lock*,
guaranteeing wire order == seq order even when many coroutines
broadcast concurrently.
Without (5), two coroutines can each get a unique seq under separate
lock acquisitions, yet the higher-seq event can reach the wire first
because asyncio scheduled its `send_text` earlier. That corrupts both
wire order and the ring buffer on resume.
Resume protocol
---------------
On reconnect, the client sends `client:resume {connection_uuid,
last_seq}`. The server:
- Returns ring-buffer events with `seq > last_seq` if available.
- Returns `agent:gap_detected` if `last_seq` is older than the
oldest buffered seq — the client falls back to a REST refresh.
- Returns the persisted terminal event (if any) when the session
is no longer in memory at all (e.g. after a process restart).
Persistence
-----------
Terminal events (status: completed/stopped/error) are written
atomically to disk so a client that comes back hours later — long
after the in-memory ring buffer has been GC'd — still sees the right
outcome instead of a spinner that never resolves. Persistence is
opportunistic: an I/O error never blocks the broadcast path.
"""
"""Per-session WS event sequencing, ring buffer, and terminal-event persistence for resilient reconnects."""
from __future__ import annotations
@@ -57,9 +12,7 @@ from typing import AsyncIterator, Optional
logger = logging.getLogger(__name__)
# Ring buffer size per session. ~500 events comfortably covers a 30s
# transient drop even in the busiest streams (thinking deltas at
# ~20Hz). Memory is bounded: ~50KB per active session.
# 500 events covers a 30s drop even at ~20Hz thinking deltas (~50KB/session).
BUFFER_LIMIT = 500
TERMINAL_STATUSES = {"completed", "stopped", "error"}
@@ -73,8 +26,7 @@ class _SessionSeqLog:
def __init__(self) -> None:
self.lock: asyncio.Lock = asyncio.Lock()
self.seq: int = 0
# Each entry: (seq, json_payload_str). Pre-serialized so a
# replay doesn't redo json.dumps for every reconnect.
# (seq, json_payload_str): pre-serialized so replays don't redo json.dumps per reconnect.
self.buffer: deque[tuple[int, str]] = deque(maxlen=BUFFER_LIMIT)
@@ -83,8 +35,7 @@ class SeqLogStore:
def __init__(self, persist_dir: Optional[str] = None) -> None:
self._per_session: dict[str, _SessionSeqLog] = {}
# Coarse lock guarding only the dict's setdefault path. Held
# for nanoseconds; never crosses an `await` past the `_get`.
# Coarse lock guards only the setdefault path; never crosses an await.
self._dict_lock = asyncio.Lock()
self._persist_dir = persist_dir
if persist_dir:
@@ -111,13 +62,7 @@ class SeqLogStore:
async def stamp(
self, session_id: str, event: str, data: dict
) -> AsyncIterator[tuple[int, str]]:
"""Atomically assign a seq, buffer it, and yield (seq, payload).
Caller is expected to perform the actual `send_text` *inside*
the `async with` block. The per-session lock is held for the
entire body, so wire order is guaranteed equal to seq order
no matter how many tasks broadcast concurrently.
"""
"""Atomically assign seq, buffer, and yield (seq, payload); caller's send must happen inside the with-block."""
log = await self._get_or_create(session_id)
async with log.lock:
log.seq += 1
@@ -135,23 +80,11 @@ class SeqLogStore:
def replay(
self, session_id: str, last_seq: int
) -> tuple[Optional[int], Optional[int], list[str]]:
"""Return (oldest_buffered_seq, newest_buffered_seq, events).
Caller decides what to do with the result:
- `events` empty AND newest_buffered_seq is None: no buffer
for this session in memory. Fall back to persisted
terminal event.
- `last_seq` < `oldest_buffered_seq`: there's a gap. Send
`agent:gap_detected`; the client REST-refreshes.
- Otherwise `events` are the missed payloads in seq order.
"""
"""Return (oldest_buffered_seq, newest_buffered_seq, events)."""
log = self._peek(session_id)
if log is None:
return (None, None, [])
# Snapshot the deque under the lock-free fast path. asyncio is
# single-threaded so a list() of a deque mutated by append is
# safe; eviction (via maxlen) is also a single-step op. We
# don't need to hold the per-session lock for a read.
# asyncio is single-threaded; deque list() is safe vs concurrent append/eviction. No lock needed for read.
snapshot = list(log.buffer)
if not snapshot:
return (None, log.seq, [])
@@ -165,23 +98,17 @@ class SeqLogStore:
log = self._peek(session_id)
return log.seq if log else 0
# ----- Terminal-event persistence -----
def _terminal_path(self, session_id: str) -> Optional[str]:
if not self._persist_dir:
return None
# session ids are uuid4 hex in this codebase, but sanitize
# against path traversal anyway.
# Session ids are uuid4 hex; sanitize anyway against path traversal.
safe = "".join(c for c in session_id if c.isalnum() or c in ("-", "_"))
if not safe:
return None
return os.path.join(self._persist_dir, f"{safe}.json")
def persist_terminal(self, session_id: str, payload_str: str) -> None:
"""Atomic write of a terminal event for post-restart clients.
Best-effort: an I/O failure must never block the broadcast.
"""
"""Atomic write of a terminal event for post-restart clients; best-effort, never blocks broadcast."""
path = self._terminal_path(session_id)
if not path:
return
@@ -206,11 +133,7 @@ class SeqLogStore:
return None
def clear(self, session_id: str) -> None:
"""Drop in-memory log + persisted terminal event.
Use on full session deletion. Closed-but-retained sessions
keep their terminal file so late reconnects still resolve.
"""
"""Drop in-memory log and persisted terminal; for full deletion only, closed-but-retained sessions keep it."""
self._per_session.pop(session_id, None)
path = self._terminal_path(session_id)
if path and os.path.exists(path):
@@ -228,5 +151,4 @@ def _default_persist_dir() -> Optional[str]:
return None
# Process-wide singleton wired to the agents data dir.
seq_log = SeqLogStore(persist_dir=_default_persist_dir())
backend/apps/agents/tools/web.py
+6 -27
View File
@@ -10,8 +10,8 @@ import httpx
from backend.apps.agents.tools.base import BaseTool, ToolContext
_HTTP_TIMEOUT = 30 # seconds
_MAX_OUTPUT_BYTES = 250 * 1024 # ~250 KB — covers ~95% of articles/wikis/docs
_HTTP_TIMEOUT = 30
_MAX_OUTPUT_BYTES = 250 * 1024 # ~250 KB covers ~95% of articles/wikis/docs.
_USER_AGENT = (
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) "
"AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
@@ -25,24 +25,15 @@ def _truncate(text: str, limit: int = _MAX_OUTPUT_BYTES) -> str:
def _strip_html(raw_html: str) -> str:
"""Naive but effective HTML → plain-text conversion."""
# Remove script/style blocks
"""Naive but effective HTML to plain-text conversion."""
text = re.sub(r"<(script|style)[^>]*>.*?</\1>", "", raw_html, flags=re.DOTALL | re.IGNORECASE)
# Remove HTML tags
text = re.sub(r"<[^>]+>", " ", text)
# Decode HTML entities
text = html.unescape(text)
# Collapse whitespace
text = re.sub(r"[ \t]+", " ", text)
text = re.sub(r"\n{3,}", "\n\n", text)
return text.strip()
# ───────────────────────────────────────────────────────────────────────────
# WebSearchTool
# ───────────────────────────────────────────────────────────────────────────
class WebSearchTool(BaseTool):
name = "WebSearch"
description = (
@@ -96,8 +87,6 @@ class WebSearchTool(BaseTool):
body = resp.text
# Parse result blocks – DuckDuckGo wraps each result in
# <div class="result ..."> ... </div>
result_blocks = re.findall(
r'<div[^>]*class="[^"]*result[^"]*"[^>]*>(.*?)</div>\s*(?=<div[^>]*class="[^"]*result|$)',
body,
@@ -109,14 +98,13 @@ class WebSearchTool(BaseTool):
if len(entries) >= num_results:
break
# Title + URL — handle both class-before-href and href-before-class
# Handle both class-before-href and href-before-class attribute orders.
link_match = re.search(
r'<a[^>]*class="[^"]*result__a[^"]*"[^>]*href="([^"]*)"[^>]*>(.*?)</a>',
block,
flags=re.DOTALL,
)
if not link_match:
# Try reversed attribute order
link_match = re.search(
r'<a[^>]*href="([^"]*)"[^>]*class="[^"]*result__a[^"]*"[^>]*>(.*?)</a>',
block,
@@ -128,7 +116,6 @@ class WebSearchTool(BaseTool):
raw_url = html.unescape(link_match.group(1))
title = _strip_html(link_match.group(2)).strip()
# Snippet
snippet_match = re.search(
r'<a[^>]*class="[^"]*result__snippet[^"]*"[^>]*>(.*?)</a>',
block,
@@ -136,7 +123,7 @@ class WebSearchTool(BaseTool):
)
snippet = _strip_html(snippet_match.group(1)).strip() if snippet_match else ""
# DuckDuckGo wraps URLs through a redirect; try to extract the real URL
# DDG wraps URLs in a redirect; extract the real one.
real_url_match = re.search(r"uddg=([^&]+)", raw_url)
if real_url_match:
from urllib.parse import unquote
@@ -152,11 +139,6 @@ class WebSearchTool(BaseTool):
return "\n\n".join(entries)
# ───────────────────────────────────────────────────────────────────────────
# WebFetchTool
# ───────────────────────────────────────────────────────────────────────────
class WebFetchTool(BaseTool):
name = "WebFetch"
description = (
@@ -202,10 +184,7 @@ class WebFetchTool(BaseTool):
is_html = "html" in content_type or resp.text.strip().startswith("<!")
if is_html:
# Prefer trafilatura for article/main-content extraction — strips
# nav, footer, ads, sidebars and returns the primary text. Falls
# back to regex HTML-strip if trafilatura can't extract (rare
# pages: pure apps, login walls, heavily JS-rendered content).
# Prefer trafilatura for main-content extraction; fall back to regex strip on apps/login walls/JS-heavy pages.
text: str | None = None
try:
import trafilatura # type: ignore
backend/apps/agents/web_mcp_server.py
+2 -25
View File
@@ -1,25 +1,5 @@
#!/usr/bin/env python3
"""
Stdio MCP server exposing `WebSearch` and `WebFetch` backed by the
OpenSwarm backend's free DuckDuckGo + trafilatura implementation.
Purpose: the Claude Code CLI's built-in `WebSearch` / `WebFetch` tools
wrap Anthropic's server-side `web_search_20250305` / `web_fetch_20250807`
which require a Claude credential somewhere (Claude subscription on
9Router, openswarm-pro cloud proxy, or direct Anthropic API key). Users
who only connect ChatGPT Plus or Gemini Advanced and don't have any
Claude-backed credential get "No credentials for provider: claude" from
the CLI and either see hallucinated or empty results.
This server is registered by `agent_manager.py` only in that gap case.
When it is registered, the built-in `WebSearch` / `WebFetch` are added
to `disallowed_tools` so the model picks our MCP-prefixed versions.
Proxies tool calls to the backend at /api/web/search and /api/web/fetch
so the DDG / trafilatura logic lives in one place
(`backend/apps/agents/tools/web.py`) and can be evolved without
restarting the MCP subprocess.
"""
"""Stdio MCP server exposing WebSearch/WebFetch; registered only when no Claude credential is available."""
import json
import os
@@ -32,10 +12,7 @@ BACKEND_AUTH = os.environ.get("OPENSWARM_AUTH_TOKEN", "")
SEARCH_URL = f"http://127.0.0.1:{BACKEND_PORT}/api/web/search"
FETCH_URL = f"http://127.0.0.1:{BACKEND_PORT}/api/web/fetch"
# Primary-provider hint set by agent_manager at spawn time. Lets the
# backend pick the corresponding native search tool (googleSearch for
# Gemini, web_search_preview for OpenAI) — so searches come out of the
# budget the user is already paying for.
# Primary-provider hint from agent_manager; backend picks the native search tool (googleSearch/web_search_preview) so searches use the user's existing budget.
PRIMARY_HINT = os.environ.get("OPENSWARM_PRIMARY_API", "") or None
TOOLS = [
backend/apps/agents/ws_manager.py
+79 -65
View File
@@ -9,19 +9,7 @@ logger = logging.getLogger(__name__)
class ConnectionManager:
"""Manages WebSocket connections and bridges HITL approval requests.
Every outbound event flows through the seq log so reconnecting
clients can replay missed events. The send happens *under* the
per-session lock yielded by `seq_log.stamp(...)`, which guarantees
wire order matches seq order even under concurrent broadcasts.
A WS disconnect (`disconnect_session`) ONLY removes the socket
from the connection registry. It does NOT cancel the underlying
agent task. The task lives on `agent_manager.tasks`; only an
explicit `agent:stop`, REST `/close`, natural completion, or
process shutdown ends a run.
"""
"""Manages WebSocket connections and HITL approval bridging; events flow through seq_log so reconnects can replay."""
def __init__(self):
self.connections: dict[str, list[WebSocket]] = {}
@@ -53,19 +41,7 @@ class ConnectionManager:
]
async def send_to_session(self, session_id: str, event: str, data: dict):
"""Broadcast a session event with monotonic sequencing.
The send to every socket happens inside the seq_log lock so a
slow/dead WS doesn't reorder events on the fast ones. If a
single send raises (broken pipe, half-open socket), we log and
continue — the ring buffer still has the event so the client
will replay it on reconnect.
For terminal status events (completed/stopped/error) we also
atomically persist the payload to disk; a client that returns
after a process restart can then resolve the spinner via
`seq_log.load_terminal(...)` instead of being stuck.
"""
"""Broadcast a session event with monotonic sequencing; terminal statuses also persist to disk."""
async with seq_log.stamp(session_id, event, data) as (seq, payload_str):
for ws in list(self.connections.get(session_id, [])):
try:
@@ -77,38 +53,17 @@ class ConnectionManager:
await ws.send_text(payload_str)
except Exception:
logger.debug("send_to_session: global send failed", exc_info=True)
# Persist terminal events under the lock so a concurrent
# `agent:status: running` can't race past and overwrite
# the disk file with a stale state.
# Persist under the lock so a concurrent running status can't race past and overwrite with stale state.
if event == "agent:status" and data.get("status") in TERMINAL_STATUSES:
seq_log.persist_terminal(session_id, payload_str)
async def replay_to(
self, session_id: str, websocket: WebSocket, last_seq: int
) -> dict:
"""Replay buffered events with seq > last_seq to one socket.
Returns a small ack envelope describing what happened so the
caller (the WS handler) can send a `server:resume_ack` frame.
Three cases:
1. `events` non-empty: replay them in order; ack carries
`from_seq`, `to_seq`.
2. No buffer at all (process restarted, session evicted)
but a persisted terminal exists: send it; ack signals
`terminal_only=True`.
3. `last_seq` predates the oldest buffered seq: emit
`agent:gap_detected`; client REST-refreshes the session.
"""
"""Replay buffered events with seq > last_seq; returns ack envelope for the resume handshake."""
oldest, newest, events = seq_log.replay(session_id, last_seq)
# Check for gap FIRST. If the client's last_seq is below the
# buffer's oldest seq, we can't deliver everything they
# missed — silently replaying only the in-buffer tail would
# leave a hole in their state. Tell them to REST-refresh
# instead, even if the tail looks safe to send.
# Treat last_seq=0 as "fresh client" — they want a full
# replay of whatever's in the buffer, not a gap signal.
# Gap-check first: if last_seq predates the buffer, signal REST-refresh; last_seq=0 means fresh client (full replay).
if last_seq > 0 and oldest is not None and last_seq < oldest - 1:
gap_payload = json.dumps({
"event": "agent:gap_detected",
@@ -132,6 +87,23 @@ class ConnectionManager:
}
if events:
# Drop already-resolved approval requests from the replay. The
# ring buffer holds every event we ever stamped, including the
# original `agent:approval_request`. Without this filter, a
# client that reconnects (e.g. after navigating away and back,
# which re-mounts AgentChat with last_seq=0) re-fires every
# past approval as if it were live, but the backing future was
# popped from pending_futures the moment the user answered, so
# the resurrected card is a dead no-op. Lifecycle is simple:
# send_approval_request() inserts into pending_futures BEFORE
# the event is stamped, and resolve_approval()/timeout/cancel
# all pop it; so "in pending_futures" is the authoritative
# is-still-live signal for the request_id. A process restart
# wipes pending_futures, which is correct because
# reconcile_on_startup also marks waiting_approval sessions as
# stopped so there's nothing to answer anyway.
events = self._filter_stale_approvals(events)
events = self._strip_replayed_closes(events)
for s in events:
try:
await websocket.send_text(s)
@@ -145,7 +117,6 @@ class ConnectionManager:
"to_seq": newest,
}
# Nothing in memory. Try a persisted terminal event.
terminal = seq_log.load_terminal(session_id)
if terminal is not None:
try:
@@ -154,20 +125,60 @@ class ConnectionManager:
pass
return {"ok": True, "replayed": 1, "terminal_only": True}
# Nothing missed, nothing to replay. Caller's caught up.
return {
"ok": True,
"replayed": 0,
"current_seq": newest if newest is not None else 0,
}
async def broadcast_global(self, event: str, data: dict):
"""Send a message to all global (dashboard) connections.
def _strip_replayed_closes(self, events: list[str]) -> list[str]:
"""Drop `agent:closed` events from a replay buffer.
Dashboard-scoped events don't go through the per-session seq
log — they're not session-bound and the dashboard WS has its
own resume story (full state refetch on reconnect).
agent:closed is a transition event ("session JUST closed") whose
frontend reducer (closeSessionFromWs) destructively deletes the
session from state.sessions. Replaying it on a fresh client (e.g.
a user who just clicked the closed chat in history) deletes the
session they're trying to open. The current closed state is
already conveyed by the REST hydrate (status=stopped, closed_at
set) and by the latest agent:status event in the replay, so
suppressing the transition replay is non-lossy.
"""
out: list[str] = []
for payload_str in events:
try:
parsed = json.loads(payload_str)
except (ValueError, TypeError):
out.append(payload_str)
continue
if parsed.get("event") == "agent:closed":
continue
out.append(payload_str)
return out
def _filter_stale_approvals(self, events: list[str]) -> list[str]:
"""Return events minus any `agent:approval_request` whose request_id
is no longer in pending_futures. JSON parse is per-event but replay
only runs on (re)connect, so it isn't a hot path.
"""
alive = self.pending_futures
out: list[str] = []
for payload_str in events:
try:
parsed = json.loads(payload_str)
except (ValueError, TypeError):
out.append(payload_str)
continue
if parsed.get("event") != "agent:approval_request":
out.append(payload_str)
continue
data = parsed.get("data") or {}
request_id = data.get("request_id")
if request_id and request_id in alive:
out.append(payload_str)
return out
async def broadcast_global(self, event: str, data: dict):
"""Send to all dashboard connections; bypasses seq_log (dashboard resumes via full state refetch)."""
payload = json.dumps({"event": event, "data": data})
for ws in list(self.global_connections):
try:
@@ -178,21 +189,24 @@ class ConnectionManager:
async def send_approval_request(
self, session_id: str, request_id: str, tool_name: str, tool_input: dict,
timeout: float = 600.0,
sensitive_pattern: str | None = None,
sensitive_label: str | None = None,
sensitive_why: str | None = None,
) -> dict:
"""Send an approval request and wait for the user's response.
Returns the approval decision dict. Times out after `timeout`
seconds (default 10 minutes) so a forgotten request doesn't
permanently park the agent.
"""
"""Send an approval request and wait for the user's decision; 10-minute timeout prevents permanent park."""
future = asyncio.get_event_loop().create_future()
self.pending_futures[request_id] = future
await self.send_to_session(session_id, "agent:approval_request", {
payload: dict = {
"request_id": request_id,
"tool_name": tool_name,
"tool_input": tool_input,
})
}
if sensitive_pattern:
payload["sensitive_pattern"] = sensitive_pattern
payload["sensitive_label"] = sensitive_label
payload["sensitive_why"] = sensitive_why
await self.send_to_session(session_id, "agent:approval_request", payload)
try:
result = await asyncio.wait_for(future, timeout=timeout)
backend/apps/auth/router.py
+9 -9
View File
@@ -15,7 +15,7 @@ POST /api/auth/signout
identity fields. Brings the user back to the sign-in gate.
POST /api/auth/identity-status {install_id?}
Local proxy to cloud /api/me/identity-status — drives the gate's
Local proxy to cloud /api/me/identity-status; drives the gate's
soft-vs-hard decision. Wraps it in our local backend so the renderer
doesn't need to know the cloud URL.
"""
@@ -88,8 +88,8 @@ async def signin_activate(body: SigninActivateRequest):
The bearer-handoff page (cloud lib/authMint.ts → bearerHandoffPage())
POSTs to this endpoint after a Google OAuth or magic-link flow. We
re-validate the bearer with the cloud — never just trust whatever
arrives at the localhost endpoint — then write user_id + email +
re-validate the bearer with the cloud; never just trust whatever
arrives at the localhost endpoint; then write user_id + email +
signin_method to settings so the renderer can dismiss the gate.
"""
if not body.token or len(body.token) < 16:
@@ -134,7 +134,7 @@ async def signin_activate(body: SigninActivateRequest):
# If the user happens to be a paying customer too (Stripe + sign-in
# share a user row by email), surface plan/expires so the chat picker
# exposes Pro models. Free-tier signups land here with plan="free"
# and expires=null — connection_mode stays own_key.
# and expires=null; connection_mode stays own_key.
if isinstance(plan, str) and plan != "free":
settings_obj.connection_mode = "openswarm-pro"
settings_obj.openswarm_bearer_token = body.token
@@ -145,7 +145,7 @@ async def signin_activate(body: SigninActivateRequest):
else:
# Free-tier: still store the bearer so future API calls can identify
# the user (used by /api/me/profile, /api/auth/signout). Do NOT flip
# connection_mode — that's reserved for paid plans only so chat
# connection_mode; that's reserved for paid plans only so chat
# routing keeps using own_key/BYO.
settings_obj.openswarm_bearer_token = body.token
settings_obj.openswarm_proxy_url = proxy
@@ -199,7 +199,7 @@ async def signout():
# the previous identity's Claude account; resuming against the new
# bearer would 404 or 401 because the new account has no record
# of that thread. Wiping it forces the SDK to start a fresh thread
# on next send (transcript replay still works — only the SDK's
# on next send (transcript replay still works; only the SDK's
# server-side resume cache is reset).
# Best-effort: failures here shouldn't block the sign-out itself.
try:
@@ -266,10 +266,10 @@ async def identity_status():
"hard_gate": False,
}
# Not signed in — defer to cloud for install-age + grace-window math.
# Not signed in; defer to cloud for install-age + grace-window math.
install_id = getattr(settings_obj, "installation_id", None)
if not install_id:
# No install_id yet (very fresh install before first sync) — hard gate.
# No install_id yet (very fresh install before first sync); hard gate.
return {"authed": False, "hard_gate": True, "install_age_days": 0, "deadline_ts": None}
proxy = _proxy_url()
@@ -290,6 +290,6 @@ async def identity_status():
except httpx.HTTPError as e:
logger.debug("identity-status cloud fetch failed: %s", e)
# Cloud unreachable — fail open with soft gate so a flaky network
# Cloud unreachable; fail open with soft gate so a flaky network
# doesn't lock the user out. Renderer will retry on next mount.
return {"authed": False, "hard_gate": False, "install_age_days": 0, "deadline_ts": None}
backend/apps/dashboards/dashboards.py
+3 -3
View File
@@ -60,7 +60,7 @@ def _migrate_if_needed():
if existing:
return
logger.info("No dashboards found — running one-time migration")
logger.info("No dashboards found; running one-time migration")
layout = DashboardLayout()
if os.path.exists(OLD_LAYOUT_FILE):
@@ -203,7 +203,7 @@ async def seed_orchestration_demo(dashboard_id: str):
agent for the user to attach to a new orchestrator. We seed a single
completed-looking session that pretends to have done research on
OpenSwarm, with messages mentioning what it found. The user then
drags it into a new agent and asks for a PDF report — which
drags it into a new agent and asks for a PDF report; which
delegates back to this seeded agent.
"""
_load(dashboard_id) # validate dashboard exists
@@ -255,7 +255,7 @@ async def seed_orchestration_demo(dashboard_id: str):
"- A Hono cloud service handles auth, billing, and account pooling.\n"
"- Built-in browser cards let agents drive web pages directly.\n"
"- Skills and Apps let users teach the system new capabilities.\n\n"
"Ready when you are — let me know what you'd like to do with this."
"Ready when you are; let me know what you'd like to do with this."
),
"timestamp": now.isoformat(),
"branch_id": "main",
backend/apps/dashboards/models.py
+29 -3
View File
@@ -36,9 +36,7 @@ class BrowserCardPosition(BaseModel):
y: float = 0
width: float = 1280
height: float = 800
# Agent session id that spawned this browser, or None for user-created.
# Used by the frontend to auto-remove the browser when its owner agent
# reaches a terminal completed/error state.
# Spawning agent session id; frontend auto-removes the browser when this agent reaches a terminal state.
spawned_by: Optional[str] = None
@@ -52,11 +50,39 @@ class NotePosition(BaseModel):
color: str = "yellow"
class WorkflowCardPosition(BaseModel):
workflow_id: str
x: float = 0
y: float = 0
width: float = 440
height: float = 520
source_session_id: Optional[str] = None
class WorkflowsHubPosition(BaseModel):
x: float = 0
y: float = 0
width: float = 1200
height: float = 640
class ConfigurePanelPosition(BaseModel):
"""Floating Action-Library panel tethered to a workflow card."""
workflow_id: str
x: float = 0
y: float = 0
width: float = 580
height: float = 600
class DashboardLayout(BaseModel):
cards: dict[str, CardPosition] = Field(default_factory=dict)
view_cards: dict[str, ViewCardPosition] = Field(default_factory=dict)
browser_cards: dict[str, BrowserCardPosition] = Field(default_factory=dict)
workflow_cards: dict[str, WorkflowCardPosition] = Field(default_factory=dict)
workflows_hub: Optional[WorkflowsHubPosition] = None
notes: dict[str, NotePosition] = Field(default_factory=dict)
configure_panels: dict[str, ConfigurePanelPosition] = Field(default_factory=dict)
expanded_session_ids: list[str] = Field(default_factory=list)
backend/apps/discord_mcp_shim/server.py
+2 -2
View File
@@ -228,7 +228,7 @@ def _call(
locally so the user gets a clear error instead of an opaque 401.
"""
if not INSTALL_ID:
return 0, "OPENSWARM_INSTALL_ID env var not set — cannot call Discord proxy"
return 0, "OPENSWARM_INSTALL_ID env var not set; cannot call Discord proxy"
url = f"{PROXY_BASE}/api/discord{path}"
if query:
@@ -282,7 +282,7 @@ def _check_guild(guild_id: str) -> str | None:
The set is sourced from OPENSWARM_DISCORD_GUILD_IDS env var (CSV) which
tools_lib.py populates from the tool's oauth_tokens.guilds. If the env
var is empty (no guild authorization yet), allow all — agent shouldn't
var is empty (no guild authorization yet), allow all; agent shouldn't
be able to spawn this MCP without an OAuth flow having happened.
"""
if not ALLOWED_GUILDS:
backend/apps/google_workspace_mcp_shim/__init__.py
View File
backend/apps/google_workspace_mcp_shim/run.py
+57
View File
@@ -0,0 +1,57 @@
"""Run google-workspace-mcp's stdio worker with token refresh redirected
through our local proxy instead of directly to oauth2.googleapis.com.
google_workspace_mcp.auth.gauth.get_credentials() hardcodes
token_uri="https://oauth2.googleapis.com/token" and refreshes with
whatever GOOGLE_WORKSPACE_CLIENT_ID/SECRET are in env on every API call.
OAuth runs through a rotation pool in openswarm-cloud, so the
refresh_token is bound to whichever pool slot minted it, not the single
client baked into the DMG. Refresh directly against Google with the
wrong client returns unauthorized_client.
This wrapper monkey-patches gauth.get_credentials before the worker
imports its tool modules, pointing token_uri at GOOGLE_WORKSPACE_TOKEN_URI
(our local proxy at /api/tools/google-oauth-token, which forwards refresh
requests to the cloud's pool-aware /api/oauth/google/refresh).
CLIENT_ID/SECRET become unused placeholders.
"""
import functools
import os
import google_workspace_mcp.auth.gauth as gauth
from google.oauth2.credentials import Credentials
@functools.lru_cache(maxsize=1)
def _patched_get_credentials():
refresh_token = os.environ.get("GOOGLE_WORKSPACE_REFRESH_TOKEN")
if not refresh_token:
raise ValueError("GOOGLE_WORKSPACE_REFRESH_TOKEN env var is required")
return Credentials(
token=None,
refresh_token=refresh_token,
token_uri=os.environ.get(
"GOOGLE_WORKSPACE_TOKEN_URI",
"https://oauth2.googleapis.com/token",
),
client_id=os.environ.get("GOOGLE_WORKSPACE_CLIENT_ID", "openswarm-proxy"),
client_secret=os.environ.get("GOOGLE_WORKSPACE_CLIENT_SECRET", "openswarm-proxy"),
)
gauth.get_credentials = _patched_get_credentials
from google_workspace_mcp import __main__ as _gw_main # noqa: E402,F401
from google_workspace_mcp.app import mcp # noqa: E402
if __name__ == "__main__":
# Upstream google_workspace_mcp.__main__.main() wraps a synchronous
# mcp.run() in asyncio.run() which throws "a coroutine was expected,
# got None" against current FastMCP. Skip it and invoke FastMCP's
# stdio loop directly. The `_gw_main` import above is what actually
# registers every tool/prompt/resource module against the shared
# `mcp` instance via its top-level imports.
mcp.run("stdio")
backend/apps/health/health.py
-6
View File
@@ -13,16 +13,10 @@ async def health_lifespan():
health = SubApp("health", health_lifespan)
######################################
# Health Check Endpoints #
######################################
@health.router.get("/check")
@typechecked
async def check() -> PlainTextResponse:
debug("Health check successful")
# Use PlainTextResponse instead of JSONResponse for AWS ALB compatibility
# ALB health checks can be sensitive to JSON responses and Content-Length headers
return PlainTextResponse(
content="OK",
status_code=status.HTTP_200_OK,
backend/apps/modes/models.py
+18 -18
View File
@@ -55,9 +55,9 @@ BUILTIN_MODES: list[Mode] = [
Mode(
id="ask",
name="Ask",
description="Read-only conversation. Browse the codebase, search the web, and discuss ideas — but no edits, shells, or file writes.",
description="Read-only conversation. Browse the codebase, search the web, and discuss ideas; but no edits, shells, or file writes.",
system_prompt=(
"You are in Ask mode — a read-only assistant. Keep responses "
"You are in Ask mode; a read-only assistant. Keep responses "
"natural and conversational. You CAN read files, search the "
"codebase, and search/fetch the web. You CANNOT edit files, run "
"shell commands, or otherwise modify anything; if the user asks "
@@ -88,21 +88,21 @@ BUILTIN_MODES: list[Mode] = [
name="App Builder",
description="Create and iterate on reusable App artifacts.",
system_prompt=(
"You are an App Builder — an AI assistant that creates self-contained "
"You are an App Builder; an AI assistant that creates self-contained "
"web apps rendered in an iframe preview.\n\n"
"Your working directory is a dedicated workspace folder pre-seeded with "
"template files. Read the existing files before making changes.\n\n"
"## Critical rules\n\n"
"- The entry point MUST be named `index.html`. Never rename it or create "
"a different HTML file as the main entry point.\n"
"- Write files immediately when you have code ready — the user sees a "
"- Write files immediately when you have code ready; the user sees a "
"live preview that auto-refreshes from these files.\n"
"- Always write the complete file content on first creation (do not use "
"Edit for partial patches on new files).\n"
"- For complex apps, split code into separate files (JS, CSS, etc.) "
"and reference them from index.html with relative paths.\n"
"- Always update meta.json with a short name and one-sentence description.\n"
"- Build beautiful, polished UIs with modern design — dark themes, smooth "
"- Build beautiful, polished UIs with modern design; dark themes, smooth "
"transitions, proper spacing, and responsive layouts.\n\n"
"Read the SKILL.md reference in your workspace for the full technical "
"specification of the App platform (available globals, file conventions, "
@@ -120,17 +120,17 @@ BUILTIN_MODES: list[Mode] = [
name="Skill Builder",
description="Create and iterate on skills using AI-assisted vibe coding.",
system_prompt=(
"You are a Skill Builder — an AI assistant that helps users create, "
"You are a Skill Builder; an AI assistant that helps users create, "
"refine, and iterate on Claude skills (SKILL.md files).\n\n"
"## How Skills Work\n\n"
"A skill is a Markdown file that teaches Claude how to perform a specific task. "
"Skills have YAML frontmatter with `name` and `description` fields, followed by "
"the skill body in Markdown. The description is the primary triggering mechanism — "
"the skill body in Markdown. The description is the primary triggering mechanism; "
"it tells Claude when to use the skill.\n\n"
"## Your Working Directory\n\n"
"Your working directory is a dedicated workspace folder for this skill. "
"Write your output directly to these files using the Write tool:\n\n"
"1. **SKILL.md** — The complete skill file with YAML frontmatter and Markdown body. "
"1. **SKILL.md**; The complete skill file with YAML frontmatter and Markdown body. "
"Example frontmatter:\n"
" ```\n"
" ---\n"
@@ -138,34 +138,34 @@ BUILTIN_MODES: list[Mode] = [
" description: When to trigger and what this skill does.\n"
" ---\n"
" ```\n\n"
"2. **meta.json** — Metadata for the skill builder UI. Always write this file. Example:\n"
"2. **meta.json**; Metadata for the skill builder UI. Always write this file. Example:\n"
' {"name":"My Skill","description":"A short description","command":"my-skill"}\n\n'
"Write these files immediately when you have content ready. The user can see "
"a live preview that auto-refreshes from these files. Always write the "
"complete file content (do not use Edit for partial patches on first creation).\n\n"
"## Skill Creation Process\n\n"
"1. **Understand intent** — Ask what the skill should do, when it should trigger, "
"1. **Understand intent**; Ask what the skill should do, when it should trigger, "
"and what the expected output format is.\n"
"2. **Draft the skill** — Write a SKILL.md with clear instructions, examples, "
"2. **Draft the skill**; Write a SKILL.md with clear instructions, examples, "
"and good progressive disclosure.\n"
"3. **Iterate** — Refine based on user feedback. Update the files each time.\n\n"
"3. **Iterate**; Refine based on user feedback. Update the files each time.\n\n"
"## Skill Writing Best Practices\n\n"
"- Keep SKILL.md under 500 lines; use bundled reference files for large content.\n"
"- The `description` frontmatter is the primary trigger. Make it slightly \"pushy\" — "
"- The `description` frontmatter is the primary trigger. Make it slightly \"pushy\"; "
"include both what the skill does AND specific contexts for when to use it.\n"
"- Use imperative form in instructions.\n"
"- Include examples with input/output pairs when helpful.\n"
"- Define output formats explicitly with templates.\n"
"- Use theory of mind — explain *why* things matter rather than just MUST directives.\n"
"- Use theory of mind; explain *why* things matter rather than just MUST directives.\n"
"- Think about edge cases, error handling, and progressive disclosure.\n\n"
"## Skill Anatomy\n\n"
"```\n"
"skill-name/\n"
"├── SKILL.md (required) — YAML frontmatter + Markdown instructions\n"
"├── SKILL.md (required); YAML frontmatter + Markdown instructions\n"
"└── Bundled Resources (optional)\n"
" ├── scripts/ — Executable code for repetitive tasks\n"
" ├── references/ — Docs loaded into context as needed\n"
" └── assets/ — Files used in output\n"
" ├── scripts/ ; Executable code for repetitive tasks\n"
" ├── references/; Docs loaded into context as needed\n"
" └── assets/ ; Files used in output\n"
"```\n\n"
"Be collaborative and flexible. If the user wants to \"just vibe\", skip the formal "
"process and iterate freely. Always write updated files so the preview stays current."
backend/apps/modes/modes.py
+1 -4
View File
@@ -14,10 +14,7 @@ from backend.config.paths import MODES_DIR as DATA_DIR
@asynccontextmanager
async def modes_lifespan():
os.makedirs(DATA_DIR, exist_ok=True)
# One-time migration: Chat was merged into Ask. Remove a stale built-in
# chat.json if it still has its is_builtin=True signature so users don't
# see two near-identical modes in the picker. Leave alone if a user has
# diverged it (we don't want to wipe customizations).
# Migration: Chat merged into Ask; drop a stale built-in chat.json but leave customized copies alone.
chat_path = os.path.join(DATA_DIR, "chat.json")
if os.path.exists(chat_path):
try:
backend/apps/nine_router.py
+52 -141
View File
@@ -30,13 +30,13 @@ NINE_ROUTER_V1 = f"{NINE_ROUTER_URL}/v1"
# cross-provider WebSearch: the CLI's WebSearch call from Codex/Gemini
# primaries used to route cleanly through 9Router's translator and hit
# Anthropic's server-side web_search (returning real results), but later
# translator changes broke that path — non-Claude primaries now see
# translator changes broke that path; non-Claude primaries now see
# "claude-haiku-4-5-20251001 unavailable" or hallucinated output.
# Pinning to 0.3.60 restores v1.0.25 behavior.
#
# Note: 0.3.60-0.4.20 ALL emit `max_tokens` (not max_completion_tokens)
# when translating Anthropic→OpenAI, which OpenAI's GPT-5 family rejects.
# The fix lives in our /api/openai-passthrough proxy — see openai_passthrough.py
# The fix lives in our /api/openai-passthrough proxy; see openai_passthrough.py
# and sync_openai_api_key for how the translation lane is rerouted via an
# `openai-compatible` provider-node that honors `baseUrl`.
NINE_ROUTER_NPM_VERSION = "0.3.60"
@@ -75,16 +75,12 @@ def _find_9router_dir() -> str | None:
_is_packaged = os.environ.get("OPENSWARM_PACKAGED") == "1"
if _is_packaged:
# Packaged Electron app — router is in extraResources
import sys
# In packaged mode, backend is at <resources>/backend/
# So router is at <resources>/router/
_resources = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
_candidate = os.path.join(_resources, "router")
if os.path.isdir(_candidate):
return _candidate
else:
# Dev mode — router is at project root
_backend_dir = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
_project_root = os.path.dirname(_backend_dir)
_candidate = os.path.join(_project_root, "router")
@@ -103,7 +99,7 @@ def _gpt5_patch_path() -> str | None:
every gpt-5* own-key session 400's because OpenAI rejects the legacy
field name and 9router (every version including 0.4.20) emits it.
Returns None if the file is missing — `subprocess.Popen` would fail
Returns None if the file is missing; `subprocess.Popen` would fail
on `node --require <missing-path>`, so the caller drops the flag and
spawns 9router unpatched (failure mode = identical to pre-patch
baseline; GPT-5 still 400's but everything else works).
@@ -121,14 +117,14 @@ def _find_node() -> str | None:
"""Find a Node.js binary (works in both dev and packaged mode).
Priority order:
1. OPENSWARM_NODE_PATH — set by electron/main.js when a real Node
1. OPENSWARM_NODE_PATH; set by electron/main.js when a real Node
binary is bundled in extraResources. Always preferred on user
machines because it (a) avoids the bouncing "exec" Dock icon
that ELECTRON_RUN_AS_NODE produces on fresh Macs and (b) starts
in ~50ms vs Electron-as-Node's 5–15s cold-start, shrinking the
in ~50ms vs Electron-as-Node's 5, 15s cold-start, shrinking the
splash window the user stares at.
2. System `node` on PATH — dev convenience.
3. ELECTRON_RUN_AS_NODE fallback — last resort. Only hits this on
2. System `node` on PATH; dev convenience.
3. ELECTRON_RUN_AS_NODE fallback; last resort. Only hits this on
packaged builds that for some reason shipped without the bundled
node payload.
"""
@@ -163,7 +159,7 @@ def _ensure_router_cached() -> str | None:
"""Ensure the npm 9router package is installed in the dev cache.
Returns the absolute path to `app/server.js` on success, or None if
npm isn't available or the install fails. Idempotent — returns
npm isn't available or the install fails. Idempotent; returns
immediately when the server file already exists.
Running `node app/server.js` directly (instead of `npx 9router`)
@@ -178,7 +174,7 @@ def _ensure_router_cached() -> str | None:
npm = shutil.which("npm")
if not npm:
logger.warning("npm not found — install Node.js to auto-start 9Router in dev.")
logger.warning("npm not found; install Node.js to auto-start 9Router in dev.")
return None
try:
@@ -242,7 +238,7 @@ async def ensure_running():
_9router_dir = _find_9router_dir()
if _is_packaged and _9router_dir:
# Packaged mode — run the pre-built standalone server staged at
# Packaged mode; run the pre-built standalone server staged at
# <resources>/router/server.js by scripts/fetch-router.sh at build time.
standalone_server = os.path.join(_9router_dir, "server.js")
if not os.path.exists(standalone_server):
@@ -253,7 +249,7 @@ async def ensure_running():
node = _find_node()
if not node:
logger.warning("Node.js not found — cannot start 9Router in packaged mode.")
logger.warning("Node.js not found; cannot start 9Router in packaged mode.")
return
logger.info("Starting 9Router (production) on port %d...", NINE_ROUTER_PORT)
@@ -268,7 +264,7 @@ async def ensure_running():
env["ELECTRON_RUN_AS_NODE"] = "1"
else:
# Dev mode — install the pinned 9router npm package into a local
# Dev mode; install the pinned 9router npm package into a local
# cache the first time run.sh boots, then spawn `node app/server.js`
# directly on subsequent launches. Bypassing the package's cli.js
# avoids its menu-bar tray icon (which users confusingly quit,
@@ -280,7 +276,7 @@ async def ensure_running():
node = _find_node()
if not node:
logger.warning("Node.js not found — cannot start 9Router in dev mode.")
logger.warning("Node.js not found; cannot start 9Router in dev mode.")
return
logger.info(
@@ -298,7 +294,7 @@ async def ensure_running():
# By default, 9Router's stdout/stderr go to /dev/null (Next.js dev mode
# is extremely chatty and floods the openswarm console otherwise). When
# debugging is needed, set OPENSWARM_DEBUG_9ROUTER=1 in the environment
# before launching the backend — output will then be appended to
# before launching the backend; output will then be appended to
# backend/data/9router.log line-buffered, which can be `tail -f`'d.
if os.environ.get("OPENSWARM_DEBUG_9ROUTER"):
_log_path = os.path.join(
@@ -323,7 +319,6 @@ async def ensure_running():
env=env,
)
# Wait up to 30 seconds for startup (production standalone is faster)
timeout = 20 if _is_packaged else 30
for _ in range(timeout * 2):
await asyncio.sleep(0.5)
@@ -352,10 +347,6 @@ def stop():
logger.info("9Router stopped")
# ---------------------------------------------------------------------------
# API proxy helpers — call 9Router's API from OpenSwarm
# ---------------------------------------------------------------------------
async def get_usage_stats(period: str = "all") -> dict | None:
"""Get usage statistics from 9Router."""
try:
@@ -378,8 +369,8 @@ async def get_latest_reasoning_tokens(model_hint: str | None = None) -> int | No
in reverse chronological order with full token breakdowns including
`reasoning_tokens` (OpenAI's `completion_tokens_details.reasoning_tokens`)
and `thoughtsTokenCount` (Gemini's). For Anthropic via 9Router this
field will be absent/zero — Anthropic doesn't break out reasoning
tokens in its API response — so callers get None and should fall
field will be absent/zero; Anthropic doesn't break out reasoning
tokens in its API response; so callers get None and should fall
back to the heuristic.
"""
if not is_running():
@@ -393,9 +384,6 @@ async def get_latest_reasoning_tokens(model_hint: str | None = None) -> int | No
if r.status_code != 200:
return None
data = r.json()
# Endpoint returns either {requests: [...]} or {data: [...]} —
# be defensive about the shape since 9Router has rolled out
# multiple variants.
requests = data.get("requests") or data.get("data") or []
for req in requests:
tokens = req.get("tokens") or req.get("usage") or {}
@@ -415,7 +403,7 @@ async def get_latest_reasoning_tokens(model_hint: str | None = None) -> int | No
async def get_providers() -> list[dict]:
"""Get all providers and their connection status from 9Router.
9Router's GET /api/providers returns `{"connections": [...]}` — we
9Router's GET /api/providers returns `{"connections": [...]}`; we
unwrap so callers always see a plain list of connection dicts.
"""
try:
@@ -432,21 +420,11 @@ async def get_providers() -> list[dict]:
return []
# ---------------------------------------------------------------------------
# API-key connection sync (Gemini AI Studio, etc.)
# ---------------------------------------------------------------------------
#
# 9Router supports both OAuth (e.g. gemini-cli) and direct API-key auth
# (provider="gemini", authType="apikey"). The two hit different Google
# quotas — OAuth uses the Code Assist free tier which is aggressively
# rate-limited (429s on Gemini 3 Pro/Flash even for paid-subscription
# users), while an AI Studio API key uses the generativelanguage.googleapis.com
# quota which is independent and far higher.
#
# We expose `google_api_key` in settings; this helper mirrors it into
# 9Router's provider-connections list so the API-key path is preferred
# when a key is set. On removal, we delete the key-based connection so
# 9Router falls back to whatever OAuth connection the user still has.
# API-key auth (provider="gemini", authType="apikey") and OAuth hit different
# Google quotas: OAuth uses the Code Assist free tier (aggressively rate-limited;
# 429s on Gemini 3 Pro/Flash even for paid users), while an AI Studio API key
# uses generativelanguage.googleapis.com (independent and far higher). We mirror
# google_api_key into 9Router so the API-key path is preferred when a key is set.
NINE_ROUTER_KEYED_NAME = "AI Studio (OpenSwarm-managed)"
NINE_ROUTER_OPENAI_KEYED_NAME = "OpenAI (OpenSwarm-managed)"
@@ -533,7 +511,7 @@ async def sync_openai_api_key(api_key: str | None) -> None:
`baseUrl` field on the connection. Only the `openai-compatible-*`
provider-node type honors `baseUrl` (verified statically against
9Router's compiled bundle). So we register our OpenAI lane AS an
openai-compatible node — same upstream protocol, different routing.
openai-compatible node; same upstream protocol, different routing.
Why we route through openai-passthrough at all: OpenAI's GPT-5 family
rejects the legacy `max_tokens` parameter with HTTP 400, but every
@@ -565,7 +543,6 @@ async def _sync_openai_compat_node(api_key: str | None) -> None:
base_url = f"http://127.0.0.1:{port}/api/openai-passthrough/v1"
managed_name = f"OpenAI{NINE_ROUTER_CUSTOM_NAME_SUFFIX}"
# List existing managed nodes — we own the prefix `cp-openai`.
try:
async with httpx.AsyncClient(timeout=5.0) as client:
r = await client.get(f"{NINE_ROUTER_API}/provider-nodes")
@@ -578,7 +555,6 @@ async def _sync_openai_compat_node(api_key: str | None) -> None:
None,
)
# Tear down when api_key is cleared.
if not api_key:
if existing_node:
try:
@@ -623,7 +599,6 @@ async def _sync_openai_compat_node(api_key: str | None) -> None:
logger.warning(f"9Router OpenAI compat node sync failed: {e}")
return
# Connection record carrying the api key, scoped to this provider node.
try:
existing_conn = await _find_keyed_connection(node_id, managed_name)
conn_payload = {
@@ -657,19 +632,10 @@ async def sync_openrouter_api_key(api_key: str | None) -> None:
)
# ---------------------------------------------------------------------------
# Custom OpenAI-compatible providers (Ollama Cloud, Together AI, local Ollama, etc.)
# ---------------------------------------------------------------------------
#
# 9Router supports arbitrary OpenAI-compatible endpoints via "provider nodes"
# (POST /api/provider-nodes with type="openai-compatible"). Each node gets a
# unique provider id like `openai-compatible-chat-<rand>` and a user-defined
# `prefix`. At request time, model_id `<prefix>/<bare_model>` routes to that
# node's baseUrl, with auth from a connection of the node's provider type.
#
# We mirror settings.custom_providers[] into 9Router with prefix `cp-<slug>`,
# letting us address each provider as `cp-<slug>/<model_id>` without colliding
# with the user's primary OpenAI key (different provider type).
# 9Router exposes arbitrary OpenAI-compatible endpoints via "provider nodes"
# (POST /api/provider-nodes, type="openai-compatible"). model_id <prefix>/<model>
# routes to that node's baseUrl. We mirror settings.custom_providers[] with
# prefix `cp-<slug>` so they don't collide with the user's primary OpenAI key.
NINE_ROUTER_CUSTOM_NAME_SUFFIX = " (OpenSwarm-managed)"
@@ -711,19 +677,14 @@ async def sync_custom_providers(providers: list) -> None:
seen_prefixes: set[str] = set()
for cp in providers or []:
# Tolerate both Pydantic instances and plain dicts.
name = getattr(cp, "name", None) or (cp.get("name") if isinstance(cp, dict) else None) or ""
base_url = getattr(cp, "base_url", None) or (cp.get("base_url") if isinstance(cp, dict) else None) or ""
api_key = getattr(cp, "api_key", None) or (cp.get("api_key") if isinstance(cp, dict) else None) or ""
if not name.strip() or not base_url.strip():
continue
# Local OpenAI-compatible servers (LM Studio, Ollama, vLLM, llama.cpp,
# text-generation-webui, etc.) ship with auth disabled by default —
# they ignore the Authorization header entirely. But 9Router still
# creates the connection with `authType: "apikey"` and would send a
# blank Bearer if api_key is "", which some servers reject as a
# malformed header. Substitute a harmless placeholder when blank;
# servers that DO require auth always have api_key set anyway.
# Local OpenAI-compat servers (LM Studio, Ollama, etc.) reject a blank
# Bearer header even with auth disabled. Substitute a placeholder; real
# auth deployments always have api_key set.
api_key = api_key.strip() or "no-auth-required"
slug = _custom_provider_slug(name)
prefix = f"cp-{slug}"
@@ -765,7 +726,6 @@ async def sync_custom_providers(providers: list) -> None:
logger.warning(f"9Router custom node {prefix} sync failed: {e}")
continue
# Ensure a connection exists for this provider node carrying the apikey.
try:
existing_conn = await _find_keyed_connection(node_id, managed_name)
conn_payload = {
@@ -793,8 +753,7 @@ async def sync_custom_providers(providers: list) -> None:
except Exception as e:
logger.warning(f"9Router custom connection {prefix} sync failed: {e}")
# Drop managed nodes that no longer correspond to any settings entry.
# DELETE on a node cascades to its connections.
# Drop managed nodes no longer in settings; DELETE cascades to connections.
for prefix, node in managed_by_prefix.items():
if prefix in seen_prefixes:
continue
@@ -818,13 +777,13 @@ async def sync_openswarm_pro_as_claude(bearer_token: str | None, proxy_url: str
path for openswarm-pro users, so the search fails with
"no credentials for provider: claude". With this sync, 9Router sees
the OpenSwarm-Pro-backed Claude connection and routes the search
call through our cloud — same quota the user's Pro subscription
call through our cloud; same quota the user's Pro subscription
already covers, no extra cost."""
if not is_running():
return
# 9Router's POST /api/providers only accepts direct-API provider ids
# for apikey auth — `claude` is the subscription/IDE id, `anthropic`
# for apikey auth; `claude` is the subscription/IDE id, `anthropic`
# is the direct-API id. Use `anthropic`.
existing = await _find_keyed_connection("anthropic", NINE_ROUTER_CLAUDE_PRO_NAME)
try:
@@ -865,31 +824,15 @@ async def sync_openswarm_pro_as_claude(bearer_token: str | None, proxy_url: str
logger.warning(f"9Router OpenSwarm-Pro Claude sync failed: {e}")
# ---------------------------------------------------------------------------
# Per-provider OAuth redirect URIs
# ---------------------------------------------------------------------------
#
# Each upstream OAuth client is registered with the identity provider against
# a specific redirect URI. Anthropic's Claude Code client is lenient — any
# `http://localhost:*/callback` works — so we can use 9Router's built-in
# callback page at port 20128 for it. OpenAI's Codex client is NOT: it's
# registered with `http://localhost:1455/auth/callback` and OpenAI rejects
# any other redirect_uri with `unknown_error` at the auth page. Google's
# Gemini CLI client accepts arbitrary localhost URIs so we keep 20128 there.
#
# For Codex specifically we spawn a one-shot HTTP listener on port 1455
# below that serves a callback page mirroring 9Router's callback page —
# postMessage to window.opener, BroadcastChannel fan-out, then close. This
# lets the frontend reuse its existing Claude/Anthropic flow unchanged
# (window.open popup + postMessage handler in Settings.tsx).
# OpenAI's Codex OAuth client is registered with a fixed redirect URI
# `http://localhost:1455/auth/callback` and rejects any other with `unknown_error`.
# Anthropic and Google's clients accept arbitrary localhost callbacks (we use
# 9Router's 20128 callback page). For Codex we spawn a one-shot listener on
# 1455 that serves the same postMessage/BroadcastChannel/localStorage relay so
# the frontend's existing popup + msgHandler flow works unchanged.
_CODEX_CALLBACK_PORT = 1455
_CODEX_CALLBACK_PATH = "/auth/callback"
# Minimal callback page inlined as bytes. Mirrors 9router/src/app/callback/
# page.js:27-55 — posts the OAuth data to window.opener via postMessage,
# BroadcastChannel, and localStorage so whatever detection path the caller
# is using will fire. Served to the Electron popup that OAuth redirects to.
_CODEX_CALLBACK_HTML = b"""<!DOCTYPE html>
<html><head><meta charset="utf-8"><title>Authorization Complete</title>
<style>body{font-family:-apple-system,system-ui,sans-serif;background:#111;color:#eee;
@@ -930,7 +873,7 @@ async def _start_codex_callback_listener(timeout: float = 300.0) -> asyncio.base
Serves GET /auth/callback with _CODEX_CALLBACK_HTML. After serving the
callback (or after `timeout` seconds with no callback) the listener
closes itself in a background task. Safe to call even if 1455 is busy —
closes itself in a background task. Safe to call even if 1455 is busy ,
logs the collision and returns None so start_oauth can still proceed and
surface whatever error OpenAI returns.
@@ -940,7 +883,7 @@ async def _start_codex_callback_listener(timeout: float = 300.0) -> asyncio.base
stuck on "Connecting…" until the 30s timeout fires. Exchanging here
(the same pattern backend/main.py uses for the Gemini callback) makes
the connection land in 9Router's DB regardless of whether the UI's
postMessage listener ever gets notified — the Settings / OnboardingModal
postMessage listener ever gets notified; the Settings / OnboardingModal
status pollers then pick it up within a couple seconds.
"""
@@ -951,7 +894,6 @@ async def _start_codex_callback_listener(timeout: float = 300.0) -> asyncio.base
# Read the request line ("GET /auth/callback?... HTTP/1.1\r\n")
raw_request_line = await asyncio.wait_for(reader.readline(), timeout=5.0)
request_line = raw_request_line.decode("latin-1", errors="replace").strip()
# Drain headers so the browser's request is fully consumed
while True:
line = await asyncio.wait_for(reader.readline(), timeout=5.0)
if not line or line in (b"\r\n", b"\n"):
@@ -1024,7 +966,6 @@ async def _start_codex_callback_listener(timeout: float = 300.0) -> asyncio.base
await writer.drain()
callback_served.set()
else:
# Unrelated request (favicon, preflight) — 404 and move on
writer.write(
b"HTTP/1.1 404 Not Found\r\n"
b"Content-Length: 0\r\n"
@@ -1043,7 +984,7 @@ async def _start_codex_callback_listener(timeout: float = 300.0) -> asyncio.base
try:
server = await asyncio.start_server(_handle, "127.0.0.1", _CODEX_CALLBACK_PORT)
except OSError as e:
# Port already in use — probably another Codex connect attempt still
# Port already in use; probably another Codex connect attempt still
# running, or an actual Codex CLI process holding 1455. Log and bail.
logger.warning(
f"Could not start Codex callback listener on port {_CODEX_CALLBACK_PORT}: {e}. "
@@ -1074,38 +1015,15 @@ async def _start_codex_callback_listener(timeout: float = 300.0) -> asyncio.base
return server
# Providers that cannot use the in-Electron `window.open` popup flow and
# must be opened in the user's system browser instead.
#
# Google enforces an "Embedded WebView Restrictions" policy on its OAuth
# consent pages that uses JS-based fingerprinting, not just user-agent
# sniffing. We tried defeating it with a combination of Chrome UA spoof +
# sandboxed webPreferences + fresh session partition + a preload script
# that patches navigator.webdriver/plugins/mimeTypes/languages/chrome and
# overrides navigator.permissions.query — it was still rejected. Google's
# detection is a moving target and actively adversarial. The supported
# workaround (and what Google recommends for Desktop app OAuth) is to run
# the flow in the user's real browser via shell.openExternal.
#
# When a provider is in this set the frontend calls
# window.openswarm.openExternal (shell.openExternal) instead of
# window.open, and the callback lands on OpenSwarm's own
# /api/subscriptions/callback endpoint (backend/main.py:138) which
# exchanges the code and serves a "Connected!" page. Detection on the
# OpenSwarm side happens via the existing status poller on the
# Settings page.
# Providers that hand off to the user's default browser instead of using
# our embedded Electron popup:
# - gemini-cli, antigravity: Google blocks embedded browsers wholesale
# ("Your browser is not supported anymore") — no UA spoof defeats it.
# - codex: OpenAI's auth.openai.com renders blank inside our popup on
# some users' machines (likely a mix of newer embed detection and
# regional access checks) and the system browser surfaces the real
# error rather than a silent blank window. Also what RFC 8252 mandates
# for native-app OAuth, so this is the correct long-term shape anyway.
# Codex's callback URL is special-cased below to stay on localhost:1455
# (OpenAI's hardcoded redirect URI) — the listener catches the system
# browser's redirect just like it caught the popup's.
# Providers whose OAuth flow MUST run in the user's real browser via
# shell.openExternal, not the in-Electron window.open popup:
# - gemini-cli, antigravity: Google's Embedded WebView Restrictions policy uses
# JS-fingerprint detection that no UA spoof defeats. RFC 8252 and Google's
# own Desktop-app OAuth guidance both prescribe the system browser.
# - codex: auth.openai.com renders blank in our popup on some machines (newer
# embed detection + regional checks); system browser surfaces the real error.
# The callback for gemini-cli/antigravity lands on /api/subscriptions/callback
# and runs the exchange server-side; codex uses its fixed 1455 listener.
_EXTERNAL_BROWSER_PROVIDERS: set[str] = {"gemini-cli", "antigravity", "codex"}
@@ -1133,7 +1051,7 @@ def _callback_uri_for_provider(provider: str) -> str:
Most providers accept 9Router's built-in callback page at port 20128.
Two special cases:
- Codex/OpenAI's OAuth client is bound to a fixed
http://localhost:1455/auth/callback URI — handled by
http://localhost:1455/auth/callback URI; handled by
_start_codex_callback_listener above.
- Gemini/Google's OAuth consent page rejects embedded browsers, so we
route the callback through OpenSwarm's backend endpoint at
@@ -1155,7 +1073,6 @@ async def start_oauth(provider: str) -> dict:
For authorization_code providers (claude, codex, gemini-cli): returns {authUrl, codeVerifier, state}
"""
async with httpx.AsyncClient(timeout=15.0) as client:
# Try device-code flow first
try:
r = await client.get(f"{NINE_ROUTER_API}/oauth/{provider}/device-code")
if r.status_code == 200:
@@ -1171,12 +1088,6 @@ async def start_oauth(provider: str) -> dict:
except Exception:
pass
# Authorization code flow. Most providers accept 9Router's own
# callback page at port 20128, but Codex's OAuth client is bound
# to a fixed http://localhost:1455/auth/callback URI — spawn an
# in-process listener on that port before returning the auth URL,
# so the popup can redirect there after login and relay the code
# back to the frontend via postMessage (same flow as Claude).
callback_url = _callback_uri_for_provider(provider)
if provider == "codex":
await _start_codex_callback_listener()
backend/apps/outputs/executor.py
+10 -10
View File
@@ -15,7 +15,7 @@ TIMEOUT_SECONDS = 30
# can find ways around this (e.g. string-encoded imports via tricks the AST
# validator can't see), but the allowlist kills the easy paths cheaply and
# pairs with cwd=tempdir + minimal env so the blast radius is small even if
# a payload slips past. Keep this list to "data shaping" libraries — no I/O,
# a payload slips past. Keep this list to "data shaping" libraries; no I/O,
# no networking, no subprocess.
_ALLOWED_MODULES = frozenset({
"json", "math", "re", "datetime", "collections", "itertools",
@@ -43,9 +43,9 @@ def get_code_warnings(code: str) -> list[str]:
"""Return human-readable warnings for AST-visible risks, without raising.
Used by `/api/outputs/execute` to surface risks to the user in the run
dialog before executing — so a legit Output that needs `pandas` doesn't
dialog before executing; so a legit Output that needs `pandas` doesn't
silently 500 with "import not allowed," it gets a "this Output uses
unsafe imports — review and click Run Anyway" affordance.
unsafe imports; review and click Run Anyway" affordance.
Returns [] for code that's fully inside the allowlist. A syntax error
is reported as a single warning rather than raised so the dialog can
@@ -97,7 +97,7 @@ def _validate_code_safety(code: str) -> None:
# Env vars we always scrub from the subprocess, regardless of strict-vs-force.
# These are the keys an attacker would actually want — install token, provider
# These are the keys an attacker would actually want; install token, provider
# API keys, cloud credentials. Everything else is local-machine convenience.
_SCRUBBED_ENV_KEYS = frozenset({
"OPENSWARM_AUTH_TOKEN",
@@ -120,13 +120,13 @@ def _minimal_env(force: bool = False) -> dict:
"""Build the env for the executor subprocess.
Strict mode (force=False): only language essentials. AST-validated code
is data-shaping only — `import os` and `open()` are blocked, so the
is data-shaping only; `import os` and `open()` are blocked, so the
subprocess can't read env vars or expand `~` anyway. Minimal env is
correct here.
Force mode (force=True): user has explicitly approved unsafe imports
via the HITL preview. They expect the code to behave like a normal
Python process — read HOME, find files, etc. Inherit the real env
Python process; read HOME, find files, etc. Inherit the real env
minus credentials, so an `open(os.path.expanduser("~/data.csv"))`
actually works instead of silently misbehaving.
@@ -166,7 +166,7 @@ async def execute_backend_code(
result to a global ``result`` dict. User print() calls are captured
separately from the result via an in-process StringIO redirect.
Security boundaries (defense in depth — none alone is sufficient):
Security boundaries (defense in depth; none alone is sufficient):
1. AST allowlist on imports + blocked-builtin call list.
2. Subprocess cwd = fresh temp dir (not the OpenSwarm process cwd).
3. Subprocess env strips PATH, all *_TOKEN / *_API_KEY inheritance.
@@ -174,9 +174,9 @@ async def execute_backend_code(
to catch AST-bypass tricks (e.g. metaclass shenanigans).
5. 30s wall-clock timeout, killed on overrun.
`skip_validation=True` bypasses #1 — intended ONLY for callers that
`skip_validation=True` bypasses #1; intended ONLY for callers that
have already surfaced the warnings to a user and gotten explicit
consent (the `/api/outputs/execute` HITL flow). #2–#5 always run.
consent (the `/api/outputs/execute` HITL flow). #2, #5 always run.
"""
if not skip_validation:
@@ -186,7 +186,7 @@ async def execute_backend_code(
"import json, sys, io, builtins\n"
# Defense-in-depth: scrub dangerous attrs off `builtins` so
# attribute-style accesses (metaclass.__subclasses__ chains) can't
# reach them. NOTE: __import__ is deliberately NOT scrubbed —
# reach them. NOTE: __import__ is deliberately NOT scrubbed ,
# Python's `import` statement bytecode reads `__import__` from
# builtins, so removing it makes EVERY import (including allowlisted
# ones like `import math`) fail with "ImportError: __import__ not
backend/apps/outputs/models.py
+1 -1
View File
@@ -127,7 +127,7 @@ class OutputExecute(BaseModel):
# running if the backend code touches anything outside the safe
# data-shaping allowlist. The UI shows those warnings to the user and
# re-submits with force=True after they click "Run Anyway." This is
# a UX gate, not a security one — anyone holding the auth token can
# a UX gate, not a security one; anyone holding the auth token can
# set force=True; the value is providing the user explicit visibility
# of what's about to execute.
force: bool = False
backend/apps/outputs/outputs.py
+152 -16
View File
@@ -129,7 +129,7 @@ def _inject_token_into_relative_urls(html: str, token: str) -> str:
relative `<link href="styles.css">` / `<script src="x.js">`, so without
this rewrite the sub-resource fetch lands at the auth middleware with no
credentials and gets a 401. Idempotent: skips URLs that already carry a
`token=` param. Skips absolute URLs (CDN, data:, etc.) — see prefix list.
`token=` param. Skips absolute URLs (CDN, data:, etc.); see prefix list.
"""
if not token:
return html
@@ -169,7 +169,20 @@ def _decode_data_param(d: str) -> tuple[str, str]:
async def outputs_lifespan():
os.makedirs(DATA_DIR, exist_ok=True)
os.makedirs(WORKSPACE_DIR, exist_ok=True)
yield
try:
yield
finally:
# Reap every per-app subprocess. Without this each `bash run.sh`
# (and its vite/uvicorn descendants) reparents to PID 1 when the
# main backend dies, leaving ghost listeners on the .env-pinned
# ports that block the next OpenSwarm launch's reload preview.
try:
from backend.apps.outputs.runtime import manager as runtime_manager
killed = await runtime_manager.stop_all()
if killed:
logger.info("outputs lifespan: reaped %d workspace runtimes on shutdown", killed)
except Exception:
logger.exception("outputs lifespan: stop_all failed")
outputs = SubApp("outputs", outputs_lifespan)
@@ -212,7 +225,7 @@ def load_output(output_id: str) -> Output | None:
# descend into. Without this skip-list the workspace endpoint reads
# `node_modules/` (300 MB of MUI source, when it's a real dir and not a
# symlink), `.venv/` (10k+ Python files from the hardlinked cache),
# `__pycache__/`, `dist/`, `.git/`, etc — every 2 seconds while the
# `__pycache__/`, `dist/`, `.git/`, etc; every 2 seconds while the
# agent is active. Result: backend CPU pegged on JSON-serializing
# auto-generated chunks the frontend will then throw away. The frontend
# already filters these for display; this skip is the real fix.
@@ -243,14 +256,14 @@ _WALK_MAX_FILE_BYTES = 256 * 1024
def _walk_directory(folder: str) -> dict[str, str]:
"""Walk a directory tree and return {relative_path: content} for all
text files the user is actually authoring. Skips build/install
directories AND truncates oversize files — both critical for the
directories AND truncates oversize files; both critical for the
polling endpoint, which is called every 2 s while the agent is
writing code and would otherwise serialize hundreds of MB per poll."""
files: dict[str, str] = {}
if not os.path.isdir(folder):
return files
for root, dirs, filenames in os.walk(folder):
# Mutate `dirs` in place — that's how os.walk skips a subtree.
# Mutate `dirs` in place; that's how os.walk skips a subtree.
# Doing it here means we never even stat the children, so a
# 10k-file `.venv/` costs ~one stat (on the dir itself) instead
# of 10k.
@@ -265,7 +278,7 @@ def _walk_directory(folder: str) -> dict[str, str]:
# mis-parsed.
rel_path = os.path.relpath(full_path, folder).replace(os.sep, "/")
try:
# Stat first — cheap, lets us skip giant files without
# Stat first; cheap, lets us skip giant files without
# opening + reading them.
size = os.path.getsize(full_path)
if size > _WALK_MAX_FILE_BYTES:
@@ -305,7 +318,7 @@ async def serve_workspace_file(workspace_id: str, filepath: str, _d: str = ""):
content = _inject_data_into_html(content, input_json, result_json, backend_url_json)
# Iframe sub-resource fetches (<link>, <script src>, <img>) drop the
# parent's ?token= query string, so rewrite the HTML to put the token
# back on every relative URL — otherwise sub-resources 401.
# back on every relative URL; otherwise sub-resources 401.
content = _inject_token_into_relative_urls(content, get_auth_token())
mime, _ = mimetypes.guess_type(filepath)
@@ -361,6 +374,118 @@ async def read_workspace(workspace_id: str):
return {"files": files, "meta": meta, "path": os.path.abspath(folder)}
def sync_output_from_meta_json(workspace_id: str) -> bool:
"""Read meta.json from the workspace folder; if it has a non-empty
name or description that differs from the linked Output row, update
the row. Returns True if anything changed.
Idempotent and best-effort: missing workspace, missing meta.json,
malformed JSON, or no linked Output all return False silently.
Why this exists: the Apps editor's React component polls meta.json
every few seconds and propagates name/description into the Output
via autosave. The canvas-chat App Builder launch has no such
poller, so apps stayed named "Untitled App" forever even after
the agent wrote a real name into meta.json. Calling this from the
session-complete hook closes that gap on the one event we know
fires exactly once per session.
"""
try:
folder = os.path.join(WORKSPACE_DIR, workspace_id)
meta_path = os.path.join(folder, "meta.json")
if not os.path.exists(meta_path):
return False
with open(meta_path) as f:
meta = json.load(f)
if not isinstance(meta, dict):
return False
name = str(meta.get("name") or "").strip()
description = str(meta.get("description") or "").strip()
if not name and not description:
return False
matching = [o for o in _load_all() if o.workspace_id == workspace_id]
if not matching:
return False
output = matching[0]
changed = False
# Only overwrite the default placeholder ("Untitled App" / "") so a
# user who explicitly renamed the app in the UI isn't clobbered by
# a stale meta.json from a prior agent turn.
if name and output.name in ("", "Untitled App") and output.name != name:
output.name = name
changed = True
if description and not output.description and output.description != description:
output.description = description
changed = True
if changed:
output.updated_at = datetime.now().isoformat()
_save(output)
return changed
except (OSError, json.JSONDecodeError, ValueError):
return False
except Exception:
logger.exception("sync_output_from_meta_json failed for %s", workspace_id)
return False
def ensure_webapp_workspace_seeded_and_registered(
workspace_id: str,
folder: str,
session_id: Optional[str] = None,
) -> Optional[str]:
"""Idempotently seed the webapp template into `folder` and register an
Output row pointing at `workspace_id`. Used by the canvas-chat launch
path so picking "App Builder" from the mode dropdown produces the same
sidebar visibility as the Apps editor's `/workspace/seed` flow.
When `session_id` is supplied, it is persisted on the Output row so the
Apps editor can reattach to the same chat history later (without this
link, double-clicking the app card opens an empty editor instead of
the conversation the user already had with the agent).
Idempotency:
- If `run.sh` already exists in the folder, skip the template copy
(matches the seed_workspace endpoint's idempotency guard).
- If any Output already points at this workspace_id, reuse it but
still attach session_id if it's missing.
Returns the output_id on success, None on failure (best-effort; the
caller's session still launches even if registration fails).
"""
try:
os.makedirs(folder, exist_ok=True)
already_seeded = os.path.exists(os.path.join(folder, "run.sh"))
if not already_seeded:
from backend.apps.outputs.runtime import _find_free_port
frontend_port = _find_free_port()
seed_webapp_template_workspace(folder, frontend_port)
with open(os.path.join(folder, "SKILL.md"), "w") as f:
f.write(load_app_builder_skill())
existing = [o for o in _load_all() if o.workspace_id == workspace_id]
if existing:
output = existing[0]
if session_id and output.session_id != session_id:
output.session_id = session_id
output.updated_at = datetime.now().isoformat()
_save(output)
return output.id
now = datetime.now().isoformat()
output = Output(
name="Untitled App",
description="",
icon="view_quilt",
files={},
workspace_id=workspace_id,
session_id=session_id,
created_at=now,
updated_at=now,
)
_save(output)
return output.id
except Exception:
logger.exception("ensure_webapp_workspace_seeded_and_registered failed for %s", workspace_id)
return None
@outputs.router.post("/workspace/seed")
async def seed_workspace(body: WorkspaceSeedRequest):
"""Create a workspace folder and pre-seed it.
@@ -377,7 +502,7 @@ async def seed_workspace(body: WorkspaceSeedRequest):
openswarm-ai/webapp-template snapshot (React + Vite + TS frontend
with an optional FastAPI backend) into the workspace, allocates a
free FRONTEND_PORT and writes it into both `.env` and
`.env.example`. BACKEND_PORT stays NONE — the agent opts in with
`.env.example`. BACKEND_PORT stays NONE; the agent opts in with
`bash backend_init.sh`. Runtime spawn flips to `bash run.sh` and
the preview pane points at `http://localhost:{FRONTEND_PORT}/`.
`body.files` is ignored in this mode; the snapshot is the source
@@ -389,7 +514,7 @@ async def seed_workspace(body: WorkspaceSeedRequest):
# An explicit non-empty `files` payload means the caller has flat-mode
# content to write (a saved legacy Output being reseeded). Don't
# clobber that with the React template even if template_mode is the
# new default — the migration helper has its own path for that.
# new default; the migration helper has its own path for that.
effective_mode = body.template_mode
if body.files:
effective_mode = "flat"
@@ -398,7 +523,7 @@ async def seed_workspace(body: WorkspaceSeedRequest):
# Idempotency guard: re-seeding an existing webapp_template
# workspace would clobber the agent's edits (the helper uses
# dirs_exist_ok=True + copytree). If `run.sh` already exists,
# the workspace was seeded on a previous visit — skip the file
# the workspace was seeded on a previous visit; skip the file
# copy and only re-derive the frontend port from .env.
from backend.apps.outputs.runtime import _find_free_port, _read_env_value
already_seeded = os.path.exists(os.path.join(folder, "run.sh"))
@@ -411,7 +536,7 @@ async def seed_workspace(body: WorkspaceSeedRequest):
else:
frontend_port = _find_free_port()
seed_webapp_template_workspace(folder, frontend_port)
# SKILL.md still goes in workspace root — agent reads it for
# SKILL.md still goes in workspace root; agent reads it for
# context. Live content (user-editable via Skills page) is
# injected into the system prompt regardless.
with open(os.path.join(folder, "SKILL.md"), "w") as f:
@@ -424,7 +549,7 @@ async def seed_workspace(body: WorkspaceSeedRequest):
# the Apps sidebar the moment the user kicks off generation.
# Previously the record only landed when the editor's autosave
# fired, which itself was gated on `files['index.html']` being
# non-empty (a flat-template invariant) — meaning React+Vite
# non-empty (a flat-template invariant); meaning React+Vite
# apps that navigated-away mid-build had no way back. The record
# is a thin pointer (name + workspace_id); the workspace itself
# remains the source of truth for the code.
@@ -456,7 +581,7 @@ async def seed_workspace(body: WorkspaceSeedRequest):
"already_seeded": already_seeded,
}
# Legacy flat path — unchanged.
# Legacy flat path; unchanged.
if body.files:
for rel_path, content in body.files.items():
full_path = os.path.normpath(os.path.join(folder, rel_path))
@@ -558,7 +683,7 @@ async def runtime_restart(workspace_id: str):
from backend.apps.outputs.runtime import manager as runtime_manager
# Restart only if something's attached; otherwise this is a no-op
# silently (a hard-reload click while the runtime was already torn
# down — we'd rather not silently respawn an orphan).
# down; we'd rather not silently respawn an orphan).
rt = runtime_manager.get(workspace_id)
if rt:
await runtime_manager.restart(workspace_id, os.path.abspath(folder))
@@ -570,6 +695,17 @@ async def runtime_get_status(workspace_id: str):
return _runtime_status_payload(workspace_id)
@outputs.router.post("/shutdown-all")
async def runtime_shutdown_all():
"""Reap every workspace subprocess. Electron POSTs this during
will-quit so app subprocesses die BEFORE the main backend gets
SIGTERM'd; without it `bash run.sh` + its vite/uvicorn descendants
reparent to PID 1 and squat on .env-pinned ports forever."""
from backend.apps.outputs.runtime import manager as runtime_manager
killed = await runtime_manager.stop_all()
return {"ok": True, "killed": killed}
@outputs.router.put("/workspace/{workspace_id}/file/{filepath:path}")
async def write_workspace_file(workspace_id: str, filepath: str, body: dict):
"""Write (create/overwrite) a single file in a workspace."""
@@ -579,7 +715,7 @@ async def write_workspace_file(workspace_id: str, filepath: str, body: dict):
folder_norm = os.path.normpath(folder)
full_path = os.path.normpath(os.path.join(folder, filepath))
# `startswith(folder_norm + os.sep)` (not just folder_norm) so a workspace
# `abc-123` can't be tricked into writing into a sibling `abc-1234-evil` —
# `abc-123` can't be tricked into writing into a sibling `abc-1234-evil` ,
# prefix-string collision rather than path-component containment. Today's
# UUID-format ids make the collision unlikely in practice, but the check
# is one character and immunizes future id schemes.
@@ -785,7 +921,7 @@ async def execute_output(body: OutputExecute):
# HITL gate: collect warnings up front. If the caller hasn't opted
# in via force=True AND the code touches anything outside the safe
# allowlist, return the warnings + the code itself so the UI can
# show a preview dialog. No subprocess is spawned on this path —
# show a preview dialog. No subprocess is spawned on this path ,
# zero-cost when warnings exist, identical-to-before when they
# don't.
if not body.force:
backend/apps/outputs/runtime.py
+183 -92
View File
@@ -1,16 +1,4 @@
"""Per-workspace persistent backend runtime.
Each App (workspace) has at most one long-running `backend.py` subprocess
managed by `AppRuntime`. Lifetime is reference-counted via the module-level
`manager` singleton: when the first ViewEditor / DashboardViewCard /
TerminalPanel attaches to a workspace, the process is spawned; when the
last detaches, it's terminated. Multiple subscribers share the same
process and the same in-memory log ring buffer.
This replaces the old one-shot `execute_backend_code` model for the
"backend serves real HTTP endpoints" use case. The one-shot path stays
around (see `executor.py`) for legacy `/api/outputs/execute` callers.
"""
"""Per-workspace persistent backend.py runtime; one AppRuntime per workspace, refcounted by manager singleton."""
import asyncio
import logging
@@ -25,70 +13,28 @@ from typing import Callable, Optional
logger = logging.getLogger(__name__)
# Recent log lines kept in memory per runtime. Lets a Terminal tab that
# opens mid-session replay the context that was already printed instead
# of seeing a blank pane. 2000 lines ≈ a few hundred KB at worst —
# bounded and predictable.
# 2000 lines per runtime; lets a Terminal tab opened mid-session replay context. ~few hundred KB at worst.
_LOG_BUFFER_LINES = 2000
# Seconds to wait after SIGTERM before escalating to SIGKILL. Most
# well-behaved Python servers shut down well under a second; this is the
# upper bound before we move on so a wedged process can't block a
# workspace tear-down forever.
# SIGTERM grace; well-behaved servers shut down under a second so 3s is enough.
_TERMINATE_GRACE_SECONDS = 3
# How long we'll wait for Vite (or whatever frontend server bash run.sh
# spawns) to bind on FRONTEND_PORT before giving up and reporting the
# frontend as "not ready." Covers cold-start `npm install` (~60-90s on
# typical hardware for the template's dependency set) plus the Vite
# bind itself. After this we keep the runtime running — the user can
# check the Terminal pane to see what went wrong — but stop blocking
# the preview pane on a port that may never come up.
# 180s covers npm install (60-90s on typical hardware) plus the Vite bind.
_FRONTEND_BIND_TIMEOUT_SECONDS = 180
# Drop from 0.5 → 0.08 because that 500ms window was ENTIRELY user-visible
# preview latency — after Vite actually binds we'd wait up to half a second
# before noticing and emitting runtime:status to the editor. 80ms TCP
# probes are cheap (async open_connection on localhost, no DNS, no
# handshake to a real upstream) and shave the perceived cold-start by
# roughly half a second. The asyncio.open_connection call has its own
# 500ms connect timeout for the failure case so a wedged listener won't
# turn this into a tight CPU loop.
# 80ms probe: dropping from 500ms was pure user-visible preview latency win; cheap on localhost.
_FRONTEND_BIND_POLL_INTERVAL = 0.08
# Process-wide mutex that serializes new-mode workspace boots so only
# ONE vite optimizeDeps run is in flight at a time. Acquired in
# `AppRuntime.start` (new-mode branch only) BEFORE the run.sh spawn,
# released by `_await_frontend_bind` the instant vite emits its
# "frontend ready" log line — or by the timeout / failure paths.
#
# Why a module-level asyncio.Lock and not part of AppRuntimeManager:
# the lock has to be acquired BEFORE the runtime is registered in
# manager.runtimes (which happens inside manager.attach's own
# `_lock`), and we can't hold both locks at once without inviting
# deadlock. Lifting to the module keeps the two locks fully
# independent — the manager lock guards the runtime dict, this one
# guards "is anyone currently mid-MUI-bundle?"
# Module-level lock so only ONE vite optimizeDeps runs at a time; must be acquired before manager._lock to avoid deadlock with manager.attach.
_vite_boot_lock = asyncio.Lock()
# Number of idle (zero-attachment) runtimes the manager keeps alive in
# its LRU before reaping the oldest. Trades memory for instant
# switch-back: clicking a previously-opened App reattaches to an
# already-running vite + uvicorn instead of paying the ~1-2s spawn
# cost. Bumped beyond 1 because the typical "App Builder" user keeps
# 2-3 in-progress apps and ping-pongs between them.
# Idle runtimes kept in LRU; trades memory for instant switch-back, beyond 1 because typical users ping-pong 2-3 apps.
_MAX_IDLE_RUNTIMES = 3
# Cap on recent error lines kept per workspace runtime. The agent only
# needs a snapshot of "what broke since my last write" — older errors
# get dropped. 50 is enough to catch a babel error message + its stack
# trace + a couple of related warnings without bloating the context.
# Cap on recent error lines the agent gets; 50 is enough for babel error + stack + a few warnings.
_RECENT_ERRORS_MAX = 50
# Regex that matches lines we want to surface back to the agent. Picks
# up the common JS/TS/Python build-error formats vite, babel, tsc, and
# uvicorn emit. Kept narrow on purpose so routine info logs and
# deprecation warnings don't pollute the agent's context.
# Narrow regex for build errors (vite, babel, tsc, uvicorn); keeps routine logs out of agent context.
import re as _re
_ERROR_PATTERNS = _re.compile(
r"(?:"
@@ -114,10 +60,10 @@ def _suspend_process_tree(proc: Optional[asyncio.subprocess.Process]) -> None:
PROCESS GROUP (negative PID) when the child is a session leader,
so vite + uvicorn + their npm/python subchildren all pause together.
No-op on Windows (SIGSTOP has no equivalent — the `OpenProcessToken` +
No-op on Windows (SIGSTOP has no equivalent; the `OpenProcessToken` +
`NtSuspendProcess` route works but isn't worth the win32 surface
here; idle Windows runtimes just stay running, which is the current
behavior). Failures here are swallowed — if the process already died
behavior). Failures here are swallowed; if the process already died
a stop signal is meaningless."""
if proc is None or os.name == "nt":
return
@@ -126,7 +72,7 @@ def _suspend_process_tree(proc: Optional[asyncio.subprocess.Process]) -> None:
return
os.kill(proc.pid, signal.SIGSTOP)
except (ProcessLookupError, PermissionError, OSError):
# Already-dead or out-of-permission — both safe to ignore.
# Already-dead or out-of-permission; both safe to ignore.
pass
@@ -181,10 +127,98 @@ def _find_free_port() -> int:
return s.getsockname()[1]
def _kill_descendant_tree(pid: int, sig_name: str = "TERM") -> None:
"""Recursively signal every descendant of `pid`, leaves-first. The
webapp template's run.sh installs `trap cleanup EXIT` (no TERM), so a
plain SIGTERM to the bash wrapper exits bash silently and leaves
vite/uvicorn grandchildren reparented to PID 1, squatting on the
workspace's ports. Walking the tree ourselves bypasses the template's
signal-handling habits entirely. POSIX uses `pgrep -P` to enumerate
direct children; Windows is covered by `taskkill /T /F` (job-object
walk). All failures are swallowed; missing PIDs mean the process
already exited, which is the desired state anyway."""
if os.name == "nt":
try:
subprocess.run(
["taskkill", "/PID", str(pid), "/T", "/F"],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
timeout=5,
)
except Exception:
pass
return
try:
out = subprocess.run(
["pgrep", "-P", str(pid)],
capture_output=True,
text=True,
timeout=2,
)
children = [int(p) for p in out.stdout.split() if p.strip().isdigit()]
except Exception:
children = []
for child in children:
_kill_descendant_tree(child, sig_name)
sig = getattr(signal, f"SIG{sig_name}", signal.SIGTERM)
for child in children:
try:
os.kill(child, sig)
except (ProcessLookupError, PermissionError, OSError):
pass
def _is_port_free(port: int) -> bool:
"""True if nothing currently holds a TCP listener on 127.0.0.1:port.
Cheap kernel-probe; resolves on bind success. Used as the cross-session
safety net: if a prior OpenSwarm run left a ghost subprocess holding
the .env-persisted FRONTEND_PORT, we detect it here and reallocate
rather than handing run.sh a port that will EADDRINUSE."""
try:
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
s.bind(("127.0.0.1", port))
return True
except OSError:
return False
def _write_env_value(env_path: str, key: str, value: str) -> None:
"""Update KEY=VALUE in an existing `.env`, preserving every other
line. Creates the file if missing. Used when a persisted port collides
with a ghost from a prior session and we have to reallocate before
spawning run.sh."""
lines: list[str] = []
found = False
if os.path.exists(env_path):
try:
with open(env_path, encoding="utf-8") as f:
lines = f.readlines()
except Exception:
lines = []
for i, raw in enumerate(lines):
stripped = raw.strip()
if not stripped or stripped.startswith("#") or "=" not in stripped:
continue
k = stripped.split("=", 1)[0].strip()
if k == key:
lines[i] = f"{key}={value}\n"
found = True
break
if not found:
if lines and not lines[-1].endswith("\n"):
lines[-1] = lines[-1] + "\n"
lines.append(f"{key}={value}\n")
try:
with open(env_path, "w", encoding="utf-8") as f:
f.writelines(lines)
except Exception:
logger.exception("failed writing %s=%s to %s", key, value, env_path)
def _is_new_mode(workspace_path: str) -> bool:
"""A workspace is "new-mode" (webapp-template scaffold) if it has a
`run.sh` at its root. Old-mode workspaces are flat `index.html`-only
apps that pre-date the template swap — they're served by OpenSwarm's
apps that pre-date the template swap; they're served by OpenSwarm's
own `/api/outputs/workspace/{ws}/serve/...` FastAPI route and have an
optional `backend.py` we spawn directly.
@@ -211,7 +245,7 @@ def _read_env_value(env_path: str, key: str) -> Optional[str]:
if k.strip() != key:
continue
v = v.strip()
# Strip an inline `# comment`. Naive — bash semantics are
# Strip an inline `# comment`. Naive; bash semantics are
# more permissive, but values we write don't contain `#`.
if "#" in v:
v = v.split("#", 1)[0].rstrip()
@@ -262,7 +296,7 @@ class AppRuntime:
self.process: Optional[asyncio.subprocess.Process] = None
self.log_buffer: deque[LogLine] = deque(maxlen=_LOG_BUFFER_LINES)
self._subscribers: set[LogSubscriber] = set()
# Recent build/runtime errors scraped from stderr — drained by
# Recent build/runtime errors scraped from stderr; drained by
# the agent's post-tool hook after Write/Edit so the agent sees
# vite/babel/uvicorn errors in its next turn and can self-fix
# instead of leaving the user with a red iframe overlay.
@@ -314,7 +348,7 @@ class AppRuntime:
without waiting for the subprocess to print anything.
- **Old-mode** (no `run.sh`): spawn `python -u backend.py` if
present, with `PORT` env var. This is the legacy path —
present, with `PORT` env var. This is the legacy path ,
unchanged so flat-index.html apps keep working.
Returns True if a process is running after this call. False is
@@ -344,7 +378,7 @@ class AppRuntime:
ok = await self._start_new_mode()
if not ok:
# Spawn failed before the bind-poll task was
# created — release synchronously so we don't
# created; release synchronously so we don't
# wedge the next workspace.
_vite_boot_lock.release()
return ok
@@ -358,20 +392,43 @@ class AppRuntime:
fp_raw = _read_env_value(env_path, "FRONTEND_PORT")
bp_raw = _read_env_value(env_path, "BACKEND_PORT")
# FRONTEND_PORT is allocated by seed_workspace; should always be
# a number. If missing, log + fall back to a fresh allocation —
# rare edge case (workspace seeded by an older OpenSwarm).
# a number. If missing, fall back to a fresh allocation (rare
# edge case: workspace seeded by an older OpenSwarm).
try:
self.frontend_port = int(fp_raw) if fp_raw else _find_free_port()
except ValueError:
self.frontend_port = _find_free_port()
# Port-collision safety net: if a ghost subprocess from a prior
# OpenSwarm run is still bound to the persisted port (force-quit,
# crash, OS killed the parent before stop_all could reap), Vite
# would EADDRINUSE silently. Re-probe and reallocate, then rewrite
# .env so the bash run.sh subprocess reads the new port.
if self.frontend_port and not _is_port_free(self.frontend_port):
new_port = _find_free_port()
self._broadcast(LogLine(
"runtime",
f"[runtime] persisted FRONTEND_PORT {self.frontend_port} is in use; reallocating to {new_port}",
))
self.frontend_port = new_port
_write_env_value(env_path, "FRONTEND_PORT", str(new_port))
# BACKEND_PORT may be the literal string "NONE" (frontend-only
# app — the common case) or a number once `backend_init.sh` has
# app; the common case) or a number once `backend_init.sh` has
# run. Only populate self.port when there's a real backend.
if bp_raw and bp_raw != "NONE":
try:
self.port = int(bp_raw)
except ValueError:
self.port = None
# Same collision check for the backend port; a leaked uvicorn
# from a prior session would otherwise block the new spawn.
if self.port and not _is_port_free(self.port):
new_port = _find_free_port()
self._broadcast(LogLine(
"runtime",
f"[runtime] persisted BACKEND_PORT {self.port} is in use; reallocating to {new_port}",
))
self.port = new_port
_write_env_value(env_path, "BACKEND_PORT", str(new_port))
else:
self.port = None
@@ -407,7 +464,7 @@ class AppRuntime:
self.process = None
return False
backend_note = f" + backend on {self.port}" if self.port else ""
self._broadcast(LogLine("runtime", f"[runtime] bash run.sh started — frontend on {self.frontend_port}{backend_note} (pid {self.process.pid})"))
self._broadcast(LogLine("runtime", f"[runtime] bash run.sh started; frontend on {self.frontend_port}{backend_note} (pid {self.process.pid})"))
self._stdout_task = asyncio.create_task(self._pipe_stream(self.process.stdout, "stdout"))
self._stderr_task = asyncio.create_task(self._pipe_stream(self.process.stderr, "stderr"))
self._wait_task = asyncio.create_task(self._await_exit())
@@ -425,7 +482,7 @@ class AppRuntime:
`frontend_url` property reads.
Also responsible for releasing the module-level `_vite_boot_lock`
— every exit path (success, process death, hard timeout) MUST
; every exit path (success, process death, hard timeout) MUST
release exactly once so the next queued workspace can start its
own vite spawn. A try/finally on the lock guarantees that even
an exception in the poll body doesn't strand the lock holding."""
@@ -451,7 +508,7 @@ class AppRuntime:
port = self.frontend_port
deadline = asyncio.get_event_loop().time() + _FRONTEND_BIND_TIMEOUT_SECONDS
while asyncio.get_event_loop().time() < deadline:
# Stop polling if the process died — pointless to keep
# Stop polling if the process died; pointless to keep
# checking a port nothing will bind.
if self.process is None or self.process.returncode is not None:
return
@@ -472,7 +529,7 @@ class AppRuntime:
f"[runtime] frontend ready at http://127.0.0.1:{port}/",
))
# Release the vite-boot mutex the INSTANT vite is
# ready — the next queued workspace can start its
# ready; the next queued workspace can start its
# own bundle now even though we'll keep streaming
# logs for this one.
_release_boot_lock()
@@ -480,12 +537,12 @@ class AppRuntime:
except (OSError, asyncio.TimeoutError):
pass
await asyncio.sleep(_FRONTEND_BIND_POLL_INTERVAL)
# Timed out — keep the runtime up (Terminal might show useful
# Timed out; keep the runtime up (Terminal might show useful
# errors) but surface why the preview never appeared.
self._broadcast(LogLine(
"runtime",
f"[runtime] frontend did NOT bind on port {port} after "
f"{_FRONTEND_BIND_TIMEOUT_SECONDS}s — check the Terminal "
f"{_FRONTEND_BIND_TIMEOUT_SECONDS}s; check the Terminal "
f"for npm/vite errors.",
))
finally:
@@ -502,7 +559,7 @@ class AppRuntime:
self.port = _find_free_port()
env = self._spawn_env_base()
env["PORT"] = str(self.port)
env["BACKEND_PORT"] = str(self.port) # alias — both common names work
env["BACKEND_PORT"] = str(self.port) # alias; both common names work
try:
# -u forces unbuffered stdout/stderr so the Terminal pane
# sees lines in real time, not whenever Python decides to
@@ -537,15 +594,21 @@ class AppRuntime:
async with self._lock:
if not self.process or self.process.returncode is not None:
# Still cancel the bind poller in case stop() races a
# never-launched runtime — defensive no-op otherwise.
# never-launched runtime; defensive no-op otherwise.
if self._frontend_ready_task and not self._frontend_ready_task.done():
self._frontend_ready_task.cancel()
return
try:
# Walk the descendant tree first so vite/uvicorn grandchildren
# die before bash exits and orphans them to PID 1. The webapp
# template's run.sh only traps EXIT, not TERM, so a flat
# SIGTERM to bash kills bash silently and leaves vite alive.
_kill_descendant_tree(self.process.pid, "TERM")
self.process.terminate()
try:
await asyncio.wait_for(self.process.wait(), timeout=_TERMINATE_GRACE_SECONDS)
except asyncio.TimeoutError:
_kill_descendant_tree(self.process.pid, "KILL")
self.process.kill()
await self.process.wait()
except ProcessLookupError:
@@ -578,7 +641,7 @@ class AppRuntime:
def _broadcast(self, line: LogLine) -> None:
self.log_buffer.append(line)
# Snapshot subscribers — they can self-remove during dispatch.
# Snapshot subscribers; they can self-remove during dispatch.
for cb in list(self._subscribers):
try:
cb(line)
@@ -587,7 +650,7 @@ class AppRuntime:
def _maybe_capture_error(self, text: str) -> None:
"""If a stderr/stdout line matches a known build-error pattern,
record it for the next agent-tool drain. Tests every line —
record it for the next agent-tool drain. Tests every line ,
cheap (single regex search) and only the matching ones land in
the buffer."""
if _ERROR_PATTERNS.search(text):
@@ -622,7 +685,7 @@ class AppRuntimeManager:
Reference-counts attachments so we don't kill a backend when one
Terminal closes while another is still subscribed. First attach
spawns; final detach moves the runtime into an LRU idle pool
instead of stopping it immediately — so re-clicking a recent App
instead of stopping it immediately; so re-clicking a recent App
is instant. The oldest runtime gets reaped once the pool exceeds
_MAX_IDLE_RUNTIMES."""
@@ -638,14 +701,14 @@ class AppRuntimeManager:
async def attach(self, workspace_id: str, workspace_path: str) -> AppRuntime:
revived = False
# Defined here so every code path below leaves it bound — the
# Defined here so every code path below leaves it bound; the
# revive-idle branch used to skip the assignment, leaving the
# post-lock `if dead is not None:` check throwing UnboundLocalError.
dead: Optional[AppRuntime] = None
async with self._lock:
rt = self.runtimes.get(workspace_id)
if rt is None:
# Maybe the runtime is sitting idle in the LRU — revive
# Maybe the runtime is sitting idle in the LRU; revive
# it without paying the spawn cost again.
idle_rt = self._idle_lru.pop(workspace_id, None)
if idle_rt is not None and idle_rt.running:
@@ -658,7 +721,7 @@ class AppRuntimeManager:
_resume_process_tree(rt.process)
else:
if idle_rt is not None:
# Stale idle entry — process died while idling.
# Stale idle entry; process died while idling.
# Drop and spawn a fresh one below; old one
# gets stopped outside the lock.
dead = idle_rt
@@ -667,7 +730,7 @@ class AppRuntimeManager:
else:
# Workspace paths shouldn't change for a given id, but if
# somehow they did (e.g. the user moved the workspace
# folder), trust the latest caller — they have the
# folder), trust the latest caller; they have the
# current truth.
rt.workspace_path = workspace_path
self._attached[workspace_id] = self._attached.get(workspace_id, 0) + 1
@@ -694,7 +757,7 @@ class AppRuntimeManager:
if rt is None:
return
# If the process is already dead, no point keeping it
# around — just clean up. Otherwise move to the LRU AND
# around; just clean up. Otherwise move to the LRU AND
# SIGSTOP the process tree so it consumes 0% CPU while
# idle. The matching SIGCONT lives in attach() above.
if not rt.running:
@@ -736,7 +799,7 @@ class AppRuntimeManager:
"""If `file_path` falls under one of the live workspace
runtimes' workspace_path, drain that workspace's recent
build/runtime errors. Returns [] if no workspace owns the path
or no errors are queued — caller can treat empty as 'all clear'.
or no errors are queued; caller can treat empty as 'all clear'.
Used by agent_manager's post-tool hook so the agent sees vite /
babel / uvicorn errors right after a Write/Edit completes."""
if not file_path:
@@ -745,7 +808,7 @@ class AppRuntimeManager:
abs_path = os.path.abspath(file_path)
except Exception:
return []
# Walk both active and idle runtimes — the user might have
# Walk both active and idle runtimes; the user might have
# navigated away from the workspace mid-build, but the agent
# could still be editing files; the LRU keeps the runtime alive
# for ~3 idle slots.
@@ -767,5 +830,33 @@ class AppRuntimeManager:
await rt.restart()
return rt
async def stop_all(self) -> int:
"""Terminate every active + idle workspace subprocess. Called on
FastAPI lifespan shutdown AND from Electron's pre-quit POST. Without
this, each `bash run.sh` (and its vite/uvicorn descendants) reparents
to PID 1 when the main backend dies, leaving ghost listeners on the
persisted FRONTEND_PORT/BACKEND_PORT that block the NEXT OpenSwarm
launch's app reload. Wakes any SIGSTOP'd idle entries before reaping
so they can run their own shutdown. Parallel via gather; with the
per-runtime 3s SIGTERM grace, worst case is one ~3s wait rather than
N*3s. Idempotent; safe to invoke from multiple shutdown paths."""
async with self._lock:
victims: list[AppRuntime] = []
for rt in list(self.runtimes.values()):
victims.append(rt)
for rt in list(self._idle_lru.values()):
_resume_process_tree(rt.process)
victims.append(rt)
self.runtimes.clear()
self._idle_lru.clear()
self._attached.clear()
if not victims:
return 0
await asyncio.gather(
*(rt.stop() for rt in victims),
return_exceptions=True,
)
return len(victims)
manager = AppRuntimeManager()
backend/apps/outputs/view_builder_templates.py
+25 -25
View File
@@ -27,7 +27,7 @@ SWARM_DEBUG_SKILL_SOURCE_PATH = os.path.join(os.path.dirname(__file__), "swarm_d
# scripts/fetch-webapp-template.sh for the snapshot fetch + patches.
WEBAPP_TEMPLATE_DIR = os.path.join(os.path.dirname(__file__), "webapp_template")
# Bundled default — used as the read-once fallback if the user-editable
# Bundled default; used as the read-once fallback if the user-editable
# copy at ~/.claude/skills/app_builder_skill.md has been removed despite
# the built-in flag (defensive; shouldn't happen in normal use).
with open(APP_BUILDER_SKILL_SOURCE_PATH, encoding="utf-8") as _f:
@@ -38,7 +38,7 @@ def load_app_builder_skill() -> str:
"""Return the live App Builder skill content. Prefers the
user-editable copy at ~/.claude/skills/app_builder_skill.md (so a
user's edit on the Skills page takes effect on the very next App
Builder agent turn — no restart, no copy-on-edit dance). Falls back
Builder agent turn; no restart, no copy-on-edit dance). Falls back
to the bundled default if the user file is somehow gone."""
user_path = os.path.expanduser("~/.claude/skills/app_builder_skill.md")
if os.path.exists(user_path):
@@ -50,7 +50,7 @@ def load_app_builder_skill() -> str:
return APP_BUILDER_SKILL_DEFAULT
# Backward-compat alias. Older callers import VIEW_BUILDER_SKILL directly —
# Backward-compat alias. Older callers import VIEW_BUILDER_SKILL directly ,
# point them at the same content as the user-editable version so a "frozen
# at import" stale copy can't drift from what the skills page shows.
VIEW_BUILDER_SKILL = APP_BUILDER_SKILL_DEFAULT
@@ -127,7 +127,7 @@ VIEW_TEMPLATE_FILES = {
# ---------------------------------------------------------------------------
def _ignore_backend(src: str, names: list[str]) -> list[str]:
"""copytree filter — when copying the template root, drop only the
"""copytree filter; when copying the template root, drop only the
top-level `backend/` directory. Subdirectories named `backend` deeper
in the tree (none today, but defensively scoped) are unaffected."""
if os.path.abspath(src) == os.path.abspath(WEBAPP_TEMPLATE_DIR):
@@ -142,13 +142,13 @@ _TEMPLATE_BACKEND_PATH = os.path.abspath(os.path.join(WEBAPP_TEMPLATE_DIR, "back
# ---------------------------------------------------------------------------
# Shared node_modules cache — every new webapp-template workspace symlinks
# Shared node_modules cache; every new webapp-template workspace symlinks
# its frontend/node_modules to a single warm directory. First-app create
# pays the ~22s npm-install cost once; every subsequent app is instant
# (just a symlink + vite startup, ~1s).
#
# Cache directory is keyed by a sha of the template's package.json, so a
# template dep bump invalidates the cache automatically — old caches sit
# template dep bump invalidates the cache automatically; old caches sit
# until the user clears ~/.openswarm/cache.
# ---------------------------------------------------------------------------
@@ -159,7 +159,7 @@ _warm_cache_thread: threading.Thread | None = None
# Pre-built node_modules archive bundled with packaged releases. Generated
# by `scripts/build-template-archive.sh` and shipped at this path inside
# the app's resources. When present (and tagged with the current
# package.json sha), extract instead of running npm — decompression is
# package.json sha), extract instead of running npm; decompression is
# ~3 s vs ~22 s for the live install. Stale archives (package.json bumped
# but archive not rebuilt) are silently ignored, so the live-install
# fallback always wins on correctness.
@@ -191,7 +191,7 @@ def _try_extract_bundled_archive(cache_dir: str, digest: str) -> bool:
)
os.makedirs(cache_dir, exist_ok=True)
# Archive root is `node_modules/`; extracting into cache_dir places
# it at the expected path. tarfile uses zlib internally for .gz —
# it at the expected path. tarfile uses zlib internally for .gz ,
# no extra dep needed.
with tarfile.open(archive_path, "r:gz") as tar:
tar.extractall(cache_dir)
@@ -214,7 +214,7 @@ def _try_extract_bundled_archive(cache_dir: str, digest: str) -> bool:
def _warm_cache_digest() -> str:
"""Sha of the template's frontend/package.json — used as the cache
"""Sha of the template's frontend/package.json; used as the cache
key + the bundled-archive filename suffix so a package.json bump
invalidates both at once."""
pkg_path = os.path.join(WEBAPP_TEMPLATE_DIR, "frontend", "package.json")
@@ -237,7 +237,7 @@ def _warm_cache_dir() -> str:
def _ensure_warm_cache() -> str | None:
"""Populate the warm-cache node_modules if missing. Returns the
absolute path to the populated `node_modules` directory, or None on
failure. Thread-safe — concurrent callers block on a single install
failure. Thread-safe; concurrent callers block on a single install
instead of racing. Idempotent and fast after the first call."""
cache_dir = _warm_cache_dir()
cache_modules = os.path.join(cache_dir, "node_modules")
@@ -259,7 +259,7 @@ def _ensure_warm_cache() -> str | None:
os.makedirs(cache_dir, exist_ok=True)
# Copy package.json + lockfile (if it exists) into the cache
# dir so npm has something to install from. We don't write
# back to the template — the lockfile generated here stays
# back to the template; the lockfile generated here stays
# local to the cache.
tmpl_pkg = os.path.join(WEBAPP_TEMPLATE_DIR, "frontend", "package.json")
tmpl_lock = os.path.join(WEBAPP_TEMPLATE_DIR, "frontend", "package-lock.json")
@@ -269,7 +269,7 @@ def _ensure_warm_cache() -> str | None:
shutil.copyfile(tmpl_lock, os.path.join(cache_dir, "package-lock.json"))
cmd = ["npm", "ci", *base_flags]
else:
# No lockfile yet — `npm install` resolves the tree and
# No lockfile yet; `npm install` resolves the tree and
# writes one into the cache dir for future use.
cmd = ["npm", "install", *base_flags]
logger.info("webapp-template: warming node_modules cache at %s", cache_dir)
@@ -291,7 +291,7 @@ def _ensure_warm_cache() -> str | None:
def _link_node_modules(workspace_dir: str) -> None:
"""After copytree, point the workspace's frontend/node_modules at
the warm-cache directory. Safe fallback — if the cache isn't ready,
the warm-cache directory. Safe fallback; if the cache isn't ready,
the workspace's run.sh will fall through to its own install path."""
cache_modules = _ensure_warm_cache()
if not cache_modules:
@@ -309,9 +309,9 @@ def _link_node_modules(workspace_dir: str) -> None:
return
elif os.path.isdir(target):
# If the dir is EMPTY (left over from copytree of the template's
# placeholder node_modules — `.gitkeep`-style scenarios) nuke it
# placeholder node_modules; `.gitkeep`-style scenarios) nuke it
# so we can symlink to the warm cache. A non-empty directory is
# treated as a real npm install — respect it and bail.
# treated as a real npm install; respect it and bail.
try:
has_content = any(True for _ in os.scandir(target))
except OSError:
@@ -331,7 +331,7 @@ def _link_node_modules(workspace_dir: str) -> None:
# ---------------------------------------------------------------------------
# Shared Python venv cache — same pattern as the node_modules cache, but
# Shared Python venv cache; same pattern as the node_modules cache, but
# for the workspace backend's FastAPI + transitive deps. Eliminates the
# ~25s `python -m venv` + `pip install -e .` that backend_init.sh
# otherwise pays per workspace.
@@ -358,7 +358,7 @@ def _warm_venv_dir() -> str:
def _ensure_warm_python_venv() -> str | None:
"""Populate the warm-cache backend venv if missing. Returns the
absolute path to the populated `.venv` directory, or None on
failure. Thread-safe and idempotent — fast return after first call."""
failure. Thread-safe and idempotent; fast return after first call."""
cache_dir = _warm_venv_dir()
venv_dir = os.path.join(cache_dir, ".venv")
sentinel = os.path.join(cache_dir, ".populated")
@@ -374,7 +374,7 @@ def _ensure_warm_python_venv() -> str | None:
# Pick the same python the workspace's run.sh would have
# picked, so the venv's binary is compatible. Includes
# bare `python` as the last fallback for Windows, where
# there's no `python3` symlink — the installer ships just
# there's no `python3` symlink; the installer ships just
# `python.exe`. On macOS/Linux the versioned candidates
# match first so we don't accidentally pick a system
# Python 2.x via the bare name.
@@ -405,7 +405,7 @@ def _ensure_warm_python_venv() -> str | None:
return None
# Install the template's dependencies (fastapi[standard],
# typeguard, transitives) — NOT the workspace's own backend,
# typeguard, transitives); NOT the workspace's own backend,
# which gets editable-installed per-workspace by run.sh after
# the cache copy. The venv layout differs by platform:
# POSIX puts executables in `bin/`, Windows in `Scripts/`,
@@ -461,7 +461,7 @@ def warm_cache_in_background() -> None:
_warm_cache_thread.start()
# Trigger pre-warm on module import — backend startup hits this and the
# Trigger pre-warm on module import; backend startup hits this and the
# installs run in parallel with the rest of the boot. By the time the
# user creates their first app, node_modules + the backend venv are
# usually ready.
@@ -495,10 +495,10 @@ def seed_webapp_template_workspace(workspace_dir: str, frontend_port: int) -> No
1. Copy `.env.example` → `.env` verbatim (preserves the upstream
defaults `FRONTEND_PORT=4949` and `BACKEND_PORT=NONE`).
2. Sed both `.env` and `.env.example` to set `FRONTEND_PORT=<port>`.
BACKEND_PORT stays NONE in both (per spec — the agent flips it
BACKEND_PORT stays NONE in both (per spec; the agent flips it
via backend_init.sh when it needs a backend).
3. Append two install-specific paths to `.env` ONLY (NOT
`.env.example` — these are absolute paths on the current
`.env.example`; these are absolute paths on the current
machine, not template defaults):
OPENSWARM_TEMPLATE_BACKEND_PATH=<abs path to master template's backend/>
OPENSWARM_DEBUGGER_PATH=<abs path to OpenSwarm's debugger/ package>
@@ -506,7 +506,7 @@ def seed_webapp_template_workspace(workspace_dir: str, frontend_port: int) -> No
the template's `backend/run.sh` to install our local debugger
before `pip install -e .`.
Idempotent within reason — re-running over an existing workspace
Idempotent within reason; re-running over an existing workspace
overwrites template files and re-asserts the env values.
"""
os.makedirs(workspace_dir, exist_ok=True)
@@ -528,10 +528,10 @@ def seed_webapp_template_workspace(workspace_dir: str, frontend_port: int) -> No
_patch_env_port(env_path, "FRONTEND_PORT", str(frontend_port))
_patch_env_port(env_example_path, "FRONTEND_PORT", str(frontend_port))
# Install-specific paths — .env only.
# Install-specific paths; .env only.
_patch_env_port(env_path, "OPENSWARM_TEMPLATE_BACKEND_PATH", _TEMPLATE_BACKEND_PATH)
_patch_env_port(env_path, "OPENSWARM_DEBUGGER_PATH", _DEBUGGER_PATH)
# Backend-venv warm-cache path — backend_init.sh checks this for a
# Backend-venv warm-cache path; backend_init.sh checks this for a
# pre-populated `.venv/` to cp -aR into the workspace instead of
# paying the ~25s venv-create + pip-install cost. Written even if
# the cache isn't ready yet; backend_init.sh re-checks at run time.
backend/apps/outputs/webapp_template/backend_init.sh
+5 -5
View File
@@ -3,7 +3,7 @@
#
# Idempotent. The workspace is seeded frontend-only (no backend/ dir,
# BACKEND_PORT=NONE). Run this script when your App needs server-side
# code — it copies the master template's backend/ into the workspace
# code; it copies the master template's backend/ into the workspace
# and flips BACKEND_PORT in both .env files to a free port.
#
# After running this, hard-reload the preview (right-click the reload
@@ -15,7 +15,7 @@ HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
cd "$HERE"
if [[ ! -f .env ]]; then
echo "ERROR: .env not found at $HERE — is this the workspace root?" >&2
echo "ERROR: .env not found at $HERE. Is this the workspace root?" >&2
exit 1
fi
@@ -26,12 +26,12 @@ source .env
set +a
if [[ "${BACKEND_PORT:-NONE}" != "NONE" ]]; then
echo "Backend already enabled on port $BACKEND_PORT — nothing to do." >&2
echo "Backend already enabled on port $BACKEND_PORT, nothing to do." >&2
exit 0
fi
if [[ -d ./backend ]]; then
echo "ERROR: ./backend/ already exists but BACKEND_PORT=NONE — your" >&2
echo "ERROR: ./backend/ already exists but BACKEND_PORT=NONE; your" >&2
echo " workspace is in an inconsistent state. Either delete" >&2
echo " ./backend/ and re-run, or set BACKEND_PORT manually." >&2
exit 1
@@ -56,7 +56,7 @@ echo "Copying backend/ from $OPENSWARM_TEMPLATE_BACKEND_PATH..."
cp -R "$OPENSWARM_TEMPLATE_BACKEND_PATH" ./backend
chmod +x ./backend/run.sh
# Reuse the warm-cache backend venv if available — this skips the
# Reuse the warm-cache backend venv if available; this skips the
# ~5s venv-create + ~20s pip-install in the workspace's backend/run.sh.
# The cache holds FastAPI + transitives pre-installed; the workspace's
# own editable install (`pip install -e .`) still runs once on first
backend/apps/service/buffer.py
+5 -5
View File
@@ -3,8 +3,8 @@
When the desktop is offline (laptop closed, no internet, cloud unreachable),
the service-sync layer can't reach `api.openswarm.com`. Rather than drop
data on the floor, we spool submissions to a small SQLite file and replay
them on the next online tick. The spool is bounded — when full, the oldest
entries are dropped — so it can never balloon to a problem.
them on the next online tick. The spool is bounded; when full, the oldest
entries are dropped; so it can never balloon to a problem.
Single file, single table, single thread guarded by a sqlite3 connection's
implicit lock. No concurrency model beyond "don't write from two processes
@@ -24,7 +24,7 @@ from typing import Iterator, Optional
logger = logging.getLogger(__name__)
# Cap the spool at 50 MB on disk. SQLite's overhead means the actual ceiling
# on retained payloads is somewhat smaller, which is fine — this is a
# on retained payloads is somewhat smaller, which is fine; this is a
# best-effort cushion, not a guaranteed retention window.
_MAX_BYTES = 50 * 1024 * 1024
@@ -63,7 +63,7 @@ def enqueue(spool_path: str, kind: str, payload: dict, *, now: float) -> None:
"INSERT INTO spool (kind, payload, created_at) VALUES (?, ?, ?)",
(kind, body, now),
)
# Cheap size check — only run trim when stat says we're over.
# Cheap size check; only run trim when stat says we're over.
try:
size = os.path.getsize(spool_path)
except OSError:
@@ -108,7 +108,7 @@ def drain(spool_path: str, batch_size: int = 50) -> list[tuple[int, str, dict]]:
try:
out.append((rid, kind, json.loads(body)))
except json.JSONDecodeError:
# Corrupt row — discard so it doesn't block draining behind it.
# Corrupt row; discard so it doesn't block draining behind it.
with _lock, _conn(spool_path) as c:
c.execute("DELETE FROM spool WHERE id = ?", (rid,))
logger.warning("Dropped corrupt spool row id=%s", rid)
backend/apps/service/client.py
+8 -8
View File
@@ -4,7 +4,7 @@ Single public surface: `submit(kind, payload)`. The desktop hands off
opaque payload dicts; the cloud at api.openswarm.com is responsible for
parsing and routing them. The desktop has no schema knowledge.
Three `kind` values are accepted — they're the routing primitive the
Three `kind` values are accepted; they're the routing primitive the
cloud needs to send the payload to the right backend handler. The shape
of `payload` is opaque from the desktop's perspective; the cloud knows
how to read it.
@@ -61,7 +61,7 @@ def _spool_path() -> str:
def set_test_sink(fn: Optional[Any]) -> None:
"""Test seam — receives every submission instead of the network."""
"""Test seam; receives every submission instead of the network."""
global _test_sink
_test_sink = fn
@@ -92,7 +92,7 @@ def _get_user_id() -> Optional[str]:
from backend.apps.settings.settings import load_settings
s = load_settings()
# Prefer the cloud-issued user_id (UUID) if the user has signed in
# via Google OAuth, magic link, or Stripe checkout — that's the
# via Google OAuth, magic link, or Stripe checkout; that's the
# authoritative identity. Falls back to user_email for installs
# that haven't completed sign-in yet (so existing onboarding-only
# installs don't lose their Person history during the v1.0.29
@@ -164,7 +164,7 @@ def _envelope() -> dict:
except Exception:
pass
# Locale: BCP 47 string ("en-US", "es-ES", etc.) injected by Electron via
# app.getLocale() — see electron/main.js. We don't fall back to Python's
# app.getLocale(); see electron/main.js. We don't fall back to Python's
# locale.getdefaultlocale() because that's deprecated, often empty, and
# returns inconsistent OS-specific values across macOS/Windows/Linux.
try:
@@ -265,7 +265,7 @@ def _log(kind: str, payload: dict) -> None:
def sync(data: dict | None = None) -> None:
"""Sync operational state to the cloud. Single entry point.
Accepts any dict — the cloud determines what it is from the shape.
Accepts any dict; the cloud determines what it is from the shape.
The desktop has no knowledge of event types, schemas, or routing.
Each call carries:
@@ -295,12 +295,12 @@ def sync(data: dict | None = None) -> None:
_schedule(_post_or_spool(_DEFAULT_SYNC_PATH, body, "s"))
# Internal routing — the cloud has one endpoint for everything.
# Internal routing; the cloud has one endpoint for everything.
_DEFAULT_SYNC_PATH = "/api/service/sync"
def submit(kind: str, payload: dict) -> None:
"""Legacy shim — routes through sync(). Kept for back-compat during
"""Legacy shim; routes through sync(). Kept for back-compat during
migration. New code should call sync() directly."""
sync(payload)
@@ -379,7 +379,7 @@ def record(
session_id: Optional[str] = None,
dashboard_id: Optional[str] = None,
) -> None:
"""Legacy collector.record() shim — splits dotted name into surface/action."""
"""Legacy collector.record() shim; splits dotted name into surface/action."""
if "." in event_type:
surface, action = event_type.split(".", 1)
else:
backend/apps/service/models.py
+1 -5
View File
@@ -1,5 +1 @@
"""(Reserved for future use; intentionally empty.)
The service-sync layer ships opaque payload dicts through `submit()` —
no Pydantic shape exposed in the public repo.
"""
"""Reserved; service-sync ships opaque payload dicts via submit(), no Pydantic shape exposed."""
backend/apps/service/ring_buffer.py
+1 -7
View File
@@ -1,10 +1,4 @@
"""Fixed-size event log for operational diagnostics.
Maintains a rolling window of the last N app events so support
diagnostics can include context about recent activity. Used by
the error report builder to attach "what just happened" when
something goes wrong.
"""
"""Fixed-size rolling event log for support diagnostics."""
from __future__ import annotations
backend/apps/service/service.py
+9 -9
View File
@@ -33,7 +33,7 @@ logger = logging.getLogger(__name__)
def _read_app_version() -> str:
# Preferred: Electron's main process injects this when spawning the
# backend (see electron/main.js — OPENSWARM_APP_VERSION). Always reliable
# backend (see electron/main.js; OPENSWARM_APP_VERSION). Always reliable
# in packaged builds because it comes from app.getVersion() rather than
# path-based file resolution.
env_v = os.environ.get("OPENSWARM_APP_VERSION", "").strip()
@@ -42,7 +42,7 @@ def _read_app_version() -> str:
# Fallback: read electron/package.json via relative path. Works in
# `bash run.sh` dev mode where the repo layout is intact, but FAILS in
# packaged dmg/exe builds because electron/package.json isn't shipped
# into Resources/ — which made every shipped install report
# into Resources/; which made every shipped install report
# app_version="unknown" pre-fix. Kept for backward compatibility with
# dev runs and as a safety net if the env var is ever unset.
try:
@@ -120,7 +120,7 @@ async def _pulse_loop():
if _pulse_count >= _pulse_batch_size:
try:
from backend.apps.agents.agent_manager import agent_manager
# Compact field names — the wire stays small and the cloud
# Compact field names; the wire stays small and the cloud
# is the only place that knows what each key means.
svc.sync({
"a": len(agent_manager.sessions), # active sessions
@@ -412,26 +412,26 @@ async def service_status():
async def post_submit(body=Body(...)):
"""Accepts three body shapes for backward compatibility:
1. Frontend `report()` shape — flat `{s, a, p, submission_id, t}`.
1. Frontend `report()` shape; flat `{s, a, p, submission_id, t}`.
This is what `frontend/src/shared/serviceClient.ts:report()` sends
on every UI interaction. Pass through unchanged so the cloud sees
it as a frontend.event.
2. Legacy `{kind, payload}` shape — used by older callers that wrapped
2. Legacy `{kind, payload}` shape; used by older callers that wrapped
the payload in a kind+payload envelope before submitting. Unwrap
and forward the payload.
3. Batched array — frontend collects up to 1s of events and sends them
3. Batched array; frontend collects up to 1s of events and sends them
as a single JSON array to cut N POSTs/sec down to 1. Each item is
processed exactly as if it had arrived as its own request.
Pre-fix this endpoint required shape #2 and silently rejected shape #1
with a 200 + `{ok:false}`, so every UI event from `report()` was
dropped — `frontend.event` count was 0 in production analytics.
dropped; `frontend.event` count was 0 in production analytics.
"""
# Shape 3: batched array. Recurse per-item so single-item handling
# logic stays in one place. Returns a single ok regardless of
# individual item shape — analytics calls aren't transactional.
# individual item shape; analytics calls aren't transactional.
if isinstance(body, list):
for item in body:
if isinstance(item, dict):
@@ -446,7 +446,7 @@ async def post_submit(body=Body(...)):
return {"ok": True}
if not isinstance(body, dict):
return {"ok": False, "error": "JSON object or array required"}
# Shape 1: frontend `report()` — flat {s, a, p, ...}
# Shape 1: frontend `report()`; flat {s, a, p, ...}
if any(k in body for k in ("s", "a", "p")):
svc.sync(body)
return {"ok": True}
backend/apps/settings/credentials.py
+12 -59
View File
@@ -1,8 +1,4 @@
"""Centralized credential resolution for LLM API calls.
Supports multiple providers: Anthropic (native), OpenAI, Gemini,
OpenRouter, and user-configured custom providers.
"""
"""Resolve LLM credentials for the configured provider."""
from __future__ import annotations
@@ -26,18 +22,14 @@ def _check_9router() -> bool:
def validate_credentials(settings: AppSettings, provider: str = "anthropic") -> None:
"""Raise ValueError if credentials are missing for the given provider.
Allows through if 9Router is running as a fallback.
Handles both display names ('Anthropic') and lowercase ('anthropic').
"""
"""Raise ValueError if the provider has no usable credentials."""
p = provider.lower().strip()
# 9Router-backed providers don't need traditional credentials
# 9Router handles its own credentials.
if p == "9router":
return
# If 9Router is running, all providers are accessible
# 9Router proxies every provider, so if it's up we don't need keys here.
if _check_9router():
return
@@ -62,21 +54,20 @@ def validate_credentials(settings: AppSettings, provider: str = "anthropic") ->
return
raise ValueError("OpenRouter API key not configured. Set it in Settings.")
elif p in ("xai", "meta", "deepseek", "mistral", "qwen", "cohere"):
# These route through OpenRouter — need either OpenRouter key or 9Router
# These providers route through OpenRouter, so its key is required.
if getattr(settings, "openrouter_api_key", None):
return
raise ValueError(f"{provider} requires an OpenRouter API key, or connect a subscription via 9Router.")
else:
# Custom provider — check if it exists in custom_providers
for cp in getattr(settings, "custom_providers", []):
if cp.name.lower() == p:
return
# Unknown provider — allow through (create_provider will handle the error)
# Let create_provider raise for unknown providers; not our job here.
return
def get_provider_credentials(settings: AppSettings, provider: str) -> dict[str, str]:
"""Return credential dict for a specific provider."""
"""Return the credential dict for the given provider."""
p = provider.lower().strip()
validate_credentials(settings, provider)
@@ -97,45 +88,17 @@ def get_provider_credentials(settings: AppSettings, provider: str) -> dict[str,
if p == "openrouter":
return {"api_key": getattr(settings, "openrouter_api_key", "") or ""}
# Custom provider
for cp in getattr(settings, "custom_providers", []):
if cp.name.lower() == p:
# Substitute a placeholder when the user left api_key blank
# — local OpenAI-compatible servers (LM Studio, Ollama, etc.)
# ignore the Bearer header but downstream callers may insist
# on non-empty values.
# Local OpenAI-compatible servers (LM Studio, Ollama) ignore the key; placeholder keeps downstream callers happy.
key = (cp.api_key or "").strip() or "no-auth-required"
return {"api_key": key, "base_url": cp.base_url}
raise ValueError(f"No credentials for provider: {provider}")
# ---------------------------------------------------------------------------
# Legacy helpers (kept for backward compat during migration)
# ---------------------------------------------------------------------------
def get_agent_sdk_env(settings: AppSettings) -> dict[str, str]:
"""Return the env dict for ClaudeAgentOptions based on connection mode.
DEPRECATED: Use create_provider() from providers.registry instead.
"""
validate_credentials(settings, "anthropic")
if getattr(settings, "connection_mode", "own_key") == "openswarm-pro":
proxy_url = getattr(settings, "openswarm_proxy_url", None) or OPENSWARM_DEFAULT_PROXY_URL
return {
"ANTHROPIC_AUTH_TOKEN": getattr(settings, "openswarm_bearer_token", ""),
"ANTHROPIC_BASE_URL": proxy_url,
}
return {"ANTHROPIC_API_KEY": settings.anthropic_api_key}
def get_anthropic_client(settings: AppSettings) -> anthropic.AsyncAnthropic:
"""Return a configured AsyncAnthropic client based on connection mode.
Priority: managed mode → 9Router subscription → API key
"""
"""Return an AsyncAnthropic client for the user's current connection mode."""
import anthropic
if getattr(settings, "connection_mode", "own_key") == "openswarm-pro":
@@ -145,11 +108,11 @@ def get_anthropic_client(settings: AppSettings) -> anthropic.AsyncAnthropic:
base_url=proxy_url,
)
# Prefer API key when set
# Prefer the user's own API key when present.
if settings.anthropic_api_key:
return anthropic.AsyncAnthropic(api_key=settings.anthropic_api_key)
# Fall back to 9Router subscription (free for users with Claude/ChatGPT/Gemini subscriptions)
# Fall back to 9Router (free for users with Claude/ChatGPT/Gemini subscriptions).
if _check_9router():
return anthropic.AsyncAnthropic(
api_key="9router",
@@ -160,17 +123,7 @@ def get_anthropic_client(settings: AppSettings) -> anthropic.AsyncAnthropic:
def get_anthropic_client_for_model(settings: AppSettings, api_model: str) -> anthropic.AsyncAnthropic:
"""Return a client configured for the given resolved model id.
When api_model carries a 9Router prefix (cc/, cx/, gc/, cp-), the client
targets 9Router directly — even if connection_mode is openswarm-pro. This
is what lets pinned-route models like "sonnet-cc" actually reach the
user's own subscription instead of getting sent through the managed proxy
with an unrecognizable model id. cp- is the prefix we use when registering
user-configured custom OpenAI-compatible providers in 9Router.
Otherwise delegates to get_anthropic_client() for the default mode-driven
routing.
"""
"""Route 9Router-prefixed models (cc/, cx/, gc/, cp-) straight to 9Router so user subscriptions reach their own accounts."""
import anthropic
if isinstance(api_model, str) and (
api_model.startswith(("cc/", "cx/", "gc/")) or api_model.startswith("cp-")
backend/apps/settings/models.py
+19 -40
View File
@@ -4,27 +4,27 @@ from typing import Optional, Any, Literal
DEFAULT_SYSTEM_PROMPT = (
"You are a personal AI assistant running inside OpenSwarm.\n\n"
"## Core Behavior\n"
"Act, don't ask. When a tool can accomplish the task, call it immediately — "
"Act, don't ask. When a tool can accomplish the task, call it immediately; "
"do not describe what you would do, do not ask for confirmation, just execute. "
"The user expects results, not plans.\n"
"If ANY available tool is relevant to the user's request, use it. Never respond "
'with "I can do X for you" or "Would you like me to..." — just do it. '
'with "I can do X for you" or "Would you like me to..."; just do it. '
"A tool call is always better than a text explanation of what the tool would do.\n"
"For multi-step tasks, chain tool calls in sequence — don't stop after one step "
"For multi-step tasks, chain tool calls in sequence; don't stop after one step "
"to ask if you should continue. Complete the entire task, then report the results.\n"
"Be adaptable. If one approach fails, try a different tool or strategy instead of "
"giving up or repeating the same action. Always stay focused on what the user "
"actually wants to accomplish — their intent matters more than the specific method.\n\n"
"actually wants to accomplish; their intent matters more than the specific method.\n\n"
"## Tool Priority\n"
"1. Connected MCP tools — fastest and most reliable. Use ToolSearch to discover "
"1. Connected MCP tools; fastest and most reliable. Use ToolSearch to discover "
"what integrations are available if you're unsure.\n"
"2. WebSearch / WebFetch — for general web lookups when no MCP tool fits.\n"
"3. BrowserAgent — last resort, only for visual interaction with websites, "
"2. WebSearch / WebFetch; for general web lookups when no MCP tool fits.\n"
"3. BrowserAgent; last resort, only for visual interaction with websites, "
"filling forms, or tasks no other tool can handle.\n\n"
"## Style\n"
"Do not narrate routine tool calls — just call the tool.\n"
"Do not narrate routine tool calls; just call the tool.\n"
"After tool calls complete, present the results directly. Do not recap which "
"tools you called or why — the user can see tool calls in the UI.\n"
"tools you called or why; the user can see tool calls in the UI.\n"
"Keep responses brief and direct. Use plain language.\n"
"If you genuinely need clarification on something ambiguous, use the "
"AskUserQuestion tool. Never ask questions inline in plain text.\n"
@@ -40,60 +40,39 @@ class AppSettings(BaseModel):
default_thinking_level: Literal["off", "low", "medium", "high", "auto"] = "auto"
zoom_sensitivity: float = 50.0
theme: str = "dark"
# App Builder workspaces seed a React template that ships with its own
# theme toggle ("Light" / "Dark" at the bottom of the sidebar). By
# default the template should follow the user's OS appearance; once
# the user explicitly toggles it inside any one app the override
# persists across every subsequently-built app via this field
# (the template fetches /api/settings on mount and PUTs back here on
# toggle, so the preference is shared even though each app runs from
# its own vite port / localStorage origin).
# null = follow system / no override; 'light' or 'dark' = sticky.
# Shared across App Builder workspaces (each runs its own vite port / localStorage origin); null = follow system.
app_template_theme_override: Optional[Literal["light", "dark"]] = None
new_agent_shortcut: str = "Meta+l"
anthropic_api_key: Optional[str] = None
browser_homepage: str = "https://www.google.com"
# Multi-provider API keys
openai_api_key: Optional[str] = None
google_api_key: Optional[str] = None
openrouter_api_key: Optional[str] = None
custom_providers: list["CustomProvider"] = Field(default_factory=list)
# Dashboard / UI preferences
auto_select_mode_on_new_agent: bool = False
expand_new_chats_in_dashboard: bool = False
auto_reveal_sub_agents: bool = True
dev_mode: bool = False
# Subscription tokens (from CLI tools — alternative to API keys)
allow_experimental_updates: bool = False
claude_subscription_token: Optional[str] = None
openai_subscription_token: Optional[str] = None
gemini_subscription_token: Optional[str] = None
# User profile (collected during onboarding)
user_name: Optional[str] = None
user_email: Optional[str] = None
user_use_case: Optional[str] = None
user_referral_source: Optional[str] = None
# Per-MCP dismissal map for the preflight suggestion modal. Keyed by
# the curated ToolDefinition.name (e.g. "Google Workspace"); value is
# an ISO timestamp of dismissal. Used by mcp_preflight._build_available_shortlist
# to suppress suggestions the user has explicitly waved off.
# Suppresses preflight suggestion modal entries the user dismissed; keyed by ToolDefinition.name, value ISO timestamp.
dismissed_mcp_suggestions: dict[str, str] = Field(default_factory=dict)
# Analytics: opted in by default, user can toggle off
analytics_opt_in: bool = True
installation_id: Optional[str] = None
first_opened_at: Optional[str] = None # ISO timestamp of first app open
# OpenSwarm Pro subscription
connection_mode: str = "own_key" # "own_key" | "openswarm-pro"
first_opened_at: Optional[str] = None
connection_mode: str = "own_key"
openswarm_bearer_token: Optional[str] = None
openswarm_proxy_url: Optional[str] = None # default resolved in credentials.py
openswarm_subscription_plan: Optional[str] = None # "hobby"|"pro"|"pro_plus"|"ultra"
openswarm_subscription_expires: Optional[str] = None # ISO 8601
openswarm_usage_cached: Optional[dict] = None # {count, limit, window_end_at}
# Identity (v1.0.29+). Populated after a successful sign-in via the cloud's
# /api/auth/signin-activate endpoint (Google OAuth or email magic link).
# Stripe checkout also populates these because the cloud's bearer-mint
# always returns user info. Distinct from user_email above which was
# historically a self-reported onboarding field — the values agree once
# sign-in completes (server-validated wins).
openswarm_proxy_url: Optional[str] = None
openswarm_subscription_plan: Optional[str] = None
openswarm_subscription_expires: Optional[str] = None
openswarm_usage_cached: Optional[dict] = None
# Server-validated identity from /api/auth/signin-activate; user_email above is the self-reported onboarding value.
user_id: Optional[str] = None
signin_method: Optional[Literal["google", "stripe", "email"]] = None
backend/apps/settings/settings.py
+11 -54
View File
@@ -37,10 +37,7 @@ async def settings_lifespan():
import asyncio as _asyncio
async def _boot_router_then_sync():
"""Start 9Router (if any apikey-routed provider is configured)
then push our key-based connections into it. Sequential because
sync_* helpers no-op when 9Router isn't running yet — running
them post-boot guarantees the connections actually land."""
"""Boot 9Router then push key-based connections (sequential: sync helpers no-op pre-boot)."""
needs_router = any([
getattr(s, "google_api_key", None),
getattr(s, "openai_api_key", None),
@@ -76,13 +73,7 @@ settings = SubApp("settings", settings_lifespan)
def _migrate_legacy_fields(raw: dict) -> dict:
"""Translate deprecated field names/values so they survive into the new schema.
Pre-launch scaffolding used `connection_mode="managed"` and
`openswarm_auth_token`; production names are `"openswarm-pro"` and
`openswarm_bearer_token`. Zero known users are affected, but keep the
mapping for safety.
"""
"""Translate deprecated pre-launch field names ('managed', 'openswarm_auth_token') into production schema."""
if raw.get("connection_mode") == "managed":
raw["connection_mode"] = "openswarm-pro"
if "openswarm_auth_token" in raw and "openswarm_bearer_token" not in raw:
@@ -102,26 +93,19 @@ def load_settings() -> AppSettings:
return AppSettings()
# Single threading.Lock guards every write to SETTINGS_FILE — protects against
# corruption from two requests racing through the file system. Async callers
# offload the actual write to the default thread pool (run_in_executor), so
# the lock works for both sync and thread-pool execution paths.
# threading.Lock guards every SETTINGS_FILE write; works for sync paths and async run_in_executor paths.
_settings_write_lock = threading.Lock()
def _atomic_write_settings(payload: dict) -> None:
"""Internal: serialise payload to SETTINGS_FILE atomically.
Always called via save_settings* — don't invoke directly."""
"""Atomic SETTINGS_FILE write; call via save_settings*, not directly."""
with _settings_write_lock:
os.makedirs(DATA_DIR, exist_ok=True)
fd, tmp = tempfile.mkstemp(prefix=".settings.", suffix=".tmp", dir=DATA_DIR)
try:
with os.fdopen(fd, "w", encoding="utf-8") as f:
json.dump(payload, f, indent=2)
# On Windows, os.replace can transiently fail with PermissionError
# if Defender or another reader holds the destination open. One
# retry after a short backoff handles every real-world case
# without masking genuine permission bugs.
# Windows: Defender can briefly lock the destination; one retry handles every real case.
for attempt in range(2):
try:
os.replace(tmp, SETTINGS_FILE)
@@ -139,24 +123,17 @@ def _atomic_write_settings(payload: dict) -> None:
def save_settings(settings_obj: AppSettings) -> None:
"""Synchronously persist settings atomically. Thread-safe.
Use from sync paths (analytics collector, lifespans). Async callers should
prefer save_settings_async to avoid blocking the event loop on Windows
where Defender scans can stretch the write to 50-200ms."""
"""Sync atomic persist; thread-safe. Async callers should prefer save_settings_async (Defender can stretch writes to 50-200ms)."""
_atomic_write_settings(settings_obj.model_dump())
async def save_settings_async(settings_obj: AppSettings) -> None:
"""Async-safe atomic save. Runs the file I/O in the default thread pool
so the FastAPI event loop stays responsive while the write completes.
Shares the threading.Lock with the sync variant for safe interleaving."""
"""Async atomic save via thread pool; shares the lock with the sync variant."""
payload = settings_obj.model_dump()
loop = asyncio.get_running_loop()
await loop.run_in_executor(None, _atomic_write_settings, payload)
# Backward-compat alias. Existing sync callers (analytics collector, analytics
# lifespan) continue to work; new async callers should use save_settings_async.
def _save_settings(settings_obj: AppSettings) -> None:
save_settings(settings_obj)
@@ -172,14 +149,12 @@ async def update_settings(body: AppSettings):
old = load_settings()
# Sync the settings state (secrets stripped).
secret_keys = {"anthropic_api_key", "openai_api_key", "google_api_key", "openrouter_api_key",
"claude_subscription_token", "openai_subscription_token", "gemini_subscription_token",
"openswarm_bearer_token", "installation_id"}
safe = {k: v for k, v in body.model_dump().items() if k not in secret_keys}
_sync(safe)
# Identify user in service-sync when profile is set/changed
if (body.user_email and body.user_email != getattr(old, "user_email", None)) or \
(body.user_name and body.user_name != getattr(old, "user_name", None)):
from backend.apps.service.client import identify as _identify
@@ -227,8 +202,7 @@ async def update_settings(body: AppSettings):
except Exception:
pass
# Boot+sync runs off the request path — ensure_running() can take 5min
# on first install (npm pull) and would freeze the event loop.
# Off the request path: ensure_running() can take 5min on first install (npm pull) and would freeze the loop.
if google_changed or openai_changed or openrouter_changed or custom_providers_changed:
async def _boot_and_sync_keys(
google_key: str | None,
@@ -275,12 +249,7 @@ async def update_settings(body: AppSettings):
any_keyed_added,
))
# When openswarm-pro mode or bearer token changes, register a `claude`
# apikey connection in 9Router that proxies through our cloud. This
# makes the CLI's built-in WebSearch work on non-Claude primaries for
# Pro users — the CLI's Anthropic delegation path now has a working
# Claude route via 9Router, instead of hitting "no credentials for
# provider: claude".
# On pro-mode/bearer change, register a `claude` apikey connection in 9Router so CLI WebSearch works on non-Claude primaries.
pro_mode_old = getattr(old, "connection_mode", None) == "openswarm-pro"
pro_mode_new = getattr(body, "connection_mode", None) == "openswarm-pro"
bearer_old = getattr(old, "openswarm_bearer_token", None)
@@ -305,25 +274,13 @@ class AppThemeOverridePayload(BaseModel):
@settings.router.get("/app-theme-override")
async def get_app_theme_override():
"""Cross-app theme preference for App Builder workspaces.
Returns the current override (or `null` for follow-system). Apps
served from the template fetch this on mount so a toggle inside
any one app sticks across every future app the user builds. Each
app workspace runs on its own vite port (separate localStorage
origin), so the backend is the only place this can live."""
"""Cross-app theme preference for App Builder workspaces; backend-held because each app uses its own localStorage origin."""
return {"mode": load_settings().app_template_theme_override}
@settings.router.put("/app-theme-override")
async def put_app_theme_override(body: AppThemeOverridePayload):
"""MERGE the theme override into AppSettings. The general PUT
/api/settings endpoint replaces the whole AppSettings object —
sending a partial body there would default every secret-bearing
field (api keys, subscription tokens), which logs the user out
and pops the SignInGate. This dedicated endpoint mutates only
`app_template_theme_override` and leaves every other field
untouched."""
"""MERGE the override; the general PUT /api/settings replaces the whole object and would blank secrets, logging the user out."""
current = load_settings()
current.app_template_theme_override = body.mode
await save_settings_async(current)
backend/apps/skill_registry/skill_registry.py
+1 -1
View File
@@ -43,7 +43,7 @@ def _parse_frontmatter(raw: str) -> tuple[dict, str]:
async def _fetch_skill_paths(client: httpx.AsyncClient) -> list[tuple[str, str]]:
"""Fetch the marketplace.json manifest and return (skill_folder, plugin_name) pairs.
Uses raw.githubusercontent.com — no GitHub API needed, no rate limiting.
Uses raw.githubusercontent.com; no GitHub API needed, no rate limiting.
"""
resp = await client.get(MANIFEST_URL)
resp.raise_for_status()
backend/apps/skills/models.py
+1 -5
View File
@@ -10,11 +10,7 @@ class Skill(BaseModel):
content: str
file_path: str = ""
command: str = ""
# Skills that OpenSwarm ships as part of the platform (e.g. the App
# Builder reference) get this flag set. The UI hides the delete
# button for them and the DELETE endpoint refuses with 409. Content
# is still editable — the whole point is that users can tune how
# the platform-internal agents behave.
# Platform-shipped skills (e.g. App Builder): UI hides delete and DELETE returns 409, but content stays editable so users can tune them.
built_in: bool = False
backend/apps/skills/skills.py
+6 -6
View File
@@ -32,7 +32,7 @@ def _save_index(index: dict[str, dict]):
# skill file we copy into ~/.claude/skills/ on first boot and tag with
# `built_in: true` in the index. Users can edit the content (their
# changes flow through to the matching agent's prompt on the next turn),
# but they can't delete the file — the DELETE endpoint refuses with 409.
# but they can't delete the file; the DELETE endpoint refuses with 409.
def _built_in_skill_registry() -> list[dict]:
# Imported lazily so this module stays cheap to import from
# everywhere (the skills outputs module pulls in pydantic+fastapi
@@ -47,7 +47,7 @@ def _built_in_skill_registry() -> list[dict]:
"name": "App Builder",
"description": (
"Reference doc the App Builder agent reads on every turn. "
"Edit this to change how every App Builder agent behaves — "
"Edit this to change how every App Builder agent behaves; "
"your edits take effect on the next turn, no restart. "
"Built-in: can be edited but not deleted."
),
@@ -58,7 +58,7 @@ def _built_in_skill_registry() -> list[dict]:
"id": "swarm_debug_skill",
"name": "swarm-debug Logger",
"description": (
"How to use `swarm_debug.debug()` in an App backend — the "
"How to use `swarm_debug.debug()` in an App backend; the "
"colored frame-aware logger that lands in the App Builder's "
"Terminal pane under [BACKEND]. Edit to teach your debugging "
"conventions to the App Builder agent. Built-in: editable, "
@@ -73,7 +73,7 @@ def _built_in_skill_registry() -> list[dict]:
def _seed_built_in_skills() -> None:
"""Copy each built-in skill into SKILLS_DIR if not already present, and
ensure the index has the `built_in: true` flag so the UI and DELETE
endpoint know to treat it specially. Idempotent — safe to call on
endpoint know to treat it specially. Idempotent; safe to call on
every boot. Doesn't overwrite the file once it exists (so user edits
are preserved across restarts and upgrades)."""
index = _load_index()
@@ -114,7 +114,7 @@ async def skills_lifespan():
try:
_seed_built_in_skills()
except Exception:
# Don't block app startup on a skill-seed failure — the worst
# Don't block app startup on a skill-seed failure; the worst
# case is the user has to manually paste the skill in once.
logger.exception("failed to seed built-in skills")
yield
@@ -296,7 +296,7 @@ async def delete_skill(skill_id: str):
status_code=409,
detail=(
f"'{skill_id}' is a built-in skill and can't be deleted "
"(edit its content instead — your edits take effect on "
"(edit its content instead; your edits take effect on "
"the next agent turn)."
),
)
backend/apps/subscription/router.py
+6 -6
View File
@@ -41,7 +41,7 @@ async def _clear_subscription(settings_obj, *, drop_bearer: bool = True) -> None
`drop_bearer=True` (the default) is the original behavior, used when the
cloud reports the bearer as revoked (401) or the subscription as past its
grace period (402) — the bearer is dead so we have to clear it.
grace period (402); the bearer is dead so we have to clear it.
`drop_bearer=False` is used by the explicit user-initiated /disconnect
endpoint: the bearer still authenticates the user's account at api.me
@@ -62,7 +62,7 @@ async def _clear_subscription(settings_obj, *, drop_bearer: bool = True) -> None
def _sync_subscription_identity(settings_obj) -> None:
"""Push the installation's current subscription state into service-sync person
properties so every event from this user is segmentable by plan /
paying-vs-free. Safe to call from hot paths — service-sync is fire-and-forget
paying-vs-free. Safe to call from hot paths; service-sync is fire-and-forget
and swallows errors internally."""
try:
from backend.apps.service.client import identify as _identify
@@ -178,7 +178,7 @@ async def status():
"connection_mode": mode,
}
# Best-effort live fetch — surface stale cache if cloud is unreachable.
# Best-effort live fetch; surface stale cache if cloud is unreachable.
# Network errors leave upstream_code=None so we keep the cached state;
# only explicit 401/402 from the cloud trigger a local clear.
live_usage = None
@@ -203,7 +203,7 @@ async def status():
logger.debug("subscription/status live fetch failed: %s", e)
# Cloud says the bearer is gone (401) or the sub is past its grace
# period (402) — drop local credentials so the desktop stops routing
# period (402); drop local credentials so the desktop stops routing
# through a dead subscription. Settings UI sees connected=False and
# falls back to the Subscribe CTA; chat reverts to own_key routing.
if upstream_code in (401, 402):
@@ -237,7 +237,7 @@ async def sync():
state forever.
No-op when not in openswarm-pro mode. Best-effort: network failures are
swallowed — the caller still gets a 200 with whatever local state we
swallowed; the caller still gets a 200 with whatever local state we
already had."""
# Lazy-import the service-sync helper so subscription/router doesn't pay the
# cost when analytics are disabled.
@@ -285,7 +285,7 @@ async def sync():
cloud_plan = data.get("plan")
period_end_ms = data.get("current_period_end")
# Only touch local fields the cloud explicitly confirmed — don't paper
# Only touch local fields the cloud explicitly confirmed; don't paper
# over missing keys with defaults that would downgrade an older record.
if cloud_plan:
settings_obj.openswarm_subscription_plan = cloud_plan
backend/apps/tools_lib/models.py
+13 -5
View File
@@ -30,17 +30,25 @@ BUILTIN_TOOLS: list[BuiltinTool] = [
BuiltinTool(name="EnterWorktree", description="Enter a git worktree for isolated work", category="system", deferred=True),
BuiltinTool(name="TaskOutput", description="Read output from a background task", category="system", deferred=True),
BuiltinTool(name="TaskStop", description="Stop a running background task", category="system", deferred=True),
BuiltinTool(name="CronCreate", description="Create a scheduled or recurring task", category="scheduling", deferred=True),
BuiltinTool(name="CronList", description="List all scheduled tasks", category="scheduling", deferred=True),
BuiltinTool(name="CronDelete", description="Delete a scheduled task", category="scheduling", deferred=True),
# CronCreate/List/Delete are kept for compatibility with the Claude
# Agent SDK's task system but are intentionally NOT how users in
# OpenSwarm should schedule recurring work. The native scheduler
# ("Schedule" button in any chat header, or /schedule slash command)
# gives the workflow a card on the canvas, a calendar entry, audit
# logs, cost caps, and a Pause-all toggle. Cron entries are invisible
# to the platform and survive uninstall. The descriptions below tell
# the agent so it picks the right path.
BuiltinTool(name="CronCreate", description="Schedule a one-off background task within the current session (NOT for recurring user workflows; use the user-visible 'Schedule' button / native workflow scheduler for anything the user wants to repeat on a real-world calendar)", category="scheduling", deferred=True),
BuiltinTool(name="CronList", description="List background tasks in the current session (NOT user-facing scheduled workflows; those live in the Workflows hub)", category="scheduling", deferred=True),
BuiltinTool(name="CronDelete", description="Delete a background task in the current session (NOT a user-facing scheduled workflow)", category="scheduling", deferred=True),
# Agent tools
BuiltinTool(name="Agent", display_name="CreateAgent", description="Spawn a sub-agent to handle a complex subtask", category="agents"),
BuiltinTool(name="InvokeAgent", description="Invoke a copy of an existing agent with a new message, preserving full conversation context", category="agents"),
# Browser delegation tools (Layer 1 — what the main agent calls)
# Browser delegation tools (Layer 1; what the main agent calls)
BuiltinTool(name="CreateBrowserAgent", description="Create a new browser and run a task on it", category="browser_delegation"),
BuiltinTool(name="BrowserAgent", description="Delegate a browser task to an existing browser agent", category="browser_delegation"),
BuiltinTool(name="BrowserAgents", description="Run multiple browser tasks in parallel on existing browsers", category="browser_delegation"),
# Browser action tools (Layer 2 — what the sub-agent executes)
# Browser action tools (Layer 2; what the sub-agent executes)
BuiltinTool(name="BrowserScreenshot", description="Capture a screenshot of the browser page", category="browser_action"),
BuiltinTool(name="BrowserNavigate", description="Navigate the browser to a URL", category="browser_action"),
BuiltinTool(name="BrowserClick", description="Click an element by CSS selector", category="browser_action"),
backend/apps/tools_lib/tools_lib.py
+146 -33
View File
@@ -14,7 +14,7 @@ from urllib.parse import urlencode
import httpx
from dotenv import load_dotenv
from fastapi import HTTPException, Query
from fastapi import HTTPException, Query, Request, Response
from fastapi.responses import HTMLResponse
from pydantic import BaseModel
from backend.config.Apps import SubApp
@@ -27,7 +27,7 @@ OPENSWARM_OAUTH_BASE_URL = os.environ.get(
"OPENSWARM_OAUTH_BASE_URL", "https://api.openswarm.com"
).rstrip("/")
from backend.config.paths import BACKEND_DIR, DATA_ROOT, TOOLS_DIR as DATA_DIR, BUILTIN_PERMISSIONS_PATH as BUILTIN_PERMS_PATH
from backend.config.paths import BACKEND_DIR, DATA_ROOT, TOOLS_DIR as DATA_DIR, BUILTIN_PERMISSIONS_PATH as BUILTIN_PERMS_PATH, TRUSTED_SENSITIVE_PATHS_PATH
load_dotenv(os.path.join(BACKEND_DIR, ".env"))
if os.environ.get("OPENSWARM_PACKAGED") == "1":
@@ -72,7 +72,7 @@ def _load(tool_id: str) -> ToolDefinition:
tool = ToolDefinition(**json.load(f))
# Migrate Discord tool configs from the old npx-based spawn (which
# broke whenever the npx cache was partially populated) to the local
# Python shim. Idempotent — if it's already on the shim, no-op.
# Python shim. Idempotent; if it's already on the shim, no-op.
if (
tool.name.lower() == "discord"
and tool.mcp_config
@@ -106,11 +106,51 @@ def save_builtin_permissions(perms: dict[str, str]):
json.dump(perms, f, indent=2)
def load_trusted_sensitive_paths() -> list[str]:
if not os.path.exists(TRUSTED_SENSITIVE_PATHS_PATH):
return []
try:
with open(TRUSTED_SENSITIVE_PATHS_PATH) as f:
data = json.load(f)
except (json.JSONDecodeError, OSError):
return []
raw = data.get("patterns") if isinstance(data, dict) else None
if not isinstance(raw, list):
return []
return [p for p in raw if isinstance(p, str) and p]
def save_trusted_sensitive_paths(patterns: list[str]):
os.makedirs(os.path.dirname(TRUSTED_SENSITIVE_PATHS_PATH), exist_ok=True)
seen: list[str] = []
for p in patterns:
if isinstance(p, str) and p and p not in seen:
seen.append(p)
with open(TRUSTED_SENSITIVE_PATHS_PATH, "w") as f:
json.dump({"patterns": seen}, f, indent=2)
@tools_lib.router.get("/builtin/permissions")
async def get_builtin_permissions():
return {"permissions": load_builtin_permissions()}
@tools_lib.router.get("/trusted-sensitive-paths")
async def get_trusted_sensitive_paths():
"""Patterns the user has opted into always-allow for sensitive-path writes."""
return {"patterns": load_trusted_sensitive_paths()}
@tools_lib.router.put("/trusted-sensitive-paths")
async def replace_trusted_sensitive_paths(body: dict):
"""Replace the full list; Settings page uses this to revoke entries."""
incoming = body.get("patterns") or []
if not isinstance(incoming, list):
return {"patterns": load_trusted_sensitive_paths()}
save_trusted_sensitive_paths([p for p in incoming if isinstance(p, str) and p])
return {"patterns": load_trusted_sensitive_paths()}
@tools_lib.router.put("/builtin/permissions")
async def update_builtin_permissions(body: dict):
valid_tools = {t.name for t in BUILTIN_TOOLS}
@@ -225,7 +265,7 @@ def _resolve_command(command: str) -> str | None:
if found:
return found
# Windows binaries need an extension. shutil.which() handles PATHEXT for
# PATH lookups, but we manually scan _extra_bin_dirs below — replicate
# PATH lookups, but we manually scan _extra_bin_dirs below; replicate
# the suffix probing here so `uvx` finds `uvx.exe`, etc.
if sys.platform == "win32":
suffixes = [""] + os.environ.get("PATHEXT", ".COM;.EXE;.BAT;.CMD").lower().split(os.pathsep)
@@ -292,21 +332,37 @@ def derive_mcp_config(tool: ToolDefinition) -> Optional[dict]:
env["PRIVATE_APP_ACCESS_TOKEN"] = tool.oauth_tokens["access_token"]
if tool.oauth_tokens.get("refresh_token"):
env["GOOGLE_WORKSPACE_REFRESH_TOKEN"] = tool.oauth_tokens["refresh_token"]
# google_workspace_mcp's auth/gauth.py requires all three of
# CLIENT_ID / CLIENT_SECRET / REFRESH_TOKEN at startup and does
# its own token refresh per API call; it ignores any
# pre-refreshed access_token. v1.0.29's cloud-proxy migration
# closes the OAuth-flow secret exposure, but the MCP still
# needs the local secret here. v1.0.30 should either fork the
# MCP to point token_uri at our cloud refresh proxy, or replace
# it with a thin in-house Gmail/Drive/Calendar wrapper that
# consumes a pre-refreshed access_token.
client_id = os.environ.get("GOOGLE_OAUTH_CLIENT_ID", "")
client_secret = os.environ.get("GOOGLE_OAUTH_CLIENT_SECRET", "")
if client_id:
env["GOOGLE_WORKSPACE_CLIENT_ID"] = client_id
if client_secret:
env["GOOGLE_WORKSPACE_CLIENT_SECRET"] = client_secret
# google_workspace_mcp's gauth.py hardcodes token_uri to
# https://oauth2.googleapis.com/token and refreshes using the
# local CLIENT_ID/SECRET on every API call. The OAuth flow
# itself runs through the cloud's rotation pool, so the
# refresh_token is bound to whichever pool slot minted it,
# not the single client baked into the DMG. Mismatch -> Google
# returns unauthorized_client. We point token_uri at a local
# proxy that forwards the refresh to our cloud's pool-aware
# /api/oauth/google/refresh endpoint; CLIENT_ID/SECRET become
# unused placeholders (gauth.py only validates non-empty).
_port = os.environ.get("OPENSWARM_PORT", "8324")
env["GOOGLE_WORKSPACE_TOKEN_URI"] = (
f"http://127.0.0.1:{_port}/api/tools/google-oauth-token"
)
env.setdefault("GOOGLE_WORKSPACE_CLIENT_ID", "openswarm-proxy")
env.setdefault("GOOGLE_WORKSPACE_CLIENT_SECRET", "openswarm-proxy")
# Google Workspace MCP: redirect spawn through our shim that
# monkey-patches gauth.get_credentials before the worker registers
# tools, so token_uri points at our local proxy. Stays a stdio
# subprocess; google-workspace-mcp gets installed into uv's
# ephemeral env via --with, same way the upstream entry-point
# invocation used to do it.
if tool.name.lower() == "google workspace" and config.get("type") == "stdio":
shim_path = os.path.join(
os.path.dirname(os.path.dirname(os.path.abspath(__file__))),
"google_workspace_mcp_shim",
"run.py",
)
config["command"] = "uv"
config["args"] = ["run", "--with", "google-workspace-mcp", "python", shim_path]
# Discord MCP runs as a small Python shim (backend.apps.discord_mcp_shim).
# We pass install_id + base URL via env so the shim subprocess doesn't
@@ -321,7 +377,7 @@ def derive_mcp_config(tool: ToolDefinition) -> Optional[dict]:
if guild_ids:
env["OPENSWARM_DISCORD_GUILD_IDS"] = ",".join(guild_ids)
# The shim runs as a subprocess and needs to import
# `backend.apps.discord_mcp_shim` — set PYTHONPATH to the project
# `backend.apps.discord_mcp_shim`; set PYTHONPATH to the project
# root (parent of the backend/ dir) so that import resolves.
_project_root = os.path.dirname(os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))))
existing_pp = env.get("PYTHONPATH") or os.environ.get("PYTHONPATH", "")
@@ -398,7 +454,7 @@ def derive_mcp_config(tool: ToolDefinition) -> Optional[dict]:
if config.get("command"):
# `python` (no version suffix) doesn't exist on a stock macOS,
# so a tool config that asks for "python" silently fails to
# spawn — Claude Agent SDK then exposes zero tools from that
# spawn; Claude Agent SDK then exposes zero tools from that
# MCP. We resolve to the actual interpreter running the
# backend (sys.executable), which is guaranteed to exist and
# have backend modules importable. `python3` and absolute
@@ -407,7 +463,7 @@ def derive_mcp_config(tool: ToolDefinition) -> Optional[dict]:
resolved_python = sys.executable or shutil.which("python3") or shutil.which("python")
if resolved_python:
config["command"] = resolved_python
# Check for bundled npm MCP servers — use Electron's Node.js instead of npx
# Check for bundled npm MCP servers; use Electron's Node.js instead of npx
if config["command"] in ("npx", "bunx"):
pkg_name = next((a for a in (config.get("args") or []) if not a.startswith("-")), None)
if pkg_name:
@@ -485,7 +541,7 @@ def derive_mcp_config(tool: ToolDefinition) -> Optional[dict]:
env = config.setdefault("env", {})
env.setdefault("PATH", _augmented_path())
env.setdefault("PYTHONPATH", "")
# Point uv/uvx at our bundled Python — avoids macOS CLT popup on fresh Macs
# Point uv/uvx at our bundled Python; avoids macOS CLT popup on fresh Macs
# and avoids downloading Python at runtime
_is_packaged = os.environ.get("OPENSWARM_PACKAGED") == "1"
_is_windows = sys.platform == "win32"
@@ -667,7 +723,7 @@ def _try_heal_npx_cache(stderr: str) -> str | None:
Why: interrupted npx installs leave a `package-lock.json` in the cache dir so
subsequent spawns reuse a partially-extracted node_modules tree, which dies at
import time. Scoped strictly to the extracted hash subdir — never touches
import time. Scoped strictly to the extracted hash subdir; never touches
anything outside `~/.npm/_npx/`.
"""
if "ERR_MODULE_NOT_FOUND" not in stderr:
@@ -789,7 +845,7 @@ async def _discover_mcp_tools_stdio(command: str, args: list[str] | None = None,
except HTTPException as e:
# Heal-on-corrupt-npx-cache still triggers from the EOF branch,
# which now includes the full stderr tail in `e.detail` — so the
# which now includes the full stderr tail in `e.detail`; so the
# ERR_MODULE_NOT_FOUND signature is still discoverable here.
if _attempt == 0 and _try_heal_npx_cache(str(e.detail) if e.detail is not None else ""):
return await _discover_mcp_tools_stdio(command, args, env, _attempt=1)
@@ -797,10 +853,10 @@ async def _discover_mcp_tools_stdio(command: str, args: list[str] | None = None,
except asyncio.TimeoutError:
# Most common cause: cold npx cache on Windows. The npm install
# persists across attempts, so a retry usually finishes against a
# warm cache. Surface npx's own progress line if we have one — it
# warm cache. Surface npx's own progress line if we have one; it
# makes the cause obvious ("downloading X...") instead of opaque.
tail_text = "".join(stderr_tail[-5:]).strip()
detail = "MCP discovery timed out — the server may still be downloading on first run"
detail = "MCP discovery timed out; the server may still be downloading on first run"
if tail_text:
preview = tail_text[-200:].replace("\n", " ").strip()
detail += f" (last output: {preview})"
@@ -931,7 +987,7 @@ def _m365_server_script() -> str:
backend/mcp-bundles/softeria-ms-365-mcp-server/dist/index.js (4.7MB).
The new path mirrors the SDK's internal layout (dist/index.js + sibling
package.json) because cli.js reads __dirname/../package.json for the
--version flag — see scripts/build-app.sh `build_mcp_bundle_dir`.
--version flag; see scripts/build-app.sh `build_mcp_bundle_dir`.
"""
_backend = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
bundle = os.path.join(
@@ -1015,7 +1071,7 @@ async def m365_device_login(tool_id: str):
login_state["status"] = "awaiting_auth"
if url_match and "microsoft" in url_match.group(1).lower():
login_state["device_code_url"] = url_match.group(1)
# Process ended — check result
# Process ended; check result
proc.wait()
remaining_stderr = proc.stderr.read() if proc.stderr else ""
login_state["output"] += remaining_stderr
@@ -1228,7 +1284,7 @@ async def oauth_cloud_claim(
data = resp.json()
tokens = data.get("tokens", {}) or {}
tool = _load(tool_id)
# Google's token endpoint doesn't include the user's email — fetch it
# Google's token endpoint doesn't include the user's email; fetch it
# from userinfo so the UI can show "you connected ericzeng@gmail.com"
# rather than the generic "Google account" placeholder.
if tool.name.lower() == "google" and tokens.get("access_token") and not tokens.get("email"):
@@ -1251,7 +1307,7 @@ def _persist_cloud_tokens(tool: ToolDefinition, tokens: dict) -> None:
"""Normalise the cloud's claim response into tool.oauth_tokens.
Per-provider shaping mirrors what the v1.0.25 local-callback flow used
to write — the rest of the app (refresh helpers, MCP env injection)
to write; the rest of the app (refresh helpers, MCP env injection)
expects exactly this shape.
"""
name = tool.name.lower()
@@ -1307,7 +1363,7 @@ async def _refresh_via_proxy(provider: str, tool: ToolDefinition, default_expiry
json={"refresh_token": refresh_token},
)
if resp.status_code == 401:
# Provider rejected — user revoked at the provider's side. Mark
# Provider rejected; user revoked at the provider's side. Mark
# as needing re-auth so the UI prompts a Reconnect.
tool.auth_status = "expired"
_save(tool)
@@ -1340,7 +1396,7 @@ async def _refresh_via_proxy(provider: str, tool: ToolDefinition, default_expiry
async def refresh_google_token(tool: ToolDefinition) -> Optional[str]:
"""Refresh an expired Google access_token via the Fly cloud-proxy.
The client_secret never leaves Fly — desktop only POSTs the
The client_secret never leaves Fly; desktop only POSTs the
refresh_token. Same pattern as Airtable/HubSpot. Pre-v1.0.29 builds
held the secret in their bundled .env; v1.0.29 removed it.
"""
@@ -1896,3 +1952,60 @@ async def telegram_password(payload: dict) -> dict:
await _tg_finalize_after_auth(tool, phone, client)
_TG_PENDING.pop(tool_id, None)
return {"ok": True}
@tools_lib.router.post("/google-oauth-token")
async def google_oauth_token_proxy(request: Request):
"""Local mimic of Google's OAuth2 token endpoint for the
google-workspace-mcp subprocess.
google-workspace-mcp's google-auth library posts form-encoded
{grant_type, refresh_token, client_id, client_secret} on every
expired-token refresh. Because OAuth runs through a cloud-side
rotation pool, the local CLIENT_ID/SECRET don't match the pool slot
that minted the refresh_token, so a direct refresh against Google
returns unauthorized_client. We accept the form-encoded shape,
discard the (mismatched) local client creds, and forward the
refresh_token to api.openswarm.com/api/oauth/google/refresh which
walks the pool to find the issuing slot. The cloud's JSON envelope
is reshaped back to Google's native token-endpoint response so
google-auth keeps working transparently.
"""
form = await request.form()
grant_type = form.get("grant_type") or ""
refresh_token = form.get("refresh_token") or ""
if grant_type != "refresh_token" or not refresh_token:
return Response(
content='{"error":"unsupported_grant_type"}',
status_code=400,
media_type="application/json",
)
try:
async with httpx.AsyncClient(timeout=15.0) as client:
upstream = await client.post(
f"{OPENSWARM_OAUTH_BASE_URL}/api/oauth/google/refresh",
json={"refresh_token": refresh_token},
)
except Exception as e:
return Response(
content=f'{{"error":"upstream_unreachable","error_description":"{e}"}}',
status_code=502,
media_type="application/json",
)
if upstream.status_code != 200:
return Response(
content=upstream.text,
status_code=upstream.status_code,
media_type="application/json",
)
tokens = (upstream.json() or {}).get("tokens") or {}
return Response(
content=json.dumps({
"access_token": tokens.get("access_token", ""),
"expires_in": tokens.get("expires_in", 3600),
"scope": tokens.get("scope", ""),
"token_type": tokens.get("token_type", "Bearer"),
}),
status_code=200,
media_type="application/json",
)
backend/apps/web/web.py
+8 -8
View File
@@ -41,7 +41,7 @@ class SearchBody(BaseModel):
num_results: int = Field(5, ge=1, le=10, description="Max results to return.")
# Hint from the MCP server about which primary provider the session
# is using. Lets us route to that provider's native search tool
# (Gemini googleSearch, OpenAI web_search_preview) when available —
# (Gemini googleSearch, OpenAI web_search_preview) when available ,
# costs come out of the user's existing primary budget.
primary: str | None = Field(None, description="Primary provider hint: 'gemini' | 'openai' | 'anthropic' | None")
@@ -53,7 +53,7 @@ class FetchBody(BaseModel):
# ---------------------------------------------------------------------------
# Helper — extract plain text from a tool's structured output list
# Helper; extract plain text from a tool's structured output list
# ---------------------------------------------------------------------------
@@ -143,7 +143,7 @@ def _format_grounded_as_fetch(grounded: dict, url: str) -> str:
if chunks:
parts.append("\nCited sources:")
for i, (title, uri) in enumerate(chunks[:5], start=1):
parts.append(f" [{i}] {title} — {uri}")
parts.append(f" [{i}] {title}; {uri}")
return "\n".join(parts)
@@ -167,7 +167,7 @@ def _resolve_openai_api_key() -> str | None:
# Cache of which 9Router subscriptions are connected. Refreshed via
# `_refresh_9r_connected()` rather than hit on every search call —
# `_refresh_9r_connected()` rather than hit on every search call ,
# 9Router's /api/providers is fast but not free, and we already
# query it from many places.
_NINE_ROUTER_CONNECTED: set[str] = set()
@@ -196,7 +196,7 @@ async def _refresh_9r_connected() -> set[str]:
}
_NINE_ROUTER_CACHE_AT = now
except Exception:
# Cache stays — best-effort.
# Cache stays; best-effort.
pass
return _NINE_ROUTER_CONNECTED
@@ -207,7 +207,7 @@ async def _gemini_grounded_via_9router(prompt: str, use_url_context: bool) -> di
search call instead of needing a separate AI Studio API key.
Routes through Anthropic-shape against 9Router's translator. We
request a tool result naturally — the translator surfaces grounded
request a tool result naturally; the translator surfaces grounded
URIs as text + cited sources in the response body. Format-shape
matches the existing `_gemini_grounded_call` so downstream
`_format_grounded_as_search_results` works unchanged."""
@@ -482,14 +482,14 @@ async def search(body: SearchBody) -> dict:
has_subscription = bool(connected & {"codex", "antigravity", "gemini-cli"})
if not (gemini_key or openai_key or has_subscription):
hint = (
"\n\n(DuckDuckGo returned no results — likely rate-limiting this IP. "
"\n\n(DuckDuckGo returned no results; likely rate-limiting this IP. "
"Connect Codex / Antigravity / Gemini CLI in Settings, or add an "
"OpenAI / Gemini API key, for reliable native search.)"
)
else:
hint = (
"\n\n(DuckDuckGo returned no results and the connected providers "
"didn't return useful results either — try rephrasing the query.)"
"didn't return useful results either; try rephrasing the query.)"
)
return {
"query": body.query,
backend/apps/workflows/__init__.py
View File
backend/apps/workflows/audit.py
+95
View File
@@ -0,0 +1,95 @@
"""Append-only audit log for workflow edits.
One JSONL file per workflow at <DATA_ROOT>/workflows/audit/<wid>.jsonl. We
diff before/after rather than snapshotting the full record so the file
stays small even after dozens of edits. Read path tails the file; we don't
keep this in memory because audits are inspected rarely.
"""
import json
import logging
import os
from datetime import datetime, timezone
from threading import Lock
from typing import Any
from backend.apps.workflows.storage import DATA_DIR
logger = logging.getLogger(__name__)
AUDIT_DIR = os.path.join(DATA_DIR, "audit")
_io_lock = Lock()
# Soft cap on bytes per audit file. When exceeded we truncate to the last
# CAP/2 bytes on next write so attackers (or a runaway PATCH loop) can't
# fill the disk. 256 KiB is ~2000 edits; we never expect to hit it.
SOFT_CAP_BYTES = 256 * 1024
def _audit_path(wid: str) -> str:
return os.path.join(AUDIT_DIR, f"{wid}.jsonl")
def _diff(before: dict, after: dict) -> dict[str, dict[str, Any]]:
"""Return only the keys whose value changed. Nested dicts are diffed
shallowly; the schedule/actions/permissions blocks are small so we just
record the whole sub-dict when any sub-key changes.
"""
changed: dict[str, dict[str, Any]] = {}
keys = set(before) | set(after)
for k in keys:
b = before.get(k)
a = after.get(k)
if b != a:
changed[k] = {"before": b, "after": a}
return changed
def log_change(wid: str, who: str, before: dict, after: dict) -> None:
diff = _diff(before, after)
if not diff:
return
entry = {
"ts": datetime.now(timezone.utc).isoformat(),
"who": who,
"diff": diff,
}
try:
with _io_lock:
os.makedirs(AUDIT_DIR, exist_ok=True)
path = _audit_path(wid)
if os.path.exists(path) and os.path.getsize(path) > SOFT_CAP_BYTES:
# Keep the tail half. Cheap, lossy, prevents pathological
# disk growth without crashing on a corrupt file.
with open(path, "rb") as f:
f.seek(-(SOFT_CAP_BYTES // 2), os.SEEK_END)
tail = f.read()
first_nl = tail.find(b"\n")
tail = tail[first_nl + 1:] if first_nl >= 0 else b""
with open(path, "wb") as f:
f.write(tail)
with open(path, "a") as f:
f.write(json.dumps(entry) + "\n")
except Exception:
logger.debug("audit log_change failed", exc_info=True)
def read_tail(wid: str, limit: int = 50) -> list[dict]:
path = _audit_path(wid)
if not os.path.exists(path):
return []
try:
with open(path) as f:
lines = f.readlines()
except Exception:
return []
out: list[dict] = []
for line in lines[-limit:]:
line = line.strip()
if not line:
continue
try:
out.append(json.loads(line))
except Exception:
continue
out.reverse()
return out
backend/apps/workflows/escalation.py
+90
View File
@@ -0,0 +1,90 @@
"""Server-side escalation timer.
The permission chain in the UI (notify -> text -> call) used to time out
client-side, which dies the moment the window closes. We move the timer
here so a run that finishes at 9am can escalate to a real text at 9:05am
whether or not the user has the app open. The text/call wire-up itself
still routes through notifier (cloud SMS bridge is wired separately); we
just own the *when*.
State lives in module-scoped dicts, not on disk. If the backend restarts
mid-escalation the chain is lost on purpose: the user is already in front
of an open app at that point (otherwise the backend wouldn't have started)
and they can ack manually. Persisting escalation state would mean
re-firing on a stale schedule after a multi-day downtime, which is worse.
"""
import asyncio
import logging
from datetime import datetime, timedelta, timezone
from typing import Optional
from backend.apps.workflows.models import PermissionTier, Workflow, WorkflowRun
logger = logging.getLogger(__name__)
_tasks: dict[str, asyncio.Task] = {} # run_id -> escalation task
_state: dict[str, dict] = {} # run_id -> {tier_idx, next_at, kind}
def _tier_delay_seconds(tier: PermissionTier) -> int:
"""Tier minutes/hours convention matches the FE: text uses minutes,
call uses hours (the UI label flips with tier.kind). We translate at
the boundary so the backend math is always in seconds."""
if tier.kind == "call":
return max(0, tier.after_minutes) * 3600
return max(0, tier.after_minutes) * 60
def schedule(wf: Workflow, run: WorkflowRun) -> None:
"""Kick off escalation for a finished run. No-op if the workflow has
only the default notify tier (i.e. nothing to escalate to)."""
tiers = wf.permissions or []
if len(tiers) <= 1:
return
# Cancel any prior task for this run (defense against a re-fire).
cancel(run.id)
task = asyncio.create_task(_runner(wf, run, tiers))
_tasks[run.id] = task
def cancel(run_id: str) -> bool:
task = _tasks.pop(run_id, None)
_state.pop(run_id, None)
if task is None:
return False
task.cancel()
return True
def status(run_id: str) -> Optional[dict]:
return _state.get(run_id)
async def _runner(wf: Workflow, run: WorkflowRun, tiers: list[PermissionTier]) -> None:
from backend.apps.workflows.notifier import send_tier
try:
# Tier 0 is the initial notify; we don't re-fire it here. Walk
# 1..N, sleeping the tier's delay before sending. If the user acks
# via /workflows/runs/{run_id}/ack, the task is cancelled.
for idx in range(1, len(tiers)):
tier = tiers[idx]
delay = _tier_delay_seconds(tier)
fire_at = datetime.now(timezone.utc) + timedelta(seconds=delay)
_state[run.id] = {
"tier_idx": idx,
"tier_kind": tier.kind,
"next_at": fire_at.isoformat(),
}
await asyncio.sleep(delay)
try:
await send_tier(wf, run, tier)
except Exception:
logger.exception("escalation send_tier failed run=%s tier=%s", run.id, tier.kind)
except asyncio.CancelledError:
pass
finally:
_state.pop(run.id, None)
_tasks.pop(run.id, None)
backend/apps/workflows/executor.py
+264
View File
@@ -0,0 +1,264 @@
"""Run a workflow by launching an agent session and feeding it the steps.
The executor is intentionally thin: it leans entirely on agent_manager's
existing launch + send_message path so a scheduled run looks identical to
a manual chat. That keeps the MCP gate, action filtering, provider
routing, retries, and history all aligned with the rest of the app.
"""
import asyncio
import logging
from datetime import datetime, timedelta, timezone
from typing import Optional
from backend.apps.agents.models import AgentConfig
from backend.apps.workflows.models import Workflow, WorkflowRun
from backend.apps.workflows import storage
logger = logging.getLogger(__name__)
# In-process map: workflow_id -> currently running run id. Prevents two
# overlapping fires for the same workflow (e.g. cron tick races a manual
# Run button) without serializing across the whole executor.
_running: dict[str, str] = {}
_running_lock = asyncio.Lock()
def _resolve_system_prompt(wf: Workflow) -> Optional[str]:
if wf.use_synced_prompt:
return None
return wf.system_prompt or None
def _resolve_allowed_tools(wf: Workflow) -> list[str]:
if not wf.actions.freeze:
return []
return list(wf.actions.configured_sets)
def _persist_run_fields(wf: Workflow, run_fields: dict, schedule_runs_count_delta: int = 0) -> None:
"""Merge run-side fields into the current on-disk workflow.
The executor holds the `wf` it was launched with; meanwhile the user
may have PATCHed unrelated fields (title, schedule, permissions...).
Saving our captured `wf` would clobber those edits. Re-read the
authoritative record from storage and only mutate the run-side fields
we own. If the workflow has been deleted while we ran, silently skip
the save so we don't resurrect a deleted record.
schedule_runs_count_delta is a small int (0 or 1) that we add to the
on-disk schedule.runs_count to avoid the same race overwriting an
in-flight bump on the user's PATCH path.
"""
fresh = storage.get_workflow(wf.id)
if fresh is None:
# Deleted while we ran. Don't resurrect.
return
for k, v in run_fields.items():
setattr(fresh, k, v)
if schedule_runs_count_delta:
fresh.schedule.runs_count = fresh.schedule.runs_count + schedule_runs_count_delta
storage.save_workflow(fresh)
def _monthly_spend_so_far(wf: Workflow) -> float:
"""Sum cost_usd across runs of `wf` started in the last 30 days.
Reads the bounded run log (200 rows max per workflow), so this is
O(history) and runs once per fire. Naive datetimes (legacy rows) are
treated as host-local then normalized to UTC by Python's astimezone.
"""
cutoff = datetime.now(timezone.utc) - timedelta(days=30)
total = 0.0
for r in storage.list_runs(wf.id, limit=200):
started = r.started_at
if started is None:
continue
if started.tzinfo is None:
started = started.astimezone(timezone.utc)
else:
started = started.astimezone(timezone.utc)
if started >= cutoff:
total += float(r.cost_usd or 0.0)
return total
async def execute(wf: Workflow, triggered_by: str = "schedule", scheduled_for: Optional[datetime] = None) -> WorkflowRun:
from backend.apps.agents.agent_manager import agent_manager
run = WorkflowRun(
workflow_id=wf.id,
status="running",
scheduled_for=scheduled_for,
started_at=datetime.now(),
triggered_by=triggered_by,
)
# Cost cap pre-check happens before claiming `_running` so a capped
# workflow doesn't block its own next fire. We still record the run so
# the user sees it in History with a clear reason.
if wf.cost_cap_usd_monthly is not None:
spent = _monthly_spend_so_far(wf)
if spent >= wf.cost_cap_usd_monthly:
run.status = "skipped"
run.error = f"Monthly cost cap reached (${spent:.2f} / ${wf.cost_cap_usd_monthly:.2f})"
run.finished_at = datetime.now()
storage.record_run(run)
_persist_run_fields(wf, {
"last_run_at": run.finished_at,
"last_run_status": "skipped",
"last_run_id": run.id,
})
return run
storage.record_run(run)
async with _running_lock:
if wf.id in _running:
run.status = "skipped"
run.error = "Previous run still active"
run.finished_at = datetime.now()
storage.record_run(run)
return run
_running[wf.id] = run.id
wf.last_run_at = run.started_at
wf.last_run_status = "running"
wf.last_run_id = run.id
_persist_run_fields(wf, {
"last_run_at": run.started_at,
"last_run_status": "running",
"last_run_id": run.id,
})
session = None
try:
steps = [s.text for s in wf.steps if s.text and s.text.strip()]
if not steps:
raise ValueError("Workflow has no steps")
config = AgentConfig(
name=wf.title or "Workflow",
model=wf.model or "sonnet",
mode=wf.mode or "agent",
provider=wf.provider or "anthropic",
system_prompt=_resolve_system_prompt(wf),
allowed_tools=_resolve_allowed_tools(wf) or [
"Read", "Edit", "Write", "Bash", "Glob", "Grep", "AskUserQuestion",
],
dashboard_id=wf.dashboard_id,
)
session = await agent_manager.launch_agent(config)
run.session_id = session.id
storage.record_run(run)
# Send each step sequentially. agent_manager.send_message is a no-op
# while a prior turn is still streaming, so we await until the
# session is idle before posting the next step. Keeps the runner
# safe regardless of how long each turn takes.
step_error: Optional[str] = None
for step in steps:
await agent_manager.send_message(session.id, step)
await _await_session_idle(session.id)
sess_state = agent_manager.sessions.get(session.id)
if sess_state is not None and getattr(sess_state, "status", None) == "error":
step_error = "Agent session entered error state"
break
run.finished_at = datetime.now()
sess_state = agent_manager.sessions.get(session.id)
if sess_state is not None:
run.cost_usd = float(getattr(sess_state, "cost_usd", 0.0) or 0.0)
if step_error is not None:
run.status = "failure"
run.error = step_error
wf.last_run_status = "failure"
elif scheduled_for is not None and (run.finished_at.replace(tzinfo=None) - scheduled_for.replace(tzinfo=None)).total_seconds() > 300:
# Started more than 5 minutes after its slot (app was closed,
# event loop backed up, etc.). Surface in History as ran_late
# so the user can tell apart "fired on time" from "caught up".
# Strip tz before the subtraction so a UTC-aware scheduled_for
# (new code path) and a naive finished_at don't raise.
run.status = "ran_late"
wf.last_run_status = "ran_late"
else:
run.status = "success"
wf.last_run_status = "success"
# Bump runs_count for scheduled fires that reached a terminal state
# other than "skipped". Manual runs don't count against max_runs.
runs_delta = 1 if (triggered_by == "schedule" and run.status in ("success", "ran_late", "failure")) else 0
storage.record_run(run)
wf.last_run_at = run.finished_at
_persist_run_fields(wf, {
"last_run_at": run.finished_at,
"last_run_status": wf.last_run_status,
}, schedule_runs_count_delta=runs_delta)
except Exception as e:
logger.exception("Workflow run failed: %s", e)
run.status = "failure"
run.error = str(e)[:500]
run.finished_at = datetime.now()
storage.record_run(run)
wf.last_run_status = "failure"
_persist_run_fields(wf, {
"last_run_status": "failure",
"last_run_at": run.finished_at,
})
finally:
# Close the workflow's agent session so closed_at is set and the
# run shows up in chat history (get_history sorts by closed_at;
# sessions with closed_at=None sort to the bottom and fall off
# the first page). close_session also drops in-memory state and
# persists the final snapshot to disk.
if session is not None:
try:
await agent_manager.close_session(session.id)
except Exception:
logger.exception("close_session failed for workflow run %s", run.id)
async with _running_lock:
_running.pop(wf.id, None)
try:
from backend.apps.workflows.notifier import notify_run_complete
await notify_run_complete(wf, run)
except Exception:
logger.debug("notifier failed", exc_info=True)
try:
from backend.apps.agents.ws_manager import ws_manager
await ws_manager.broadcast_global("workflow:run", {
"workflow_id": wf.id,
"run": run.model_dump(mode="json"),
})
except Exception:
pass
return run
async def _await_session_idle(session_id: str, timeout_s: float = 600.0) -> None:
"""Block until the agent session reaches a non-running terminal state.
Polls cheaply (50ms) since the agent_manager doesn't expose a per-session
completion future. Bounded by timeout_s so a stuck step doesn't hang the
runner forever.
"""
from backend.apps.agents.agent_manager import agent_manager
deadline = asyncio.get_event_loop().time() + timeout_s
while True:
sess = agent_manager.sessions.get(session_id)
if not sess:
return
task = agent_manager.tasks.get(session_id)
if task is not None and task.done():
return
status = getattr(sess, "status", None)
if status in ("completed", "error", "stopped"):
return
if asyncio.get_event_loop().time() > deadline:
raise TimeoutError(f"Step exceeded {timeout_s}s on session {session_id}")
await asyncio.sleep(0.05)
backend/apps/workflows/models.py
+145
View File
@@ -0,0 +1,145 @@
from pydantic import BaseModel, ConfigDict, Field, field_validator
from typing import Optional, Literal, Any
from datetime import datetime
from uuid import uuid4
# Each "tier" in the permission chain: notify in app, fall through to text
# after N minutes if no response, then to call after a further N minutes/hours.
# Matches images 17 to 19 (Schedule edit). Order in the list = escalation order.
class PermissionTier(BaseModel):
kind: Literal["notify", "text", "call"] = "notify"
after_minutes: int = 0
phone: Optional[str] = None
class ScheduleConfig(BaseModel):
enabled: bool = False
# Bounds keep the scheduler from blowing up on malformed input. The
# FE clamps these too, but defense-in-depth: a misbehaving agent
# tool, an old JSON file, or a curl-wielding power user shouldn't
# be able to crash _next_fire_after by passing hour=99.
repeat_every: int = Field(default=1, ge=1, le=365)
repeat_unit: Literal["day", "week", "month"] = "week"
on_days: list[int] = Field(default_factory=list)
hour: int = Field(default=9, ge=0, le=23)
minute: int = Field(default=0, ge=0, le=59)
# IANA zone name (e.g. "America/Los_Angeles") or "local" for legacy
# records that predate explicit tz. storage._load_all_from_disk coerces
# "local" to the host zone in memory; we leave it on disk until the
# user's next save so backup/sync tools don't see spurious churn.
timezone: str = "local"
on_missed: Literal["skip", "run_once", "run_all"] = "skip"
# Optional end conditions. None = forever / unbounded. Schedule auto-
# disables once either is satisfied; scheduler._tick zeroes out
# next_run_at and flips enabled=False so the UI reflects reality.
ends_at: Optional[datetime] = None
max_runs: Optional[int] = Field(default=None, ge=1)
runs_count: int = Field(default=0, ge=0)
@field_validator("on_days")
@classmethod
def _clean_on_days(cls, v: list[int]) -> list[int]:
# Backend uses JS-style weekday (Sun=0..Sat=6). Drop entries
# outside that range so a malformed PATCH can't trip the
# scheduler later, and dedupe while preserving order.
seen: set[int] = set()
out: list[int] = []
for d in v or []:
if isinstance(d, int) and 0 <= d <= 6 and d not in seen:
seen.add(d)
out.append(d)
return out
class ActionsConfig(BaseModel):
prevent_unused: bool = False
freeze: bool = False
configured_sets: list[str] = Field(default_factory=list)
class WorkflowStep(BaseModel):
id: str = Field(default_factory=lambda: uuid4().hex)
text: str = ""
class Workflow(BaseModel):
# validate_assignment is load-bearing for the PATCH /workflows/{id} path
# (workflows.py:update_workflow setattr's raw dicts from body.model_dump
# straight onto the cached Workflow). Without coercion the nested
# schedule/steps/actions/permissions fields become plain dicts in
# memory, and every downstream call; scheduler tick, executor.execute,
# subsequent PATCHes; crashes on `.enabled` / `.text`.
model_config = ConfigDict(validate_assignment=True)
id: str = Field(default_factory=lambda: uuid4().hex)
title: str = "Untitled workflow"
description: str = ""
icon: str = ""
system_prompt: Optional[str] = None
use_synced_prompt: bool = True
steps: list[WorkflowStep] = Field(default_factory=list)
actions: ActionsConfig = Field(default_factory=ActionsConfig)
schedule: ScheduleConfig = Field(default_factory=ScheduleConfig)
permissions: list[PermissionTier] = Field(
default_factory=lambda: [PermissionTier(kind="notify")]
)
source_session_id: Optional[str] = None
dashboard_id: Optional[str] = None
model: str = "sonnet"
mode: str = "agent"
provider: str = "anthropic"
created_at: datetime = Field(default_factory=datetime.now)
updated_at: datetime = Field(default_factory=datetime.now)
last_run_at: Optional[datetime] = None
last_run_status: Optional[Literal["success", "failure", "ran_late", "running", "skipped"]] = None
last_run_id: Optional[str] = None
next_run_at: Optional[datetime] = None
cost_cap_usd_monthly: Optional[float] = None
class WorkflowRun(BaseModel):
id: str = Field(default_factory=lambda: uuid4().hex)
workflow_id: str
status: Literal["running", "success", "failure", "ran_late", "skipped"] = "running"
scheduled_for: Optional[datetime] = None
started_at: datetime = Field(default_factory=datetime.now)
finished_at: Optional[datetime] = None
session_id: Optional[str] = None
error: Optional[str] = None
cost_usd: float = 0.0
triggered_by: Literal["schedule", "manual", "retry"] = "schedule"
class WorkflowCreate(BaseModel):
title: str = "Untitled workflow"
description: str = ""
icon: str = ""
system_prompt: Optional[str] = None
use_synced_prompt: bool = True
steps: list[WorkflowStep] = Field(default_factory=list)
actions: ActionsConfig = Field(default_factory=ActionsConfig)
schedule: ScheduleConfig = Field(default_factory=ScheduleConfig)
permissions: Optional[list[PermissionTier]] = None
source_session_id: Optional[str] = None
dashboard_id: Optional[str] = None
model: Optional[str] = None
mode: Optional[str] = None
provider: Optional[str] = None
cost_cap_usd_monthly: Optional[float] = None
class WorkflowUpdate(BaseModel):
title: Optional[str] = None
description: Optional[str] = None
icon: Optional[str] = None
system_prompt: Optional[str] = None
use_synced_prompt: Optional[bool] = None
steps: Optional[list[WorkflowStep]] = None
actions: Optional[ActionsConfig] = None
schedule: Optional[ScheduleConfig] = None
permissions: Optional[list[PermissionTier]] = None
model: Optional[str] = None
mode: Optional[str] = None
provider: Optional[str] = None
cost_cap_usd_monthly: Optional[float] = None
backend/apps/workflows/notifier.py
+55
View File
@@ -0,0 +1,55 @@
"""Permission/escalation chain notifier.
The notify tier broadcasts a ws event the renderer picks up. The text/call
tiers route through the cloud SMS bridge once enabled; until it's enabled
we fall back to an extra ws notify with a `fallback: true` marker so the
renderer can label it honestly ("Text-me fallback: cloud SMS not wired").
The *when* of escalation is owned by apps/workflows/escalation.py.
"""
import logging
from datetime import datetime
from backend.apps.workflows.models import PermissionTier, Workflow, WorkflowRun
logger = logging.getLogger(__name__)
def _base_payload(wf: Workflow, run: WorkflowRun) -> dict:
return {
"workflow_id": wf.id,
"workflow_title": wf.title,
"run_id": run.id,
"status": run.status,
"session_id": run.session_id,
"started_at": run.started_at.isoformat() if isinstance(run.started_at, datetime) else run.started_at,
"finished_at": run.finished_at.isoformat() if isinstance(run.finished_at, datetime) else run.finished_at,
}
async def notify_run_complete(wf: Workflow, run: WorkflowRun) -> None:
from backend.apps.agents.ws_manager import ws_manager
from backend.apps.workflows import escalation
payload = _base_payload(wf, run)
await ws_manager.broadcast_global("workflow:notify", payload)
# Kick off server-side escalation only if there are additional tiers
# beyond the default notify. The escalation runner will sleep + call
# send_tier per tier.
escalation.schedule(wf, run)
async def send_tier(wf: Workflow, run: WorkflowRun, tier: PermissionTier) -> None:
"""Send a single escalation tier. Today the text/call paths fall back
to an in-app notify with `fallback: true` and the tier kind set so the
renderer can show "Text-me fallback (cloud SMS not wired)."
"""
from backend.apps.agents.ws_manager import ws_manager
payload = _base_payload(wf, run)
payload["tier_kind"] = tier.kind
payload["tier_phone"] = (tier.phone or "")[-4:] if tier.phone else None
payload["fallback"] = True # flip to False once the cloud SMS bridge is wired
await ws_manager.broadcast_global("workflow:notify", payload)
logger.info("workflow tier=%s fallback fired wf=%s run=%s", tier.kind, wf.id, run.id)
backend/apps/workflows/scheduler.py
+352
View File
@@ -0,0 +1,352 @@
"""In-process cron-style scheduler.
One long-lived asyncio task wakes on the next-due workflow boundary, fires
matching workflows, then re-computes. We deliberately avoid one-task-per-
workflow (turns rescheduling into a thundering re-spawn problem). On
startup we walk persisted workflows once, decide what to do about missed
fires via on_missed, and queue each.
Schedule semantics:
unit=day: fires every repeat_every days at hour:minute
unit=week: fires on the listed weekday(s) every repeat_every weeks
unit=month: fires on the original day-of-month every repeat_every months
Wall-clock math runs in the workflow's IANA timezone, then we convert to
UTC at the boundary. This is the only safe way to honor DST (a "9am
Monday" schedule must remain 9am local across spring-forward / fall-back).
Legacy records with timezone="local" are coerced to the host zone in
memory by storage._load_all_from_disk; the on-disk file is not rewritten
until the user's next save.
"""
import asyncio
import calendar
import logging
import os
from datetime import datetime, timedelta, timezone
from typing import Optional
from zoneinfo import ZoneInfo, ZoneInfoNotFoundError
from backend.apps.workflows.models import Workflow, ScheduleConfig
from backend.apps.workflows import storage, executor
logger = logging.getLogger(__name__)
_loop_task: Optional[asyncio.Task] = None
_wake = asyncio.Event()
_host_tz_cache: Optional[ZoneInfo] = None
def _host_tz() -> ZoneInfo:
global _host_tz_cache
if _host_tz_cache is not None:
return _host_tz_cache
name = os.environ.get("OPENSWARM_TIMEZONE", "").strip()
if not name:
try:
from tzlocal import get_localzone_name # type: ignore
name = get_localzone_name() or ""
except Exception:
name = ""
try:
_host_tz_cache = ZoneInfo(name) if name else ZoneInfo("UTC")
except ZoneInfoNotFoundError:
_host_tz_cache = ZoneInfo("UTC")
return _host_tz_cache
def _resolve_tz(tz: str) -> ZoneInfo:
if not tz or tz == "local":
return _host_tz()
try:
return ZoneInfo(tz)
except ZoneInfoNotFoundError:
return _host_tz()
def _as_utc(dt: Optional[datetime]) -> Optional[datetime]:
"""Normalize an arbitrary stored datetime to aware-UTC.
Pydantic deserializes naive ISO strings as naive datetimes. Treat such
values as host-local (matches the pre-tz codepath that wrote them) so
comparisons against datetime.now(timezone.utc) don't raise.
"""
if dt is None:
return None
if dt.tzinfo is None:
return dt.replace(tzinfo=_host_tz()).astimezone(timezone.utc)
return dt.astimezone(timezone.utc)
def _add_months(dt: datetime, months: int) -> datetime:
"""Add months preserving day-of-month, clamping only if the target month
is shorter (e.g. Jan 31 + 1mo → Feb 28/29). Wall-clock arithmetic; the
caller is responsible for tz attachment.
"""
total = dt.month - 1 + months
year = dt.year + total // 12
month = total % 12 + 1
day = min(dt.day, calendar.monthrange(year, month)[1])
return dt.replace(year=year, month=month, day=day)
def _js_weekday(d: datetime) -> int:
"""Frontend uses JS getDay() convention (Sun=0..Sat=6). Python's
datetime.weekday() is Mon=0..Sun=6. Wire format stays JS-style so the
on_days array round-trips between FE and BE without translation in two
places."""
return (d.weekday() + 1) % 7
def _next_fire_after(sched: ScheduleConfig, ref_utc: datetime) -> Optional[datetime]:
if not sched.enabled:
return None
tz = _resolve_tz(sched.timezone)
ref_local = ref_utc.astimezone(tz)
base = ref_local.replace(second=0, microsecond=0)
candidate = base.replace(hour=sched.hour, minute=sched.minute)
if candidate <= ref_local:
candidate = candidate + timedelta(days=1)
if sched.repeat_unit == "day":
step = max(1, sched.repeat_every)
# Walk forward in step-day increments until we find a slot strictly
# after `ref_local`. Cheap because step is small.
while candidate <= ref_local:
candidate = candidate + timedelta(days=step)
return candidate.astimezone(timezone.utc)
if sched.repeat_unit == "week":
allowed = sched.on_days or [_js_weekday(ref_local)]
for _ in range(0, 14):
if _js_weekday(candidate) in allowed and candidate > ref_local:
return candidate.astimezone(timezone.utc)
candidate = candidate + timedelta(days=1)
return candidate.astimezone(timezone.utc)
if sched.repeat_unit == "month":
target_day = ref_local.day
step = max(1, sched.repeat_every)
c = candidate.replace(day=min(target_day, calendar.monthrange(candidate.year, candidate.month)[1]))
while c <= ref_local:
c = _add_months(c, step)
return c.astimezone(timezone.utc)
return None
def compute_next_fire(wf: Workflow, ref: Optional[datetime] = None) -> Optional[datetime]:
ref_utc = _as_utc(ref) if ref is not None else datetime.now(timezone.utc)
return _next_fire_after(wf.schedule, ref_utc)
def fires_in_window(wf: Workflow, days: int = 30) -> int:
"""Count fires from now through `days` days from now. Used by the
cost-estimate response. Honors end conditions so the projection doesn't
over-count after ends_at or max_runs. Caps the walk at 1000 fires to
guard pathological sub-day schedules (none today, but cheap insurance).
"""
sched = wf.schedule
if not sched.enabled:
return 0
if sched.max_runs is not None and sched.runs_count >= sched.max_runs:
return 0
cursor_utc = datetime.now(timezone.utc)
end_utc = cursor_utc + timedelta(days=days)
ends_at_utc = _as_utc(sched.ends_at)
if ends_at_utc is not None and ends_at_utc < end_utc:
end_utc = ends_at_utc
remaining_budget = (
sched.max_runs - sched.runs_count if sched.max_runs is not None else 1000
)
count = 0
while count < min(1000, remaining_budget):
nxt = _next_fire_after(sched, cursor_utc)
if nxt is None or nxt > end_utc:
break
count += 1
cursor_utc = nxt
return count
def kick() -> None:
_wake.set()
def _end_condition_hit(wf: Workflow, now_utc: datetime) -> bool:
s = wf.schedule
ends_at = _as_utc(s.ends_at)
if ends_at is not None and now_utc >= ends_at:
return True
if s.max_runs is not None and s.runs_count >= s.max_runs:
return True
return False
def _disable_schedule(wf: Workflow) -> None:
wf.schedule.enabled = False
wf.next_run_at = None
storage.save_workflow(wf)
async def _tick() -> None:
now_utc = datetime.now(timezone.utc)
if storage.get_paused():
return
due: list[Workflow] = []
for wf in storage.list_workflows():
if not wf.schedule.enabled:
continue
if _end_condition_hit(wf, now_utc):
_disable_schedule(wf)
continue
nra = _as_utc(wf.next_run_at)
if nra and nra <= now_utc:
due.append(wf)
for wf in due:
scheduled_for = _as_utc(wf.next_run_at)
nxt = _next_fire_after(wf.schedule, now_utc)
wf.next_run_at = nxt
storage.save_workflow(wf)
asyncio.create_task(_fire(wf, scheduled_for=scheduled_for))
async def _fire(wf: Workflow, scheduled_for: Optional[datetime]) -> None:
try:
await executor.execute(wf, triggered_by="schedule", scheduled_for=scheduled_for)
except Exception:
logger.exception("scheduler fire failed for workflow=%s", wf.id)
def _seconds_until_next() -> float:
now_utc = datetime.now(timezone.utc)
soonest: Optional[datetime] = None
for wf in storage.list_workflows():
if not wf.schedule.enabled:
continue
nra = _as_utc(wf.next_run_at)
if nra is None:
continue
if soonest is None or nra < soonest:
soonest = nra
if soonest is None:
return 60.0
delta = (soonest - now_utc).total_seconds()
return max(1.0, min(delta, 60.0))
async def _loop() -> None:
logger.info("workflow scheduler loop started")
while True:
try:
await _tick()
except Exception:
logger.exception("scheduler tick error")
try:
await asyncio.wait_for(_wake.wait(), timeout=_seconds_until_next())
except asyncio.TimeoutError:
pass
_wake.clear()
def _mark_stuck_runs_failed() -> None:
"""Any run marked 'running' that survives a backend restart is dead.
The owning event loop is gone, so there's no way to resume. Mark it
failed once at startup instead of letting the History tab show a
forever-spinning row that misleads the user.
"""
now = datetime.now()
for wf in storage.list_workflows():
for r in storage.list_runs(wf.id, limit=200):
if r.status == "running":
storage.update_run(
r.id,
status="failure",
error="OpenSwarm closed before this run finished.",
finished_at=now,
)
def reconcile_on_startup() -> None:
"""Walk persisted workflows once and resolve missed fires per policy.
Missed-run policies:
skip -> roll forward to next future fire, ignore missed
run_once -> if any fires were missed, schedule a single catch-up at now
run_all -> not actually run_all in v1 (would burn tokens); same as run_once
but we mark the run.status as ran_late so the UI surfaces it
"""
now_utc = datetime.now(timezone.utc)
for wf in storage.list_workflows():
if not wf.schedule.enabled:
wf.next_run_at = None
storage.save_workflow(wf)
continue
if _end_condition_hit(wf, now_utc):
_disable_schedule(wf)
continue
nra = _as_utc(wf.next_run_at)
missed = bool(nra and nra <= now_utc)
if missed and wf.schedule.on_missed in ("run_once", "run_all"):
# Keep next_run_at <= now_utc so the very next tick fires it.
# Normalize to a UTC-aware value so future comparisons don't
# trip on naive legacy datetimes.
wf.next_run_at = nra
storage.save_workflow(wf)
else:
wf.next_run_at = _next_fire_after(wf.schedule, now_utc)
storage.save_workflow(wf)
async def start() -> None:
global _loop_task
if _loop_task is not None:
return
_mark_stuck_runs_failed()
reconcile_on_startup()
_loop_task = asyncio.create_task(_loop())
async def stop() -> None:
global _loop_task
if _loop_task is None:
return
_loop_task.cancel()
try:
await _loop_task
except (asyncio.CancelledError, Exception):
pass
_loop_task = None
def list_active() -> list[dict]:
"""Snapshot of currently-running workflow runs.
Reads executor._running (workflow_id -> run_id) and joins against the
workflow cache for titles. Used by GET /workflows/active so the tray
and the auto-updater veto can both ask "are any runs in flight?"
without holding the executor lock.
"""
out: list[dict] = []
snapshot = dict(executor._running)
for wid, run_id in snapshot.items():
wf = storage.get_workflow(wid)
title = wf.title if wf else ""
started_at = None
if wf:
for r in storage.list_runs(wid, limit=10):
if r.id == run_id:
started_at = r.started_at.isoformat() if isinstance(r.started_at, datetime) else r.started_at
break
out.append({
"workflow_id": wid,
"run_id": run_id,
"title": title,
"started_at": started_at,
})
return out
backend/apps/workflows/storage.py
+197
View File
@@ -0,0 +1,197 @@
"""On-disk store for workflows + workflow runs.
Layout under DATA_ROOT/workflows/:
<id>.json workflow record
runs/<workflow_id>.json bounded log (latest N) of runs for that workflow
A separate runs file per workflow keeps history reads O(history size) instead
of O(total runs across all workflows). The workflow record only carries
last_run_* / next_run_at summary fields; full history lives in the runs file.
"""
import json
import os
from threading import Lock
from typing import Optional
from backend.config.paths import DATA_ROOT
from backend.apps.workflows.models import Workflow, WorkflowRun
DATA_DIR = os.path.join(DATA_ROOT, "workflows")
RUNS_DIR = os.path.join(DATA_DIR, "runs")
PAUSED_FILE = os.path.join(DATA_DIR, "paused.json")
_io_lock = Lock()
_workflow_cache: dict[str, Workflow] = {}
_runs_cache: dict[str, list[WorkflowRun]] = {}
_cache_loaded = False
_paused = False
def _resolve_host_tz_name() -> str:
"""Best-effort IANA name for the host. Mirrors apps/service/client.py."""
name = os.environ.get("OPENSWARM_TIMEZONE", "").strip()
if not name:
try:
from tzlocal import get_localzone_name # type: ignore
name = get_localzone_name() or ""
except Exception:
name = ""
return name or "UTC"
# Keep this much run history per workflow on disk. Older runs are pruned;
# the History tab caps at ~20 anyway, and unbounded growth turned the JSON
# read into a real cost on hot-reload of the schedule page.
RUNS_PER_WORKFLOW = 200
def _ensure_dirs() -> None:
os.makedirs(DATA_DIR, exist_ok=True)
os.makedirs(RUNS_DIR, exist_ok=True)
def _wf_path(wid: str) -> str:
return os.path.join(DATA_DIR, f"{wid}.json")
def _runs_path(wid: str) -> str:
return os.path.join(RUNS_DIR, f"{wid}.json")
def _load_all_from_disk() -> None:
global _cache_loaded, _paused
_ensure_dirs()
_workflow_cache.clear()
_runs_cache.clear()
host_tz = _resolve_host_tz_name()
for fname in os.listdir(DATA_DIR):
if not fname.endswith(".json") or fname == "paused.json":
continue
try:
with open(os.path.join(DATA_DIR, fname)) as f:
wf = Workflow(**json.load(f))
# Coerce legacy timezone="local" to the host IANA zone in
# memory only. We don't rewrite the file here so backup/sync
# tooling doesn't see mtime churn on every startup; the next
# user-driven save migrates the on-disk record naturally.
if wf.schedule.timezone == "local":
wf.schedule.timezone = host_tz
_workflow_cache[wf.id] = wf
except Exception:
continue
if os.path.exists(RUNS_DIR):
for fname in os.listdir(RUNS_DIR):
if not fname.endswith(".json"):
continue
wid = fname[:-5]
try:
with open(os.path.join(RUNS_DIR, fname)) as f:
arr = json.load(f)
_runs_cache[wid] = [WorkflowRun(**r) for r in arr]
except Exception:
_runs_cache[wid] = []
# Load the global pause flag if it's been set previously.
if os.path.exists(PAUSED_FILE):
try:
with open(PAUSED_FILE) as f:
_paused = bool(json.load(f).get("paused", False))
except Exception:
_paused = False
_cache_loaded = True
def init() -> None:
with _io_lock:
_load_all_from_disk()
def list_workflows() -> list[Workflow]:
if not _cache_loaded:
init()
return list(_workflow_cache.values())
def get_workflow(wid: str) -> Optional[Workflow]:
if not _cache_loaded:
init()
return _workflow_cache.get(wid)
def save_workflow(wf: Workflow) -> Workflow:
with _io_lock:
_ensure_dirs()
_workflow_cache[wf.id] = wf
with open(_wf_path(wf.id), "w") as f:
json.dump(wf.model_dump(mode="json"), f, indent=2)
return wf
def delete_workflow(wid: str) -> bool:
with _io_lock:
existed = wid in _workflow_cache
_workflow_cache.pop(wid, None)
_runs_cache.pop(wid, None)
wf_file = _wf_path(wid)
if os.path.exists(wf_file):
os.remove(wf_file)
rf = _runs_path(wid)
if os.path.exists(rf):
os.remove(rf)
return existed
def list_runs(wid: str, limit: int = 50) -> list[WorkflowRun]:
if not _cache_loaded:
init()
runs = _runs_cache.get(wid, [])
return runs[-limit:][::-1]
def record_run(run: WorkflowRun) -> WorkflowRun:
with _io_lock:
_ensure_dirs()
arr = _runs_cache.setdefault(run.workflow_id, [])
# Replace prior entry with same id if we're updating an in-flight run.
for i, prior in enumerate(arr):
if prior.id == run.id:
arr[i] = run
break
else:
arr.append(run)
# Bound the per-workflow history to keep disk + memory cheap.
if len(arr) > RUNS_PER_WORKFLOW:
del arr[: len(arr) - RUNS_PER_WORKFLOW]
with open(_runs_path(run.workflow_id), "w") as f:
json.dump([r.model_dump(mode="json") for r in arr], f, indent=2)
return run
def get_paused() -> bool:
if not _cache_loaded:
init()
return _paused
def set_paused(value: bool) -> bool:
global _paused
with _io_lock:
_ensure_dirs()
_paused = bool(value)
with open(PAUSED_FILE, "w") as f:
json.dump({"paused": _paused}, f)
return _paused
def update_run(run_id: str, **fields) -> Optional[WorkflowRun]:
if not _cache_loaded:
init()
for arr in _runs_cache.values():
for i, r in enumerate(arr):
if r.id == run_id:
updated = r.model_copy(update=fields)
arr[i] = updated
with _io_lock:
with open(_runs_path(updated.workflow_id), "w") as f:
json.dump([x.model_dump(mode="json") for x in arr], f, indent=2)
return updated
return None
backend/apps/workflows/workflows.py
+441
View File
@@ -0,0 +1,441 @@
import asyncio
import logging
from contextlib import asynccontextmanager
from datetime import datetime
from typing import Optional
from fastapi import HTTPException, Header, Request
from backend.config.Apps import SubApp
from backend.apps.workflows.models import (
Workflow,
WorkflowCreate,
WorkflowUpdate,
WorkflowRun,
)
from backend.apps.workflows import storage, scheduler, executor, audit, escalation
logger = logging.getLogger(__name__)
def _scan_cron_for_openswarm() -> list[str]:
"""Surface OS-level scheduled-task entries that reference us.
macOS + Linux: read `crontab -l`. Windows: query `schtasks` for any
task whose command/path contains 'openswarm'. Best-effort across all
three; any failure (no tool installed, permission denied, parse
error) just returns []. Surfaced to the FE so the Workflows hub can
offer a one-click migration banner to convert into native workflows.
"""
import subprocess
import platform as _platform
findings: list[str] = []
if _platform.system() == "Windows":
try:
proc = subprocess.run(
["schtasks", "/query", "/fo", "CSV", "/v"],
capture_output=True, text=True, timeout=4,
)
if proc.returncode != 0:
return []
for line in (proc.stdout or "").splitlines():
if "openswarm" in line.lower() and not line.lstrip().startswith('"#'):
findings.append(line.strip())
except Exception:
return []
return findings
# macOS + Linux
try:
proc = subprocess.run(
["crontab", "-l"],
capture_output=True, text=True, timeout=2,
)
if proc.returncode != 0:
return []
out = proc.stdout or ""
return [line.strip() for line in out.splitlines() if "openswarm" in line.lower() and not line.strip().startswith("#")]
except Exception:
return []
_cron_findings: list[str] = []
@asynccontextmanager
async def workflows_lifespan():
storage.init()
await scheduler.start()
# Cheap one-shot scan for prior cron entries that reference us. We
# don't migrate automatically; the FE shows a banner with a "Convert
# to OpenSwarm scheduled tasks" button so the user is in control.
global _cron_findings
_cron_findings = _scan_cron_for_openswarm()
try:
yield
finally:
await scheduler.stop()
workflows = SubApp("workflows", workflows_lifespan)
def _derive_icon(wf: Workflow) -> str:
"""Cheap icon hint used until proper auto-icon generation lands.
Pull the first emoji from the title, falling back to the first
letter. Keeps the Search list (image 2 annotation) populated without
waiting on the LLM-based icon generator.
"""
title = (wf.title or "").strip()
for ch in title:
if ord(ch) > 0x2700:
return ch
if title:
return title[:1].upper()
return "W"
@workflows.router.get("/list")
async def list_workflows(dashboard_id: Optional[str] = None):
items = storage.list_workflows()
if dashboard_id:
items = [w for w in items if not w.dashboard_id or w.dashboard_id == dashboard_id]
items.sort(key=lambda w: w.updated_at or w.created_at, reverse=True)
# Enrich with cost_estimate so calendar tooltips and the WorkflowsHub
# list don't have to round-trip to GET /workflows/{id} per row. Cheap
# because fires_in_window walks at most ~30 fires per workflow.
return {"workflows": [_enriched(w) for w in items]}
@workflows.router.post("/create")
async def create_workflow(body: WorkflowCreate):
actions = body.actions
# Scheduled workflows default to freeze=on for safety. The user can
# flip "Full agent access" in the editor with an explicit confirm.
# Source-session creates inherit the chat's tool choices so we leave
# them alone there (the source session itself already vetted the
# blast radius).
if body.schedule.enabled and not actions.freeze and not body.source_session_id:
actions = actions.model_copy(update={"freeze": True})
wf = Workflow(
title=body.title,
description=body.description,
icon=body.icon,
system_prompt=body.system_prompt,
use_synced_prompt=body.use_synced_prompt,
steps=body.steps,
actions=actions,
schedule=body.schedule,
permissions=body.permissions or [],
source_session_id=body.source_session_id,
dashboard_id=body.dashboard_id,
model=body.model or "sonnet",
mode=body.mode or "agent",
provider=body.provider or "anthropic",
cost_cap_usd_monthly=body.cost_cap_usd_monthly,
)
if not wf.icon:
wf.icon = _derive_icon(wf)
if wf.schedule.enabled:
wf.next_run_at = scheduler.compute_next_fire(wf)
# Force-generate title + description from the steps in a single aux
# call. Previously we only filled missing description, leaving stale
# session names ("Inbox check") as titles. One round-trip, both
# fields, overwrites whatever shallow draft the FE sent.
try:
title, description = await _generate_title_and_description(wf)
if title:
wf.title = title
if description:
wf.description = description
except Exception:
pass
storage.save_workflow(wf)
scheduler.kick()
return _enriched(wf)
async def _generate_title_and_description(wf: Workflow) -> tuple[str, str]:
"""Single aux-model call returning (title, description).
Uses strict JSON output so both fields come back in one round-trip.
Returns ("", "") on any failure so the caller can write back
unconditionally without dropping the workflow create.
"""
if not wf.steps:
return "", ""
try:
from backend.apps.agents.providers.registry import resolve_aux_model
from backend.apps.agents.providers.registry import get_anthropic_client_for_model
from backend.apps.settings.settings import load_settings as _ls
except Exception:
return "", ""
settings = _ls()
try:
aux_model, _ = await resolve_aux_model(settings, preferred_tier="haiku")
client = get_anthropic_client_for_model(settings, aux_model)
except Exception:
return "", ""
steps_lines = "\n".join(f"{i+1}. {s.text}" for i, s in enumerate(wf.steps) if s.text)
prompt = (
"You name and describe a saved automation routine that the user "
"can re-run later. The routine is defined ONLY by the numbered "
"steps below; treat those as the user's instructions to the "
"agent.\n\n"
"Return STRICT JSON, nothing else, no code fence:\n"
" {\"title\": string, \"description\": string}\n\n"
"title rules:\n"
"- 2 to 5 words, Title Case\n"
"- Starts with a verb-noun pair when possible (e.g. \"Summarize "
"Daily Emails\")\n"
"- No emoji, no quotes, no trailing punctuation\n\n"
"description rules:\n"
"- 1 to 2 sentences, under 30 words total\n"
"- Describes the concrete WORK the routine performs for the user, "
"not metadata about itself. Examples of GOOD output:\n"
" \"Reads recent Gmail, ranks urgency, and emails you a PDF "
"digest each Sunday at 9am.\"\n"
" \"Pulls today's calendar plus inbox, writes a Notion brief, "
"and texts you the link.\"\n"
"- Examples of BAD output you MUST AVOID verbatim:\n"
" \"This is an AI-generated description...\"\n"
" \"Auto-generated description used to wrap workflows...\"\n"
" Any sentence that talks about the description itself\n"
"- Start with a verb. Do NOT start with \"This\", \"A\", \"An\", "
"\"The workflow\", \"This routine\".\n\n"
f"Steps:\n{steps_lines}"
)
import json
import re as _re
def _extract_json_object(s: str) -> Optional[dict]:
"""Find the first {...} block and json.loads it. Handles code
fences, prose preambles, and trailing chatter that some aux
models like to add."""
s = s.strip()
if s.startswith("```"):
s = _re.sub(r"^```(?:json)?\s*", "", s, flags=_re.IGNORECASE)
s = _re.sub(r"\s*```\s*$", "", s)
# Greedy brace match; falls through to direct json.loads if no
# braces are visible at all.
start = s.find("{")
end = s.rfind("}")
if start != -1 and end != -1 and end > start:
s = s[start : end + 1]
try:
return json.loads(s)
except Exception:
return None
try:
# Prefill the assistant turn with `{` so the model is steered into
# emitting JSON from the first token. The Anthropic API treats a
# trailing assistant message as a prefill; we'll glue it back on
# before parsing.
resp = await client.messages.create(
model=aux_model,
max_tokens=240,
messages=[
{"role": "user", "content": prompt},
{"role": "assistant", "content": "{"},
],
)
text = ""
if isinstance(resp.content, list):
for block in resp.content:
if getattr(block, "type", None) == "text":
text += getattr(block, "text", "")
raw = "{" + text.strip() if not text.strip().startswith("{") else text.strip()
data = _extract_json_object(raw)
if not data:
logger.warning("description gen: failed to parse aux model output: %s", raw[:400])
return "", ""
title = (data.get("title") or "").strip()[:80]
description = (data.get("description") or "").strip()[:500]
if not description:
logger.warning("description gen: empty description from aux model. Raw: %s", raw[:400])
return title, description
except Exception as e:
logger.warning("description gen: aux model call failed: %s", e)
return "", ""
def _last_run_cost(wid: str) -> float:
for r in storage.list_runs(wid, limit=10):
if r.status in ("success", "ran_late") and r.cost_usd:
return float(r.cost_usd)
return 0.0
def _enriched(wf: Workflow) -> dict:
"""Serialize a workflow with a cost_estimate block attached.
monthly_usd assumes future fires cost the same as the last successful
fire. Surfaces honestly as "at last run's cost" in the UI so users
understand it's a projection, not a quota.
"""
base = wf.model_dump(mode="json")
last = _last_run_cost(wf.id)
fires = scheduler.fires_in_window(wf, days=30)
base["cost_estimate"] = {
"monthly_usd": round(last * fires, 4),
"last_run_usd": round(last, 4),
"fires_per_month": fires,
}
return base
@workflows.router.get("/active")
async def list_active_runs():
"""Snapshot of currently-running workflow runs. Used by the tray and
the auto-updater veto."""
return {"active": scheduler.list_active()}
@workflows.router.post("/pause-all")
async def pause_all_schedules():
storage.set_paused(True)
scheduler.kick()
return {"paused": True}
@workflows.router.post("/resume-all")
async def resume_all_schedules():
storage.set_paused(False)
scheduler.kick()
return {"paused": False}
@workflows.router.get("/paused")
async def get_paused_state():
return {"paused": storage.get_paused()}
@workflows.router.get("/cron/findings")
async def cron_findings():
"""Cron entries we found at startup that reference OpenSwarm. The
FE renders a one-time banner inviting users to convert them; we
return the raw lines so the user can verify before migrating."""
return {"entries": list(_cron_findings)}
@workflows.router.get("/cloud/sms/status")
async def cloud_sms_status():
"""Probe used by the FE to decide whether to show the 'falls back to
in-app notify' acknowledgement on the text/call tiers. Returns
enabled=False until the cloud SMS bridge ships."""
return {"enabled": False}
@workflows.router.post("/runs/{run_id}/ack")
async def ack_run(run_id: str):
cancelled = escalation.cancel(run_id)
return {"acked": True, "had_pending_escalation": cancelled}
@workflows.router.get("/runs/{run_id}/escalation")
async def get_run_escalation(run_id: str):
state = escalation.status(run_id)
return {"state": state}
@workflows.router.get("/{workflow_id}")
async def get_workflow(workflow_id: str):
wf = storage.get_workflow(workflow_id)
if not wf:
raise HTTPException(status_code=404, detail="Workflow not found")
return _enriched(wf)
@workflows.router.get("/{workflow_id}/audit")
async def get_workflow_audit(workflow_id: str, limit: int = 50):
wf = storage.get_workflow(workflow_id)
if not wf:
raise HTTPException(status_code=404, detail="Workflow not found")
return {"entries": audit.read_tail(workflow_id, limit=limit)}
@workflows.router.patch("/{workflow_id}")
async def update_workflow(
workflow_id: str,
body: WorkflowUpdate,
if_match: Optional[str] = Header(default=None, alias="If-Match"),
):
wf = storage.get_workflow(workflow_id)
if not wf:
raise HTTPException(status_code=404, detail="Workflow not found")
# Optimistic concurrency: if the client passed If-Match, verify it
# matches the current updated_at. Stale writes (another window or a
# mid-edit background fire) get a 409 so the FE can prompt to reload
# instead of silently clobbering the other actor's changes. Missing
# header = legacy client, allow through (back-compat with the
# frontend's pre-409 code path; FE rolls out If-Match immediately).
if if_match:
current_stamp = wf.updated_at.isoformat() if hasattr(wf.updated_at, "isoformat") else str(wf.updated_at)
# Strip quotes a well-behaved HTTP client might add per RFC 7232.
if if_match.strip().strip('"') != current_stamp:
raise HTTPException(
status_code=409,
detail={
"error": "stale_update",
"message": "This workflow changed in another window or by a recent run. Reload and try again.",
"current_updated_at": current_stamp,
},
)
before = wf.model_dump(mode="json")
data = body.model_dump(exclude_unset=True)
for k, v in data.items():
setattr(wf, k, v)
wf.updated_at = datetime.now()
if not wf.icon:
wf.icon = _derive_icon(wf)
wf.next_run_at = scheduler.compute_next_fire(wf) if wf.schedule.enabled else None
storage.save_workflow(wf)
audit.log_change(wf.id, "user", before, wf.model_dump(mode="json"))
scheduler.kick()
return _enriched(wf)
@workflows.router.delete("/{workflow_id}")
async def delete_workflow(workflow_id: str):
existed = storage.delete_workflow(workflow_id)
if not existed:
raise HTTPException(status_code=404, detail="Workflow not found")
scheduler.kick()
return {"ok": True}
@workflows.router.post("/{workflow_id}/run")
async def run_workflow_now(workflow_id: str):
wf = storage.get_workflow(workflow_id)
if not wf:
raise HTTPException(status_code=404, detail="Workflow not found")
# executor.execute() owns the run record. Don't pre-create a stub here
# or we end up with two rows per manual fire (one orphan "running"
# row from this handler plus the real one from the executor).
pre_ids = {r.id for r in storage.list_runs(wf.id, limit=10)}
asyncio.create_task(executor.execute(wf, triggered_by="manual"))
# Poll briefly for the newly created run id. We also surface the
# run's status + error string when it lands quickly (e.g. cost-cap
# short-circuit, _running collision) so the FE can render a toast
# instead of silently switching to History.
for _ in range(25):
for r in storage.list_runs(wf.id, limit=10):
if r.id not in pre_ids and r.triggered_by == "manual":
return {
"run_id": r.id,
"status": r.status,
"error": r.error,
}
await asyncio.sleep(0.01)
return {"run_id": "", "status": None, "error": None}
@workflows.router.get("/{workflow_id}/runs")
async def list_workflow_runs(workflow_id: str, limit: int = 50):
wf = storage.get_workflow(workflow_id)
if not wf:
raise HTTPException(status_code=404, detail="Workflow not found")
runs = storage.list_runs(workflow_id, limit=limit)
return {"runs": [r.model_dump(mode="json") for r in runs]}
backend/auth.py
+27 -181
View File
@@ -1,28 +1,4 @@
"""Per-install auth token for the localhost API.
OpenSwarm's backend runs a FastAPI server on `127.0.0.1:<random-port>`
and streams sensitive agent data (tool inputs, approval requests,
messages) over WebSockets. Without auth, any webpage loaded in any
browser on the same machine can connect to those endpoints — WebSockets
aren't subject to Same-Origin Policy — and impersonate the user.
This module issues a cryptographically random token on first boot,
writes it 0600 to `<DATA_ROOT>/auth.token`, and reuses it on subsequent
restarts (so dev-mode hot-reload doesn't break the renderer's cached
copy). The token is regenerated only when the file is missing or empty.
Only code running as the same OS user can read the file.
Delivery to legitimate consumers:
- Electron main process reads the file and exposes it to the renderer
via a contextBridge method in preload.js (NOT plain window global).
- Our Python MCP subprocesses receive it via env var
`OPENSWARM_AUTH_TOKEN` that agent_manager passes when spawning.
- The Claude Code CLI we spawn receives it as `ANTHROPIC_API_KEY` in
env; the anthropic-proxy route trusts that value.
None of those paths are accessible from a third-party webpage.
"""
"""Per-install bearer token gating the localhost API and WS streams."""
from __future__ import annotations
@@ -38,13 +14,7 @@ _TOKEN: str = ""
def _write_atomic(path: str, data: str, mode: int = 0o600) -> None:
"""Write `data` to `path` atomically with the given file mode.
Uses `os.open(..., O_CREAT|O_WRONLY|O_TRUNC, mode)` + rename so the
final file is never world-readable and never left half-written if
the backend crashes mid-write. Windows-safe (rename of a file over
an existing one works on NTFS when the source was just closed).
"""
"""Atomic write to `path` at the given file mode; never world-readable or half-written."""
os.makedirs(os.path.dirname(path), exist_ok=True)
tmp = path + ".tmp"
fd = os.open(tmp, os.O_CREAT | os.O_WRONLY | os.O_TRUNC, mode)
@@ -60,30 +30,8 @@ def _write_atomic(path: str, data: str, mode: int = 0o600) -> None:
def init_auth_token() -> str:
"""Initialise the per-install auth token, persisting to disk.
Behaviour: prefer an existing token on disk; only mint a fresh one
when the file is absent or empty. This matters for two cases:
1. Dev mode (`bash run.sh`) — uvicorn's WatchFiles reload restarts
the worker process and re-runs init_auth_token. If we generated
a fresh token every reload, Electron's cached token (read once
at app boot) would mismatch and every authed request 401s
until the user fully restarts. Preserving the on-disk token
keeps Electron and the backend in sync across reloads.
2. Packaged builds — the user can restart the backend (Quit + reopen)
without the renderer reloading. Same mismatch risk, same fix.
Security trade-off: we no longer rotate the token on every restart.
The threat model that rotation was protecting against (a stale token
sitting in a log/crash dump being usable later) is marginal — anyone
who can read the artifact can also re-read the on-disk token, and
real rotation requires the file to be deleted (e.g. by signing out
or wiping the data root). Net: dev-mode reliability wins.
"""
"""Load the per-install token from disk, or mint one if missing; reused across restarts so Electron's cached copy stays valid."""
global _TOKEN
# Try existing on-disk token first.
try:
if os.path.exists(AUTH_TOKEN_FILE):
with open(AUTH_TOKEN_FILE, "r", encoding="utf-8") as f:
@@ -95,7 +43,6 @@ def init_auth_token() -> str:
)
return _TOKEN
except Exception as e:
# Fall through to fresh generation on any read error.
logger.warning(f"auth: failed to read existing token, generating new: {e}")
_TOKEN = secrets.token_urlsafe(32)
@@ -103,9 +50,7 @@ def init_auth_token() -> str:
_write_atomic(AUTH_TOKEN_FILE, _TOKEN, mode=0o600)
logger.info(f"auth: wrote token to {AUTH_TOKEN_FILE} (mode 0600)")
except Exception as e:
# Fail open is NOT an option here — if we can't write the file,
# Electron can't read it, and the user sees a broken app. But
# don't hard-crash the backend either; log loudly.
# If we can't write the file, Electron can't read it; log loudly but don't crash.
logger.error(f"auth: failed to write token file: {e}")
return _TOKEN
@@ -116,25 +61,13 @@ def get_auth_token() -> str:
class _TokenScrubFilter(logging.Filter):
"""Logging filter that redacts the install token from any log record.
The token leaks into logs in a few mundane ways: subprocess env dicts
that get logged when an MCP server fails to spawn, urllib retry logs
that include `?token=...` query strings, exception tracebacks that
print the response body of a failed proxied request. None of those
are intentional but they all happen, and the file ends up in crash
dumps / shared bug reports / hosted log aggregators. This filter is
pure defense in depth — behavior is unchanged when the token is
absent from a record.
"""
"""Logging filter that redacts the install token from log records (defense in depth)."""
_PLACEHOLDER = "<REDACTED:openswarm-token>"
@staticmethod
def _args_might_contain_token(args) -> bool:
"""Cheap pre-check: does any positional arg or dict-arg value mention
the token? Avoids the cost of `record.getMessage()` (which eagerly
does %-formatting) on the >99% of log lines that don't touch it."""
"""Cheap pre-check; avoids eager %-formatting on the >99% of records that don't mention the token."""
if not args:
return False
items = args if isinstance(args, (tuple, list)) else (args,)
@@ -149,13 +82,7 @@ class _TokenScrubFilter(logging.Filter):
@classmethod
def _scrub_args(cls, args):
"""Replace token within args in-place-equivalent, preserving the
original tuple/dict shape. Uvicorn's AccessFormatter unpacks
record.args as a 5-tuple (client_addr, method, full_path,
http_version, status_code); blanking args to None — what the
previous slow path did — triggered `cannot unpack non-iterable
NoneType object` for every access-logged line that contained
`?token=...`. Returns the same object if nothing was rewritten."""
"""Scrub token from args while preserving tuple/dict shape; uvicorn's AccessFormatter unpacks args as a 5-tuple and explodes on None."""
if args is None:
return args
if isinstance(args, dict):
@@ -181,27 +108,11 @@ class _TokenScrubFilter(logging.Filter):
def filter(self, record: logging.LogRecord) -> bool: # pragma: no cover (defensive)
if not _TOKEN:
return True
# Fast path: the overwhelming majority of log records don't mention
# the token at all. Two cheap string-in-string scans (raw msg + each
# arg) are far cheaper than forcing record.getMessage(), which would
# do eager %-formatting on every record in the process.
# Fast path: skip eager %-formatting on records that don't mention the token.
raw_msg = record.msg if isinstance(record.msg, str) else ""
if _TOKEN not in raw_msg and not self._args_might_contain_token(record.args):
return True
# Slow path. Two-step scrub so we cover both shapes:
# 1. In-place rewrite of record.msg and any string in record.args
# (or string-valued dict entry). Preserves args shape so
# uvicorn's AccessFormatter — which unpacks record.args as a
# 5-tuple and would explode on args=None — keeps working.
# 2. Render via record.getMessage() and check the substituted
# output. If a token survived step 1 (because it was buried
# inside a nested structure or a custom object's repr, e.g.
# `logger.info("env: %s", env_dict)` where the dict's repr
# exposes the value), bake the redacted final string into
# record.msg and clear args. This last-resort path only
# trips for records that the in-place pass couldn't reach,
# and uvicorn access logs never hit it (their args are
# always primitive strings/ints, fully scrubbed by step 1).
# Slow path: in-place args rewrite (preserves shape for AccessFormatter), then re-render to catch tokens buried in custom reprs.
try:
if isinstance(record.msg, str) and _TOKEN in record.msg:
record.msg = record.msg.replace(_TOKEN, self._PLACEHOLDER)
@@ -216,9 +127,7 @@ class _TokenScrubFilter(logging.Filter):
except Exception:
pass
except Exception:
# Never let the scrubber suppress a log line — if formatting
# fails for any reason, fall through and let normal handling
# proceed (worst case the token leaks for that one record).
# Never let the scrubber suppress a log line; worst case the token leaks for that one record.
pass
return True
@@ -227,27 +136,7 @@ _scrubber_installed = False
def install_token_scrubber() -> None:
"""Attach the token-scrubbing filter to every log handler in the process.
Why not just `root.addFilter(...)`: filters attached to a Logger only
fire on records emitted DIRECTLY on that logger. Records propagated up
from child loggers (uvicorn.access, uvicorn.error, websockets, etc.)
flow into root's HANDLERS without ever consulting root's logger-level
filters. So a logger-level install silently misses the access log —
which is exactly the line that contains `?token=...` query params.
This implementation:
1. Walks every currently-registered logger (root + everything in
`Logger.manager.loggerDict`) and attaches the scrubber to each
of their handlers.
2. Monkey-patches `Logger.addHandler` so any handler installed
AFTER this call (uvicorn configures its loggers during startup,
after main.py finishes importing) also gets the scrubber.
3. Keeps the root-logger filter as belt-and-suspenders for the
records that ARE emitted directly on root.
Idempotent — repeated calls are a no-op.
"""
"""Attach the scrubbing filter to every existing AND future log handler; logger-level filters miss propagated child records."""
global _scrubber_installed
if _scrubber_installed:
return
@@ -258,7 +147,6 @@ def install_token_scrubber() -> None:
if not any(isinstance(f, _TokenScrubFilter) for f in handler.filters):
handler.addFilter(scrubber)
# 1. Existing handlers across every known logger.
loggers: list[logging.Logger] = [logging.getLogger()]
for logger in logging.root.manager.loggerDict.values():
if isinstance(logger, logging.Logger):
@@ -267,9 +155,7 @@ def install_token_scrubber() -> None:
for h in list(logger.handlers):
_attach(h)
# 2. Future handlers — patch Logger.addHandler so anything attached
# after this point (uvicorn finishing its log config, plugins
# that reconfigure logging on their own) also gets scrubbed.
# Patch addHandler so handlers attached later (uvicorn finishes log config after main.py imports) get the scrubber too.
_original_addHandler = logging.Logger.addHandler
def _patched_addHandler(self: logging.Logger, hdlr: logging.Handler) -> None:
@@ -278,7 +164,6 @@ def install_token_scrubber() -> None:
logging.Logger.addHandler = _patched_addHandler # type: ignore[assignment]
# 3. Belt-and-suspenders on root logger itself.
root = logging.getLogger()
if not any(isinstance(f, _TokenScrubFilter) for f in root.filters):
root.addFilter(scrubber)
@@ -286,19 +171,9 @@ def install_token_scrubber() -> None:
_scrubber_installed = True
# Paths that never require auth. These are the public surface.
# Auth-exempt paths: external redirects with their own nonce/state validation, plus the bootstrap health probe.
_AUTH_EXEMPT_EXACT = {
# External OAuth providers redirect the user's browser here. The
# browser has no way to inject our bearer token (it's a 302 from
# Google/Anthropic/etc). The `state` query param is already a
# one-time nonce validated against `_pending_oauth`.
"/api/subscriptions/callback",
# Same pattern for the per-tool OAuth flow (Notion / Google Workspace /
# Airtable / HubSpot / Discord). The browser hits this with ?code=...&state=...
# after the user approves on the provider's site; the `state` param is
# the tool_id which we cross-check against _pending_oauth in tools_lib.py.
# Without this exemption the redirect lands a 401 page in the user's
# browser — see tools_lib.py:1156 where redirect_uri is constructed.
"/api/tools/oauth/callback",
# Spotify OAuth uses its own callback path because our app credentials
# are local (backend/.env) and the redirect target is OpenSwarm's
@@ -309,34 +184,23 @@ _AUTH_EXEMPT_EXACT = {
# has no way to inject our bearer token; the install_id check inside
# the handler is what binds the request to this user.
"/api/tools/oauth/cloud-claim",
# Bearer-handoff endpoints called by api.openswarm.com's success page
# AFTER Stripe checkout / Google sign-in / magic-link sign-in. The
# request POSTs the just-minted cloud bearer; the handler then re-
# validates it against the cloud (/api/me or /api/auth/signin-activate).
# The browser has no way to attach our per-install token here — the
# cloud-validated bearer in the body is the actual auth mechanism.
"/api/subscription/activate",
"/api/auth/signin-activate",
"/api/version",
# Local Google OAuth token-endpoint proxy: hit by the
# google-workspace-mcp subprocess we spawn. It doesn't (and can't
# easily) carry the install bearer in google-auth's refresh post.
# Localhost binding is the gate, and the route does nothing the
# public api.openswarm.com/api/oauth/google/refresh doesn't already
# do for any internet caller, so no new attack surface.
"/api/tools/google-oauth-token",
}
# Path prefixes that never require auth. Trailing slash optional.
_AUTH_EXEMPT_PREFIX = (
# Electron's boot handshake polls /api/health/check before it has a
# token (the HTTP port is up before main.js calls loadAuthToken()).
# Use a prefix so /api/health/check — and any future sub-route — is
# covered without re-introducing the bootstrap deadlock that an
# exact "/api/health" match caused.
# Electron polls /api/health/check before loading the token.
"/api/health",
# OpenAI API pass-through. 9Router calls this with the user's
# OpenAI Bearer token (sk-…), NOT our local auth token, so our
# middleware would reject. Localhost-only network boundary is the
# security gate — the route only forwards to api.openai.com and
# never touches user data on this machine. See
# backend/apps/agents/openai_passthrough.py for why this exists.
# 9Router proxies OpenAI requests with the user's sk-... bearer, not our local token; localhost-only is the gate.
"/api/openai-passthrough",
# FastAPI's default health/docs/schema surface (packaged app never
# ships /docs, but be defensive).
"/docs",
"/openapi",
"/redoc",
@@ -366,20 +230,9 @@ def extract_bearer(header_value: str | None) -> str:
def request_matches_token(request_headers: dict, query_params: dict | None = None) -> bool:
"""Validate that an incoming HTTP / WS request carries our token.
Accepts any of:
- `Authorization: Bearer <token>`
- `x-openswarm-token: <token>` (custom header for callers that
can't easily set Authorization — e.g. future CLI clients)
- `?token=<token>` query param (WS only; browsers can't easily
set custom WS headers, so the token rides in the URL)
The token comparison is constant-time via `secrets.compare_digest`.
"""
"""Validate that an HTTP/WS request carries our token (Bearer, x-openswarm-token, or ?token=); constant-time compare."""
if not _TOKEN:
# Backend started without auth init — fail closed. This should
# only happen in test fixtures that intentionally bypass main.
# Backend not initialized: fail closed. Only test fixtures that bypass main hit this.
return False
candidates: list[str] = []
@@ -407,14 +260,10 @@ def request_matches_token(request_headers: dict, query_params: dict | None = Non
return False
# Origin allowlist for WS handshakes. Electron's renderer loads from
# `file://` when packaged; `http://localhost:3000` (Vite dev server) and
# `http://127.0.0.1:3000` in dev. A bare `null` Origin is sent by some
# Electron contexts.
# WS Origin allowlist: Electron packaged is file://, dev is localhost:3000, some Electron contexts send bare "null".
_ORIGIN_ALLOWLIST_DEV = {
"http://localhost:3000",
"http://127.0.0.1:3000",
# Electron may load prod build from file:// or an app:// scheme.
"file://",
"null",
}
@@ -423,16 +272,13 @@ _ORIGIN_ALLOWLIST_DEV = {
def is_origin_allowed(origin: str | None) -> bool:
"""True if the WS connection's Origin header is from our app."""
if origin is None:
# No Origin header = curl / native WS client / MCP subprocess.
# Token check is still required, so allow.
# Native WS client / curl / MCP subprocess: token check still required, so allow.
return True
if origin in _ORIGIN_ALLOWLIST_DEV:
return True
# file:// origins in Electron prod sometimes include paths like
# file:///Applications/OpenSwarm.app/... — match by prefix.
# Packaged Electron file:// includes paths like file:///Applications/OpenSwarm.app/...; match by prefix.
if origin.startswith("file://"):
return True
# localhost + any port (dev servers, tools the developer is running).
if origin.startswith("http://localhost:") or origin.startswith("http://127.0.0.1:"):
return True
return False
backend/config/Apps.py
+1 -2
View File
@@ -37,8 +37,7 @@ class MainApp:
yield
self.app = FastAPI(lifespan=lifespan)
# Include all sub-app routers in the main app with their prefixes
for sub_app in sub_apps:
self.app.include_router(
sub_app.router,
backend/config/install_id.py
+2 -19
View File
@@ -1,13 +1,4 @@
"""Per-install identifier.
A UUID4 generated on first run, persisted at ``<DATA_ROOT>/install_id``
with 0600 perms. Used to bind an in-flight OAuth claim to the install
that started it, so a leaked session_id alone is useless.
Not a secret. Not a user identity. Not stable across reinstalls
(reinstalling generates a new ID, by design — the previous install's
in-flight OAuth flows shouldn't follow the user across reinstalls).
"""
"""Per-install UUID4 binding in-flight OAuth claims to the install that started them."""
from __future__ import annotations
@@ -21,12 +12,7 @@ _cached: str | None = None
def get_install_id() -> str:
"""Return the persistent install_id, generating + persisting on first call.
Idempotent across processes — if the file already exists we read it.
Concurrent first-call from two processes is safe: both write a UUID,
last-writer-wins, neither side cares which one is canonical.
"""
"""Return the persistent install_id, generating and persisting on first call."""
global _cached
if _cached:
return _cached
@@ -40,13 +26,10 @@ def get_install_id() -> str:
except FileNotFoundError:
pass
except Exception:
# Corrupt file — overwrite below.
pass
fresh = str(uuid.uuid4())
os.makedirs(os.path.dirname(_INSTALL_ID_FILE) or ".", exist_ok=True)
# 0600 so other accounts on the same machine can't read it. We're not
# treating it as a secret, but no reason to be sloppy.
fd = os.open(_INSTALL_ID_FILE, os.O_CREAT | os.O_WRONLY | os.O_TRUNC, 0o600)
try:
os.write(fd, fresh.encode("utf-8"))
backend/config/paths.py
+3 -12
View File
@@ -1,10 +1,4 @@
"""Centralised path definitions for the OpenSwarm backend.
In dev mode (default) data lives under ``backend/data/``.
When packaged as a desktop app, Electron sets ``OPENSWARM_PACKAGED=1`` and
data is stored in a platform-appropriate location
(``~/Library/Application Support/OpenSwarm/data/`` on macOS).
"""
"""Path definitions: dev under backend/data/, packaged under platform app-support."""
import os
import sys
@@ -34,12 +28,9 @@ OUTPUTS_WORKSPACE_DIR = os.path.join(DATA_ROOT, "outputs_workspace")
SKILLS_WORKSPACE_DIR = os.path.join(DATA_ROOT, "skills_workspace")
DASHBOARD_LAYOUT_DIR = os.path.join(DATA_ROOT, "dashboard_layout")
BUILTIN_PERMISSIONS_PATH = os.path.join(DATA_ROOT, "builtin_permissions.json")
TRUSTED_SENSITIVE_PATHS_PATH = os.path.join(DATA_ROOT, "trusted_sensitive_paths.json")
# Per-install auth token for the localhost WS + HTTP API. Regenerated
# every backend start. Only code running as the current OS user (Electron
# main process, our Python MCP subprocesses, the Claude Code CLI we
# spawn) can read this file. Webpages loaded in any browser on the
# machine cannot — which is the whole point. See auth.py.
# Per-install auth token for the localhost API; see auth.py.
AUTH_TOKEN_FILE = os.path.join(DATA_ROOT, "auth.token")
BACKEND_DIR = _BACKEND_DIR
backend/main.py
+73 -199
View File
@@ -20,13 +20,8 @@ logger = logging.getLogger(__name__)
from fastapi.responses import JSONResponse, HTMLResponse
from fastapi import Request
# In-memory store for pending OAuth flows (state -> {provider, code_verifier, redirect_uri})
_pending_oauth: dict[str, dict] = {}
# Recently-completed OAuth states so the /api/subscriptions/callback handler
# can distinguish a legitimate duplicate callback (browser prefetch, refresh,
# or Google redirect retry after a slow first response) from a truly stale
# request. Bounded FIFO — drops the oldest entries once it grows past
# _MAX_COMPLETED_OAUTH so it can't leak memory.
# Bounded FIFO of recently-completed OAuth states; lets the callback distinguish duplicate hits (prefetch, refresh) from stale.
_completed_oauth: list[str] = []
_MAX_COMPLETED_OAUTH = 64
@@ -35,7 +30,6 @@ def _mark_oauth_completed(state: str) -> None:
if state in _completed_oauth:
return
_completed_oauth.append(state)
# Trim head if we've outgrown the bound
while len(_completed_oauth) > _MAX_COMPLETED_OAUTH:
_completed_oauth.pop(0)
from backend.config.Apps import MainApp
@@ -56,15 +50,15 @@ from backend.apps.auth.router import auth
from backend.apps.web.web import web
from backend.apps.agents.anthropic_proxy import anthropic_proxy
from backend.apps.telegram_bot.listener import telegram_bot
from backend.apps.workflows.workflows import workflows
from fastapi.middleware.cors import CORSMiddleware
from fastapi import WebSocket, WebSocketDisconnect
import json
main_app = MainApp([health, agents, skills, tools_lib, modes, settings, mcp_registry, skill_registry, outputs, dashboards, service, subscription, auth, web, anthropic_proxy, telegram_bot])
main_app = MainApp([health, agents, skills, tools_lib, modes, settings, mcp_registry, skill_registry, outputs, dashboards, service, subscription, auth, web, anthropic_proxy, workflows, telegram_bot])
app = main_app.app
# Generate per-install auth token BEFORE we bind the HTTP port. By the
# time any request lands, the token file exists. See backend/auth.py.
# Generate per-install auth token BEFORE we bind the HTTP port so the token file exists by the time any request lands.
from backend.auth import (
init_auth_token,
install_token_scrubber,
@@ -73,18 +67,11 @@ from backend.auth import (
is_origin_allowed,
)
init_auth_token()
# Install the log scrubber AFTER the token exists so any log line that
# accidentally embeds it (subprocess env dumps, urllib retry traces,
# proxied-request error bodies) gets redacted before hitting handlers.
# Install log scrubber AFTER token exists so any log line embedding it gets redacted before handlers see it.
install_token_scrubber()
# CORS: previously wide open (`allow_origins=["*"]`), which combined with
# `allow_credentials=True` was a security footgun — any external origin
# could CORS-preflight us. Now restricted to Electron renderer origins +
# localhost dev servers. The token middleware below provides the
# *primary* defense; CORS is defense-in-depth so a misconfigured page
# can't even reach us.
# CORS restricted to Electron renderer + localhost dev; token middleware below is the primary defense.
app.add_middleware(
CORSMiddleware,
allow_origins=[
@@ -97,49 +84,22 @@ app.add_middleware(
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
# Every cross-origin POST from the Electron renderer (file:// → http://localhost:8324)
# carries Authorization: Bearer, which CORS classifies as non-simple and
# forces a preflight OPTIONS before EACH POST. With no max_age the browser
# re-preflights on a tight schedule (~5 s in Chromium); under heavy
# interaction we observed a 1:1 OPTIONS-to-POST ratio in the dev log,
# doubling roundtrip count for no reason. Caching the preflight result
# for 10 minutes drops that to one OPTIONS per ~600 POSTs.
# Cache preflight 10min; without this Chromium re-preflights every ~5s and we saw 1:1 OPTIONS:POST in dev logs.
max_age=600,
)
@app.middleware("http")
async def _auth_middleware(request: Request, call_next):
"""Reject HTTP requests without our per-install bearer token.
Exemptions (see `auth.is_path_exempt`):
- `/api/subscriptions/callback` — external OAuth redirects
- `/api/health`, `/api/version` — Electron boot handshake
- `OPTIONS` preflights — browsers don't send Authorization on them
Anything else requires `Authorization: Bearer <token>` OR
`x-openswarm-token: <token>`. Failure responds with 401 and a short
JSON error — no upstream handler sees the request.
The anthropic-proxy route (`/api/anthropic-proxy/v1/*`) is NOT
exempt. Its caller (the Claude Code CLI we spawn) is configured
with `ANTHROPIC_API_KEY=<our_token>` so the CLI's `x-api-key`
header carries our token — which `request_matches_token` accepts
via its auth-header branches.
"""
# Preflights never carry Authorization.
"""Reject HTTP requests without our per-install bearer token."""
if request.method == "OPTIONS":
response = await call_next(request)
elif is_path_exempt(request.url.path):
response = await call_next(request)
else:
# Accept Authorization Bearer, x-openswarm-token, OR x-api-key
# (CLI path — CLI sends x-api-key with our token as value).
headers = dict(request.headers)
x_api_key = headers.get("x-api-key") or headers.get("X-API-Key")
# Accept `?token=<token>` query param too. Required for browser-driven
# GETs that can't set headers — notably the App Builder iframe loading
# /api/outputs/.../serve/index.html via <iframe src="...">.
# Accept ?token= for browser-driven GETs that can't set headers (App Builder iframe).
auth_ok = request_matches_token(headers, query_params=dict(request.query_params))
if not auth_ok and x_api_key:
import secrets as _s
@@ -156,29 +116,13 @@ async def _auth_middleware(request: Request, call_next):
)
response = await call_next(request)
# Private-Network-Access header for the one remaining public-origin
# path (OAuth callback). Harmless on other requests.
# PNA header for the OAuth callback (one remaining public-origin path); harmless elsewhere.
response.headers.setdefault("Access-Control-Allow-Private-Network", "true")
return response
@app.websocket("/ws/agents/{session_id}")
async def websocket_session(websocket: WebSocket, session_id: str):
"""Per-session WS endpoint with resume + heartbeat.
Resilience contract (see backend/apps/agents/seq_log.py):
- Every server→client event carries a monotonic `seq` per session.
- On (re)connect the client sends `client:hello` with its
last-seen seq; the server replays missed events (or emits
`agent:gap_detected` if the gap is too large) and answers
with `server:hello` carrying the current high-water seq.
- `client:ping` → `server:pong` heartbeat (default 25s) so
silent socket deaths (NAT idle drop, laptop sleep) are
detected without waiting for the next outbound frame.
- `WebSocketDisconnect` only removes the socket from the
connection registry. The agent task keeps running. The only
things that end a run are: natural completion, explicit
`agent:stop`, REST `/close`, or process shutdown.
"""
"""Per-session WS endpoint with resume + heartbeat (see seq_log.py for the contract)."""
if not _ws_auth_ok(websocket):
return
await ws_manager.connect_session(session_id, websocket)
@@ -190,12 +134,6 @@ async def websocket_session(websocket: WebSocket, session_id: str):
payload = msg.get("data", {})
if event == "client:hello":
# Resume handshake. The client sends this immediately
# after the WS opens, with `last_seq` = the highest
# seq it has applied. We replay anything newer; on
# first connect last_seq=0 and replay() correctly
# returns nothing (empty buffer) or the persisted
# terminal event for already-finished sessions.
last_seq = int(payload.get("last_seq") or 0)
connection_uuid = payload.get("connection_uuid") or ""
ack = await ws_manager.replay_to(session_id, websocket, last_seq)
@@ -210,10 +148,6 @@ async def websocket_session(websocket: WebSocket, session_id: str):
},
}))
elif event == "client:ping":
# Heartbeat. Cheap, keeps NATs/firewalls from
# silently dropping the connection. Carry the
# client's nonce back so it can match pong→ping for
# round-trip latency tracking if it wants.
await websocket.send_text(json.dumps({
"event": "server:pong",
"session_id": session_id,
@@ -235,6 +169,7 @@ async def websocket_session(websocket: WebSocket, session_id: str):
"behavior": payload.get("behavior", "deny"),
"message": payload.get("message"),
"updated_input": payload.get("updated_input"),
"trust_pattern": bool(payload.get("trust_pattern")),
})
elif event == "agent:edit_message":
from backend.apps.agents.agent_manager import agent_manager
@@ -247,16 +182,11 @@ async def websocket_session(websocket: WebSocket, session_id: str):
from backend.apps.agents.agent_manager import agent_manager
await agent_manager.stop_agent(session_id)
except WebSocketDisconnect:
# Drops the socket from the connection list. Does NOT cancel
# the agent task — that's intentional. See module docstring.
# Drops the socket but does NOT cancel the agent task; that's intentional.
ws_manager.disconnect_session(session_id, websocket)
def _ws_auth_ok(websocket: WebSocket) -> bool:
"""Validate token + origin before accepting a WS. Returns True if OK.
On failure closes with 4401 (custom app-level code) and returns False
— the caller must NOT call `websocket.accept()` or read any data.
"""
"""Validate token + origin before accepting a WS; closes with 4401 on failure."""
headers = dict(websocket.headers)
qp = dict(websocket.query_params)
origin = headers.get("origin") or headers.get("Origin")
@@ -264,9 +194,8 @@ def _ws_auth_ok(websocket: WebSocket) -> bool:
origin_ok = is_origin_allowed(origin)
if not (token_ok and origin_ok):
reason = "bad token" if not token_ok else f"bad origin ({origin})"
logger.warning(f"ws: rejecting connection to {websocket.url.path} — {reason}")
# Can't `await websocket.close()` before accept(), so schedule the
# close in a task. The client receives a 403 on handshake.
logger.warning(f"ws: rejecting connection to {websocket.url.path}: {reason}")
# Can't await close() before accept(); schedule it so the client sees a 403 on handshake.
import asyncio as _asyncio
_asyncio.create_task(websocket.close(code=4401))
return False
@@ -275,22 +204,14 @@ def _ws_auth_ok(websocket: WebSocket) -> bool:
@app.websocket("/ws/outputs/runtime/{workspace_id}/logs")
async def websocket_runtime_logs(websocket: WebSocket, workspace_id: str):
"""Stream the persistent app-backend's stdout/stderr to the Terminal
pane. On connect we replay the runtime's ring buffer so a Terminal
tab opened mid-session sees the context it missed, then we tail
every subsequent line until disconnect."""
"""Stream the app-backend's stdout/stderr to the Terminal pane; replays ring buffer on connect, tails after."""
if not _ws_auth_ok(websocket):
return
await websocket.accept()
from backend.apps.outputs.runtime import manager as runtime_manager
rt = runtime_manager.get(workspace_id)
if rt is None:
# No active runtime — surface that to the client and close. The
# frontend will call /runtime/start and reconnect. Also emit a
# status frame with is_new_mode (computed from disk) so the
# preview pane shows the "starting preview…" placeholder for
# webapp_template workspaces instead of falling back to the
# legacy /serve/index.html URL (which 404s in new-mode).
# No runtime: emit is_new_mode status so webapp_template workspaces show the starting-preview placeholder, not the legacy 404ing serve URL.
try:
from backend.apps.outputs.outputs import _runtime_status_payload
status = _runtime_status_payload(workspace_id)
@@ -306,10 +227,7 @@ async def websocket_runtime_logs(websocket: WebSocket, workspace_id: str):
finally:
await websocket.close()
return
# Buffer log lines from the synchronous subscriber callback into an
# asyncio.Queue we can `await` on the WS sender side. The subscribe
# call replays the ring buffer synchronously, so the queue gets
# primed with existing lines before we enter the loop.
# Bridge sync subscriber callback to async sender; subscribe replays the ring buffer synchronously, priming the queue.
queue: asyncio.Queue[tuple[str, str]] = asyncio.Queue()
def _on_line(line) -> None:
@@ -335,11 +253,6 @@ async def websocket_runtime_logs(websocket: WebSocket, workspace_id: str):
}
try:
# Initial status frame so the client knows port/running state
# without a second HTTP round-trip. `frontend_url` is the
# new-mode preview pointer (Vite dev server); `backend_url` is
# the workspace's optional FastAPI backend (old-mode backend.py
# OR new-mode post-backend_init.sh).
await websocket.send_text(json.dumps(_build_status_frame()))
while True:
stream, text = await queue.get()
@@ -348,12 +261,7 @@ async def websocket_runtime_logs(websocket: WebSocket, workspace_id: str):
"workspace_id": workspace_id,
"data": {"stream": stream, "text": text},
}))
# Runtime-level events (start, frontend-ready, exit) flow
# through the same log channel with stream="runtime". When
# the client sees one, it usually wants the fresh status —
# bind-ready in particular flips frontend_url from null
# to the Vite URL and the preview pane has to know to
# switch over. Re-push status after every runtime line.
# Re-push status on runtime events: bind-ready flips frontend_url from null to the Vite URL and the preview pane needs to switch.
if stream == "runtime":
await websocket.send_text(json.dumps(_build_status_frame()))
except WebSocketDisconnect:
@@ -380,6 +288,7 @@ async def websocket_dashboard(websocket: WebSocket):
"behavior": payload.get("behavior", "deny"),
"message": payload.get("message"),
"updated_input": payload.get("updated_input"),
"trust_pattern": bool(payload.get("trust_pattern")),
})
elif event == "browser:result":
ws_manager.resolve_browser_command(
@@ -392,8 +301,7 @@ async def websocket_dashboard(websocket: WebSocket):
@app.post("/api/browser/command")
async def browser_command(request: Request):
"""HTTP endpoint called by the browser MCP server subprocess.
Proxies commands to the frontend via WebSocket and waits for results."""
"""Browser MCP subprocess endpoint; proxies commands to frontend over WS and waits for results."""
body = await request.json()
action = body.get("action", "")
browser_id = body.get("browser_id", "")
@@ -410,7 +318,7 @@ async def browser_command(request: Request):
@app.get("/api/subscriptions/pending/{state}")
async def subscriptions_pending(state: str):
"""Return pending OAuth data for a state param. Called by 9Router's callback page."""
"""Return pending OAuth data for a state param; called by 9Router's callback page."""
pending = _pending_oauth.get(state)
if not pending:
return JSONResponse({"error": "not found"}, status_code=404,
@@ -436,34 +344,18 @@ _SUCCESS_HTML = (
@app.get("/api/subscriptions/callback")
async def subscriptions_callback(request: Request):
"""Catch OAuth redirect from provider, exchange code via 9Router, close window.
Must be idempotent: the browser can legitimately hit this URL more than
once (Chrome prefetch, user refresh, Google retrying a slow first
redirect). The first call consumes `_pending_oauth[state]`, so a second
call would otherwise render a misleading "Session expired" even though
the connection is already saved. To handle that, we track recently-
completed state values in `_completed_oauth` and return the success
page whenever we see a duplicate.
"""
"""Catch OAuth redirect from provider, exchange code via 9Router, close window; idempotent against prefetch/refresh duplicates."""
code = request.query_params.get("code", "")
state = request.query_params.get("state", "")
error = request.query_params.get("error", "")
if error:
# Escape both inputs — `error_description` and `error` are attacker-
# controllable query params and the endpoint is auth-exempt, so an
# unescaped interpolation here is a reflected XSS in the localhost
# origin (loadable inside the Electron app context, where same-origin
# JS has access to the install token).
# Escape: error/error_description are attacker-controllable and this endpoint is auth-exempt, so raw interpolation is reflected XSS in the localhost origin.
desc = html.escape(request.query_params.get("error_description", error))
return HTMLResponse(f'<html><body style="background:#1a1a1a;color:#fff;display:flex;align-items:center;justify-content:center;height:100vh;font-family:sans-serif"><div style="text-align:center"><h2>Authorization failed</h2><p style="color:#888">{desc}</p></div></body></html>')
pending = _pending_oauth.pop(state, None)
if not pending:
# Either a duplicate callback for a state we've already exchanged,
# or a truly stale state. Duplicates are the expected case —
# Chrome's prefetcher and some extensions speculatively GET URLs.
if state and state in _completed_oauth:
logger.info(f"Duplicate OAuth callback for state {state[:8]}... (already completed)")
return HTMLResponse(_SUCCESS_HTML)
@@ -475,10 +367,7 @@ async def subscriptions_callback(request: Request):
await exchange_oauth(pending["provider"], code, pending["redirect_uri"], pending["code_verifier"], state)
except Exception as e:
logger.warning(f"OAuth exchange failed for provider={pending.get('provider')}: {e}")
# Escape the exception message — upstream OAuth provider errors can
# echo back attacker-influenced strings (e.g. error_description from
# the original request URL), and this response is rendered in the
# localhost origin.
# Escape: upstream OAuth errors can echo attacker-influenced strings and this response renders in the localhost origin.
safe_e = html.escape(str(e))
return HTMLResponse(f'<html><body style="background:#1a1a1a;color:#fff;display:flex;align-items:center;justify-content:center;height:100vh;font-family:sans-serif"><div style="text-align:center"><h2>Connection failed</h2><p style="color:#888">{safe_e}</p></div></body></html>')
@@ -489,8 +378,7 @@ async def subscriptions_callback(request: Request):
@app.post("/api/browser-agent/run")
async def browser_agent_run(request: Request):
"""Run one or more browser sub-agents in parallel.
Called by the browser_agent_mcp_server stdio subprocess."""
"""Run one or more browser sub-agents in parallel; called by the browser_agent_mcp_server stdio subprocess."""
from backend.apps.settings.settings import load_settings
from backend.apps.agents.browser_agent import run_browser_agents
@@ -516,28 +404,14 @@ async def browser_agent_run(request: Request):
@app.post("/api/mcp-meta/{action}")
async def mcp_meta(action: str, request: Request):
"""Back the openswarm-mcp-meta stdio MCP server.
Actions:
- list: enumerate installed MCPs, separated by active vs available.
- search: rank by description match against a query.
- activate: append to session.active_mcps + flag needs_fork=True so the
next turn rebuilds options with the newly-activated server. Validates
server_name against the canonical registry; unknown names return the
valid options instead of activating (anti-hallucination).
"""
"""Back the openswarm-mcp-meta stdio MCP server: list, search, activate."""
from backend.apps.agents.agent_manager import agent_manager
from backend.apps.tools_lib.tools_lib import _load_all as load_all_tools, _sanitize_server_name
body = await request.json()
parent_session_id = body.get("parent_session_id", "")
# Aliases that broaden the search corpus for common user intents. Without
# these, MCPSearch("email") fails to surface Google Workspace because
# the tool's stored description says "Gmail" not "email". Keys are
# sanitized server names; values are extra search-hint tokens appended
# to the haystack. Only generic synonyms — anything that's already in
# the description doesn't need to be listed.
# Synonym aliases broaden the search corpus so MCPSearch("email") surfaces Google Workspace despite its description saying "Gmail".
_SERVER_SEARCH_ALIASES: dict[str, list[str]] = {
"google-workspace": [
"email", "inbox", "mail", "gmail", "calendar", "schedule",
@@ -579,8 +453,7 @@ async def mcp_meta(action: str, request: Request):
if not (t.mcp_config and t.enabled and t.auth_status in ("configured", "connected")):
continue
sanitized = _sanitize_server_name(t.name)
# Pull tool sub-action names from tool_permissions._tool_descriptions
# so MCPSearch can match against capability names (e.g. "send_email").
# Pull sub-action names so MCPSearch can match against capability names like "send_email".
action_names: list[str] = []
try:
td = (t.tool_permissions or {}).get("_tool_descriptions", {})
@@ -613,11 +486,7 @@ async def mcp_meta(action: str, request: Request):
servers = _connected_servers()
session = agent_manager.sessions.get(parent_session_id) if parent_session_id else None
active_set = set(session.active_mcps) if session else set()
# Ranking: substring hits across name+description+sub-tool names+
# generic-purpose aliases. The aliases are what let "email" match
# google-workspace even though the description says "Gmail".
# Active-first tiebreak so the model prefers servers it has already
# activated when both score equally.
# Substring rank across name+desc+sub-tools+aliases; active-first tiebreak.
scored: list[tuple[int, dict]] = []
for s in servers:
extras = s.get("_search_extras", "")
@@ -625,9 +494,7 @@ async def mcp_meta(action: str, request: Request):
score = 0
for tok in query.split():
if tok and tok in hay:
# Hits in the canonical name count more; alias hits
# count once so a "drive" query doesn't beat the actual
# Drive tool description.
# Canonical name hits weight 2; alias hits 1 so "drive" doesn't beat the actual Drive description.
if tok in s["name"]:
score += 2
elif tok in s["description"].lower():
@@ -662,14 +529,7 @@ async def mcp_meta(action: str, request: Request):
session.active_mcps.append(server_name)
session.needs_fork = True
# When the session has prior turns, fork_session alone won't
# make the bundled CLI re-read mcp_servers — the transport
# snapshot at launch time is what serves tool schemas. Force a
# full fresh-session restart so the next turn rebuilds with the
# newly-activated server in its mcp_servers dict from scratch.
# First-turn activations don't need this (the SDK session hasn't
# locked in yet). One-time ~200-400ms cold start on the auto-
# continuation turn that fires right after this anyway.
# Mid-session activations need a fresh-session restart: the CLI snapshots mcp_servers at launch, so fork_session alone won't re-read schemas.
if session.sdk_session_id:
session.needs_fresh_session = True
try:
@@ -681,22 +541,41 @@ async def mcp_meta(action: str, request: Request):
})
except Exception:
logger.exception("Failed to broadcast post-activate session status")
pass # MCP activation captured via session dump on close
# Auto-continue: flag the session so that after its current turn
# ends (which is the turn that contains this MCPActivate tool
# call), the agent loop dispatches a synthetic "continue" turn
# with the freshly-activated tools available. Race-free — read
# at the natural turn-boundary inside _run_agent_loop instead of
# racing a background task against the turn's completion path.
# Turns the typical 3-prompt flow ("check email" → MCPActivate
# → "do it") into a 1-prompt flow.
# Auto-continue: read at turn boundary in _run_agent_loop (race-free); collapses the typical 3-prompt flow into 1.
session.pending_continuation = True
# Enumerate the just-activated server's callable tool names so the
# continuation turn can call them directly. Without this the model
# often burns a turn on tool-discovery guesses (Bash "mcp list",
# Ls /toolbox, ToolSearch fallbacks) before landing on the right
# mcp__server__action name. Cap at 16 + clip descriptions so the
# prompt stays bounded for kitchen-sink servers (google-workspace
# exposes ~30 tools). Best-effort; any lookup failure silently
# falls back to the same prompt this code shipped with before.
tool_hint = ""
try:
for t in load_all_tools():
if _sanitize_server_name(t.name) != server_name:
continue
descs = (t.tool_permissions or {}).get("_tool_descriptions", {}) or {}
if not descs:
break
lines: list[str] = []
for sub_name, desc in list(descs.items())[:16]:
short = (desc or "").strip().split("\n", 1)[0][:120]
visible = f"mcp__{server_name}__{sub_name}"
lines.append(f"- `{visible}`: {short}" if short else f"- `{visible}`")
if lines:
more = "" if len(descs) <= 16 else f"\n(+ {len(descs) - 16} more; call ToolSearch with the server name for the rest)"
tool_hint = "\n\nCallable tools on this server:\n" + "\n".join(lines) + more
break
except Exception:
logger.exception("activate: failed to build tool hint for %s", server_name)
session.pending_continuation_prompt = (
"[mcp:auto-continue] The MCP server you requested has been "
f"activated (`{server_name}`). Continue with the user's original "
"request now using the newly-available tools — do NOT ask "
"for confirmation."
"request now using the newly-available tools; do NOT ask "
"for confirmation." + tool_hint
)
return JSONResponse({"status": "activated", "server_name": server_name, "auto_continue": True})
@@ -706,12 +585,7 @@ async def mcp_meta(action: str, request: Request):
@app.post("/api/agents/sessions/{session_id}/compact")
async def session_compact(session_id: str):
"""Force a compaction pass on a session (Phase 2 /compact slash cmd).
Cheap programmatic summarization (no aux LLM call), so it's safe to
invoke at any time. Sets needs_fork=True so the next turn rebuilds
options and ships the compacted prefix.
"""
"""Force a compaction pass on a session (/compact slash cmd); programmatic summary, no aux LLM call."""
from backend.apps.agents.agent_manager import agent_manager
from backend.apps.agents.ws_manager import ws_manager as _ws
session = agent_manager.sessions.get(session_id)
@@ -729,14 +603,10 @@ async def session_compact(session_id: str):
@app.post("/api/agents/sessions/{session_id}/clear")
async def session_clear(session_id: str):
"""Reset a session to a fresh sdk_session_id (Phase 2 /clear slash cmd).
Preserves session.messages (so the chat UI keeps the visible history)
but clears the SDK-side conversation by minting a new sdk_session_id.
Also drops active_mcps so the user starts fresh.
"""
"""Wipe the session's UI history AND its SDK convo state (/clear slash cmd, Reset history button)."""
from backend.apps.agents.agent_manager import agent_manager
from backend.apps.agents.ws_manager import ws_manager as _ws
from backend.apps.agents.models import MessageBranch
session = agent_manager.sessions.get(session_id)
if not session:
return JSONResponse({"error": "session not found"}, status_code=404)
@@ -746,6 +616,11 @@ async def session_clear(session_id: str):
session.tokens = {"input": 0, "output": 0}
session.cost_usd = 0.0
session.needs_fork = False
session.messages = []
session.pending_approvals = []
session.branches = {"main": MessageBranch(id="main")}
session.active_branch_id = "main"
session.tool_group_meta = {}
await _ws.send_to_session(session_id, "agent:status", {
"session_id": session_id,
"status": session.status,
@@ -760,8 +635,7 @@ async def session_clear(session_id: str):
@app.post("/api/invoke-agent/run")
async def invoke_agent_run(request: Request):
"""Fork an existing agent session and send it a new message.
Called by the invoke_agent_mcp_server stdio subprocess."""
"""Fork an existing agent session and send a new message; called by invoke_agent_mcp_server."""
body = await request.json()
session_id = body.get("session_id", "")
message = body.get("message", "")
@@ -804,7 +678,7 @@ if __name__ == "__main__":
import uvicorn.config
class _ReadyServer(uvicorn.Server):
"""Subclass that prints a machine-readable READY line on startup."""
"""Prints a machine-readable READY line on startup."""
async def startup(self, sockets=None):
await super().startup(sockets)
print(f"READY:PORT={args.port}", flush=True)
backend/tests/test_auth_router.py
+4 -4
View File
@@ -1,6 +1,6 @@
"""Smoketests for the desktop-side auth subapp.
These tests don't hit the real cloud — they patch httpx so we can simulate
These tests don't hit the real cloud; they patch httpx so we can simulate
each cloud response and assert the local persistence + identify-status
logic is right across every gate-dismissal path the renderer cares about.
"""
@@ -80,7 +80,7 @@ def test_signin_activate_persists_user_id(client, reset_settings):
def test_signin_activate_paid_user_flips_pro_mode(client, reset_settings):
"""A signed-in user who already has a Stripe subscription should also
flip into openswarm-pro routing — covers the Google-then-Stripe and
flip into openswarm-pro routing; covers the Google-then-Stripe and
Stripe-then-Google merge cases."""
fake_response = AsyncMock()
fake_response.status_code = 200
@@ -127,7 +127,7 @@ def test_signin_activate_invalid_token_returns_401(client, reset_settings):
def test_signin_activate_short_token_rejected_locally(client, reset_settings):
"""Short tokens rejected before we even hit the cloud — saves a round trip."""
"""Short tokens rejected before we even hit the cloud; saves a round trip."""
r = client.post(
"/api/auth/signin-activate",
json={"token": "short", "signin_method": "google"},
@@ -136,7 +136,7 @@ def test_signin_activate_short_token_rejected_locally(client, reset_settings):
# ---------------------------------------------------------------------------
# /api/auth/identity-status — gate-state for the renderer
# /api/auth/identity-status; gate-state for the renderer
# ---------------------------------------------------------------------------
def test_identity_status_signed_in_user_returns_authed_true(client, reset_settings):
backend/tests/test_disconnect_resilience.py
+8 -8
View File
@@ -9,7 +9,7 @@ terminal state. After these fixes the contract should be:
2. Every event the server emits is replayable, in order, with no
duplicates and no gaps, after any number of disconnects.
3. Terminal events (completed/stopped/error) are always observable
by a client that reconnects later — even if the only persistence
by a client that reconnects later; even if the only persistence
of the event is the on-disk terminal log.
4. Concurrent broadcasts (thinking deltas + tool calls + status
changes from many tasks) preserve seq order == wire order.
@@ -82,7 +82,7 @@ def _patch_persist_dir():
def _build_app(seq_log):
"""Replicates main.py's WS handler + adds a /test/emit endpoint
so the test thread can drive event emission through the same
event loop as the WS handler — avoiding the cross-loop hazards
event loop as the WS handler; avoiding the cross-loop hazards
of `asyncio.run()` mid-test."""
from backend.apps.agents.ws_manager import ws_manager
@@ -159,7 +159,7 @@ async def _emit_run(session_id: str, n_events: int, terminate: str | None = "com
"message_id": "m1",
"delta": f"chunk-{start + i}",
})
# Yield to the scheduler so other coroutines interleave —
# Yield to the scheduler so other coroutines interleave ,
# this is what surfaces the seq race if locking is wrong.
await asyncio.sleep(0)
@@ -275,7 +275,7 @@ def test_resume_after_disconnect_recovers_all_events(_patch_persist_dir):
def test_terminal_event_visible_after_full_eviction(_patch_persist_dir):
"""If the in-memory log is wiped (process restart simulation),
a reconnecting client should still see the terminal event from
disk — never a phantom 'running' spinner."""
disk; never a phantom 'running' spinner."""
app = _build_app(_patch_persist_dir)
sid = "session-evict-term-1"
@@ -449,7 +449,7 @@ def test_concurrent_broadcast_preserves_order(trial, _patch_persist_dir):
# ---------------------------------------------------------------------------
# Auth/security smoke: the WS endpoint here is unauth'd by design (test
# scaffolding) — but main.py's _ws_auth_ok must remain in place. This
# scaffolding); but main.py's _ws_auth_ok must remain in place. This
# test pins that contract so a future refactor can't accidentally
# strip it.
# ---------------------------------------------------------------------------
@@ -485,7 +485,7 @@ def test_terminate_during_disconnect_is_observable(trial, _patch_persist_dir):
# Disconnected. Emit the rest + terminate while WS is gone.
_emit(client, sid, n=n_post, terminate="completed")
# Reconnect. We expect to receive everything from last_seq+1
# through to the terminal — possibly via disk if the buffer
# through to the terminal; possibly via disk if the buffer
# rolled (it won't here; numbers are small).
with client.websocket_connect(f"/ws/agents/{sid}") as ws:
ws.send_text(json.dumps({"event": "client:hello", "data": {"last_seq": last_seq, "connection_uuid": "c2"}}))
@@ -534,10 +534,10 @@ def test_main_ws_endpoints_still_gated_by_auth(_patch_persist_dir):
src = open(os.path.join(os.path.dirname(__file__), "..", "main.py")).read()
assert "_ws_auth_ok(websocket)" in src, (
"main.py WS endpoints must still call _ws_auth_ok before accepting "
"the connection — otherwise any local web page can read agent traffic."
"the connection; otherwise any local web page can read agent traffic."
)
# And the disconnect handler must NOT call any task-cancel helper
# — that's the regression we're guarding against.
#; that's the regression we're guarding against.
assert "stop_agent" not in src.split("WebSocketDisconnect")[1].split("def ")[0], (
"WebSocketDisconnect handler must not cancel the agent task."
)
backend/tests/test_outputs_runtime_cleanup.py
+303
View File
@@ -0,0 +1,303 @@
"""End-to-end test for the per-workspace runtime cleanup + port collision fix.
What this proves:
1. AppRuntimeManager.stop_all() reaps active runtimes.
2. AppRuntimeManager.stop_all() reaps idle (LRU) runtimes too.
3. AppRuntimeManager.stop_all() resumes SIGSTOP'd idle runtimes before reaping (otherwise the SIGTERM is queued and the process never dies).
4. _is_port_free() correctly detects collisions.
5. _write_env_value() updates a single key without clobbering siblings.
6. _start_new_mode() rewrites .env's FRONTEND_PORT when the persisted port is in use, and the spawned child sees the rewritten value.
7. Same collision-rewrite happens for BACKEND_PORT when it's not "NONE".
Run with: backend/.venv/bin/python backend/tests/test_outputs_runtime_cleanup.py
"""
import asyncio
import os
import shutil
import socket
import subprocess
import sys
import tempfile
import time
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), "..", "..")))
from backend.apps.outputs.runtime import (
AppRuntime,
AppRuntimeManager,
_find_free_port,
_is_port_free,
_read_env_value,
_write_env_value,
)
# --- Fixture: matches the production webapp_template run.sh signal habits ---
# trap cleanup EXIT only, no TERM. Reproduces the actual bug: SIGTERM kills
# bash silently, EXIT trap doesn't fire on uncaught signal, python child gets
# reparented to launchd. _kill_descendant_tree must walk the tree to nuke it.
FAKE_RUN_SH = """#!/bin/bash
set -e
if [ -f .env ]; then
set -a; . ./.env; set +a
fi
echo "[fake-run] FRONTEND_PORT=${FRONTEND_PORT:-unset} pid=$$"
python3 -c "
import socket, time, os
s = socket.socket()
s.bind(('127.0.0.1', int(os.environ['FRONTEND_PORT'])))
s.listen(1)
print(f'[fake-run] bound on {os.environ[\\"FRONTEND_PORT\\"]}', flush=True)
while True:
time.sleep(1)
" &
PYTHON_PID=$!
# Mirror the real template: EXIT trap only. bash's default SIGTERM handler
# exits without running EXIT, so this MUST NOT keep our descendant alive
# if our kill-tree walker works correctly.
cleanup() { kill $PYTHON_PID 2>/dev/null; }
trap cleanup EXIT
wait $PYTHON_PID
"""
def _make_fake_workspace(tmp: str, frontend_port: int, backend_port: str = "NONE") -> str:
ws = os.path.join(tmp, "ws")
os.makedirs(ws)
with open(os.path.join(ws, "run.sh"), "w") as f:
f.write(FAKE_RUN_SH)
os.chmod(os.path.join(ws, "run.sh"), 0o755)
with open(os.path.join(ws, ".env"), "w") as f:
f.write(f"# header comment\nSOMETHING_ELSE=untouched\nFRONTEND_PORT={frontend_port}\nBACKEND_PORT={backend_port}\nTRAILING=keep\n")
return ws
def _pid_alive(pid: int) -> bool:
try:
os.kill(pid, 0)
return True
except OSError:
return False
# --- Test 1: helpers ---
def test_is_port_free():
p = _find_free_port()
assert _is_port_free(p), "freshly-allocated port should be free"
s = socket.socket()
s.bind(("127.0.0.1", p))
s.listen(1)
try:
assert not _is_port_free(p), "_is_port_free must return False while bound"
finally:
s.close()
print("PASS test_is_port_free")
def test_write_env_value():
with tempfile.TemporaryDirectory() as tmp:
env = os.path.join(tmp, ".env")
with open(env, "w") as f:
f.write("A=1\nB=2\nC=3\n# comment\n")
_write_env_value(env, "B", "999")
assert _read_env_value(env, "A") == "1"
assert _read_env_value(env, "B") == "999"
assert _read_env_value(env, "C") == "3"
# New key appended.
_write_env_value(env, "D", "new")
assert _read_env_value(env, "D") == "new"
# Comment line + sibling values preserved.
with open(env) as f:
body = f.read()
assert "# comment" in body, "comment line dropped"
assert "A=1" in body and "C=3" in body
print("PASS test_write_env_value")
# --- Test 2: stop_all reaps an active runtime (real spawn). ---
async def test_stop_all_kills_active():
with tempfile.TemporaryDirectory() as tmp:
port = _find_free_port()
ws = _make_fake_workspace(tmp, port)
m = AppRuntimeManager()
rt = await m.attach("ws1", ws)
assert rt.running, "runtime should be running after attach"
pid = rt.process.pid
# Wait for the child python to actually bind the port.
for _ in range(40):
if not _is_port_free(port):
break
await asyncio.sleep(0.05)
else:
raise AssertionError(f"fake child never bound on {port}")
killed = await m.stop_all()
assert killed >= 1, f"stop_all reported {killed} reaped"
# Bash + python child must be gone within the grace window.
for _ in range(60):
if not _pid_alive(pid):
break
await asyncio.sleep(0.1)
else:
raise AssertionError(f"pid {pid} still alive after stop_all")
# Port must be released too.
for _ in range(40):
if _is_port_free(port):
break
await asyncio.sleep(0.05)
else:
raise AssertionError(f"port {port} not released after stop_all")
assert not m.runtimes and not m._idle_lru, "manager should be empty after stop_all"
print("PASS test_stop_all_kills_active")
# --- Test 3: stop_all reaps an idle (LRU + SIGSTOP'd) runtime. ---
async def test_stop_all_kills_idle():
with tempfile.TemporaryDirectory() as tmp:
port = _find_free_port()
ws = _make_fake_workspace(tmp, port)
m = AppRuntimeManager()
rt = await m.attach("ws-idle", ws)
pid = rt.process.pid
# Detach -> moves into LRU + SIGSTOP'd. If stop_all forgets to
# SIGCONT before SIGTERM, the kill queues and the process hangs.
await m.detach("ws-idle")
assert "ws-idle" in m._idle_lru, "should be in idle LRU"
# Confirm the process is suspended (T state on Linux, T on darwin).
# Skip the OS check; just rely on the eventual kill working.
killed = await m.stop_all()
assert killed == 1
for _ in range(60):
if not _pid_alive(pid):
break
await asyncio.sleep(0.1)
else:
raise AssertionError("idle process never died, stop_all probably didn't SIGCONT first")
for _ in range(40):
if _is_port_free(port):
break
await asyncio.sleep(0.05)
else:
raise AssertionError("port from idle runtime not released")
print("PASS test_stop_all_kills_idle")
# --- Test 4: persisted port collision triggers .env rewrite + new spawn. ---
async def test_port_collision_reallocates_env():
with tempfile.TemporaryDirectory() as tmp:
squatted_port = _find_free_port()
ws = _make_fake_workspace(tmp, squatted_port)
# Squat the persisted port so the runtime can't use it.
squatter = socket.socket()
squatter.bind(("127.0.0.1", squatted_port))
squatter.listen(1)
try:
m = AppRuntimeManager()
rt = await m.attach("ws-collide", ws)
# Wait for either spawn-failure or new port binding.
for _ in range(40):
if rt.frontend_port and rt.frontend_port != squatted_port:
break
await asyncio.sleep(0.05)
assert rt.frontend_port != squatted_port, \
f"frontend_port should have changed from {squatted_port}, got {rt.frontend_port}"
# .env should reflect the new port (so run.sh and subsequent
# restarts pick it up too).
written = _read_env_value(os.path.join(ws, ".env"), "FRONTEND_PORT")
assert written == str(rt.frontend_port), \
f".env not rewritten; expected {rt.frontend_port}, found {written}"
# Sibling .env keys untouched.
assert _read_env_value(os.path.join(ws, ".env"), "SOMETHING_ELSE") == "untouched"
assert _read_env_value(os.path.join(ws, ".env"), "TRAILING") == "keep"
await m.stop_all()
finally:
squatter.close()
print("PASS test_port_collision_reallocates_env")
# --- Test 5: stop_all is idempotent. ---
async def test_stop_all_idempotent():
m = AppRuntimeManager()
n = await m.stop_all()
assert n == 0
n = await m.stop_all()
assert n == 0
print("PASS test_stop_all_idempotent")
# --- Test 6: vite-like grandchild dies even with EXIT-only trap. ---
async def test_descendant_tree_killed_despite_exit_only_trap():
"""Regression for the actual prod bug: webapp_template run.sh has only
`trap cleanup EXIT` (no TERM), so a flat SIGTERM to bash exits bash
silently and reparents the vite/uvicorn grandchild to PID 1. stop()
must walk the descendant tree to nuke the grandchild explicitly."""
with tempfile.TemporaryDirectory() as tmp:
port = _find_free_port()
ws = _make_fake_workspace(tmp, port)
m = AppRuntimeManager()
rt = await m.attach("ws-tree", ws)
bash_pid = rt.process.pid
# Wait until the python grandchild is actually listening on the port,
# so we know it exists as a separate process.
for _ in range(60):
if not _is_port_free(port):
break
await asyncio.sleep(0.05)
else:
raise AssertionError("grandchild never bound the port")
# Find the grandchild PID via pgrep -P (same call our walker uses).
out = subprocess.run(
["pgrep", "-P", str(bash_pid)],
capture_output=True, text=True, timeout=2,
)
grand_pids = [int(p) for p in out.stdout.split() if p.strip().isdigit()]
assert grand_pids, "expected at least one bash child"
# The python process may be one further level down (`python -c ...` is
# the leaf, bash spawned via `&` puts it directly under bash).
all_descendants: list[int] = []
def collect(pid: int) -> None:
r = subprocess.run(
["pgrep", "-P", str(pid)],
capture_output=True, text=True, timeout=2,
)
for line in r.stdout.split():
if line.strip().isdigit():
pid_i = int(line)
all_descendants.append(pid_i)
collect(pid_i)
for g in grand_pids:
all_descendants.append(g)
collect(g)
await m.stop_all()
# Every descendant must be gone, not just bash.
for _ in range(80):
still_alive = [p for p in all_descendants if _pid_alive(p)]
if not still_alive:
break
await asyncio.sleep(0.1)
else:
raise AssertionError(
f"descendants still alive after stop_all: {still_alive} "
"(EXIT-only trap let them escape)"
)
for _ in range(40):
if _is_port_free(port):
break
await asyncio.sleep(0.05)
else:
raise AssertionError(f"port {port} still held by ghost grandchild")
print("PASS test_descendant_tree_killed_despite_exit_only_trap")
async def main():
test_is_port_free()
test_write_env_value()
await test_stop_all_idempotent()
await test_stop_all_kills_active()
await test_stop_all_kills_idle()
await test_port_collision_reallocates_env()
await test_descendant_tree_killed_despite_exit_only_trap()
print("\nALL TESTS PASSED")
if __name__ == "__main__":
asyncio.run(main())
backend/tests/test_phase1_stress.py
+7 -7
View File
@@ -28,7 +28,7 @@ os.environ.setdefault("OPENSWARM_DATA_DIR", _TMPROOT)
# ---------------------------------------------------------------------------
# Group 1 — Message.client_message_id
# Group 1; Message.client_message_id
# ---------------------------------------------------------------------------
@@ -51,7 +51,7 @@ def test_message_round_trips_client_id():
def test_message_legacy_payload_without_client_id():
"""Older session JSON files won't have the field — must still load."""
"""Older session JSON files won't have the field; must still load."""
from backend.apps.agents.models import Message
legacy = {
@@ -82,7 +82,7 @@ def test_client_message_id_collision_resistance():
# ---------------------------------------------------------------------------
# Group 2 — Mode migration: chat → ask
# Group 2; Mode migration: chat → ask
# ---------------------------------------------------------------------------
@@ -196,7 +196,7 @@ def test_reconcile_idempotent():
# ---------------------------------------------------------------------------
# Group 6 — Notes layout serialization
# Group 6; Notes layout serialization
# ---------------------------------------------------------------------------
@@ -254,11 +254,11 @@ def test_notes_stress_many_round_trips():
# ---------------------------------------------------------------------------
# Group 7 — Concurrent send_message dedupe stress
# Group 7; Concurrent send_message dedupe stress
#
# Real-world scenario: user mashes Enter quickly. 50 concurrent sends
# each with a unique client_message_id must produce 50 echoed messages
# carrying the right ids. Pure pydantic / asyncio test — no real
# carrying the right ids. Pure pydantic / asyncio test; no real
# agent loop.
# ---------------------------------------------------------------------------
@@ -266,7 +266,7 @@ def test_notes_stress_many_round_trips():
@pytest.mark.asyncio
async def test_concurrent_send_message_unique_client_ids():
"""100 parallel Message constructions with unique client_message_ids
must round-trip independently — no cross-talk on the dataclass."""
must round-trip independently; no cross-talk on the dataclass."""
from backend.apps.agents.models import Message
async def make_one(i: int):
backend/tests/test_schedule_e2e.py
+277
View File
@@ -0,0 +1,277 @@
"""End-to-end smoke: does a scheduled workflow actually fire when its
time hits, with the full scheduler loop running?
Runs the real scheduler.start() loop with the executor mocked so we
don't need a live agent_manager. Then arms a workflow whose
next_run_at is one second in the future, waits, and asserts the
mocked executor was called.
Run:
cd backend && .venv/bin/python -m pytest tests/test_schedule_e2e.py -v
"""
from __future__ import annotations
import asyncio
from datetime import datetime, timedelta, timezone
from unittest.mock import AsyncMock
from zoneinfo import ZoneInfo
import pytest
pytestmark = pytest.mark.asyncio
@pytest.fixture(autouse=True)
def isolated_data_dir(monkeypatch, tmp_path):
from backend.apps.workflows import storage as _storage
from backend.apps.workflows import escalation as _escalation
from backend.apps.workflows import audit as _audit
from backend.apps.workflows import scheduler as _scheduler
monkeypatch.setattr(_storage, "DATA_DIR", str(tmp_path / "workflows"))
monkeypatch.setattr(_storage, "RUNS_DIR", str(tmp_path / "workflows" / "runs"))
monkeypatch.setattr(_storage, "PAUSED_FILE", str(tmp_path / "workflows" / "paused.json"))
monkeypatch.setattr(_storage, "_workflow_cache", {})
monkeypatch.setattr(_storage, "_runs_cache", {})
monkeypatch.setattr(_storage, "_cache_loaded", False)
monkeypatch.setattr(_storage, "_paused", False)
monkeypatch.setattr(_audit, "AUDIT_DIR", str(tmp_path / "workflows" / "audit"))
# Module-level scheduler state survives across tests; reset it so
# each test gets a fresh _wake Event bound to its own event loop.
_scheduler._loop_task = None
_scheduler._wake = asyncio.Event()
_escalation._tasks.clear()
_escalation._state.clear()
yield
def _make_wf(**overrides):
from backend.apps.workflows.models import Workflow, ScheduleConfig, WorkflowStep
base = dict(
title="smoke",
steps=[WorkflowStep(text="hi")],
schedule=ScheduleConfig(
enabled=True, repeat_unit="day", repeat_every=1,
hour=9, minute=0, timezone="America/Los_Angeles",
),
)
base.update(overrides)
return Workflow(**base)
async def test_loop_fires_due_workflow(monkeypatch):
"""Arm a workflow to fire ~now and assert the executor was actually
invoked by the scheduler loop within the test window. Note the save
happens AFTER scheduler.start() so reconcile_on_startup doesn't
clobber next_run_at."""
from backend.apps.workflows import storage, scheduler, executor
fired = asyncio.Event()
captured: dict = {}
async def fake_execute(wf, triggered_by="schedule", scheduled_for=None):
captured["wf_id"] = wf.id
captured["triggered_by"] = triggered_by
captured["scheduled_for"] = scheduled_for
from backend.apps.workflows.models import WorkflowRun
run = WorkflowRun(
workflow_id=wf.id,
status="success",
scheduled_for=scheduled_for,
started_at=datetime.now(timezone.utc),
finished_at=datetime.now(timezone.utc),
triggered_by=triggered_by,
)
storage.record_run(run)
fired.set()
return run
monkeypatch.setattr(executor, "execute", fake_execute)
await scheduler.start()
try:
wf = _make_wf()
wf.next_run_at = datetime.now(timezone.utc) + timedelta(seconds=1)
storage.save_workflow(wf)
scheduler.kick() # force immediate tick
# Wait up to 5s for the fire to land.
await asyncio.wait_for(fired.wait(), timeout=5.0)
finally:
await scheduler.stop()
assert captured.get("wf_id") == wf.id
assert captured.get("triggered_by") == "schedule"
runs = storage.list_runs(wf.id, limit=10)
assert len(runs) == 1
assert runs[0].status == "success"
# Scheduler should have rolled next_run_at forward to a future slot.
after = storage.get_workflow(wf.id)
assert after.next_run_at is not None
assert after.next_run_at > datetime.now(timezone.utc)
async def test_disabled_workflow_does_not_fire(monkeypatch):
"""Master switch off => loop never invokes the executor even if
next_run_at is in the past."""
from backend.apps.workflows import storage, scheduler, executor
fake = AsyncMock()
monkeypatch.setattr(executor, "execute", fake)
wf = _make_wf()
wf.schedule.enabled = False
wf.next_run_at = datetime.now(timezone.utc) - timedelta(seconds=10)
storage.save_workflow(wf)
await scheduler.start()
try:
scheduler.kick()
await asyncio.sleep(2.0)
finally:
await scheduler.stop()
fake.assert_not_called()
async def test_paused_state_blocks_all_fires(monkeypatch):
"""Global pause flag wins over per-workflow enabled state."""
from backend.apps.workflows import storage, scheduler, executor
fake = AsyncMock()
monkeypatch.setattr(executor, "execute", fake)
wf = _make_wf()
wf.next_run_at = datetime.now(timezone.utc) - timedelta(seconds=1)
storage.save_workflow(wf)
storage.set_paused(True)
await scheduler.start()
try:
scheduler.kick()
await asyncio.sleep(2.0)
finally:
await scheduler.stop()
fake.assert_not_called()
storage.set_paused(False)
async def test_reconcile_skip_rolls_past_missed(monkeypatch):
"""on_missed='skip' + a missed next_run_at => startup rolls forward
to the next future fire without queuing a catch-up."""
from backend.apps.workflows import storage, scheduler
wf = _make_wf()
wf.schedule.on_missed = "skip"
# Stash a missed fire 6 hours ago.
wf.next_run_at = datetime.now(timezone.utc) - timedelta(hours=6)
storage.save_workflow(wf)
scheduler.reconcile_on_startup()
after = storage.get_workflow(wf.id)
assert after.next_run_at is not None
assert after.next_run_at > datetime.now(timezone.utc)
async def test_reconcile_run_once_keeps_missed(monkeypatch):
"""on_missed='run_once' => startup leaves next_run_at in the past so
the first tick fires a catch-up."""
from backend.apps.workflows import storage, scheduler
wf = _make_wf()
wf.schedule.on_missed = "run_once"
missed = datetime.now(timezone.utc) - timedelta(hours=6)
wf.next_run_at = missed
storage.save_workflow(wf)
scheduler.reconcile_on_startup()
after = storage.get_workflow(wf.id)
assert after.next_run_at <= datetime.now(timezone.utc)
async def test_create_workflow_schedules_next_fire():
"""POST-like create path: enabled schedule => next_run_at populated
by compute_next_fire."""
from backend.apps.workflows.models import Workflow, ScheduleConfig, WorkflowStep
from backend.apps.workflows import scheduler
wf = Workflow(
title="t",
steps=[WorkflowStep(text="hi")],
schedule=ScheduleConfig(
enabled=True, repeat_unit="week", repeat_every=1, on_days=[0],
hour=9, minute=0, timezone="America/Los_Angeles",
),
)
nxt = scheduler.compute_next_fire(wf)
assert nxt is not None
assert nxt > datetime.now(timezone.utc)
tz = ZoneInfo("America/Los_Angeles")
local = nxt.astimezone(tz)
assert local.weekday() == 6 # Python: Sunday
assert (local.hour, local.minute) == (9, 0)
async def test_next_run_at_advances_after_fire(monkeypatch):
"""After a fire the loop should re-compute next_run_at into the
future and persist it, so the same fire can't repeat in the same
minute."""
from backend.apps.workflows import storage, scheduler, executor
fired = asyncio.Event()
async def fake_execute(wf, triggered_by="schedule", scheduled_for=None):
from backend.apps.workflows.models import WorkflowRun
run = WorkflowRun(
workflow_id=wf.id, status="success", scheduled_for=scheduled_for,
started_at=datetime.now(timezone.utc), finished_at=datetime.now(timezone.utc),
triggered_by=triggered_by,
)
storage.record_run(run)
fired.set()
return run
monkeypatch.setattr(executor, "execute", fake_execute)
await scheduler.start()
try:
wf = _make_wf()
armed_at = datetime.now(timezone.utc) + timedelta(seconds=1)
wf.next_run_at = armed_at
storage.save_workflow(wf)
scheduler.kick()
await asyncio.wait_for(fired.wait(), timeout=5.0)
# Give the loop one extra tick to persist next_run_at.
await asyncio.sleep(0.2)
finally:
await scheduler.stop()
after = storage.get_workflow(wf.id)
assert after.next_run_at is not None
assert after.next_run_at > armed_at, "scheduler did not advance next_run_at past the slot it just fired"
async def test_kick_wakes_loop_before_timeout(monkeypatch):
"""kick() should wake the loop early so manual schedule edits don't
have to wait a full minute for the next tick boundary."""
from backend.apps.workflows import storage, scheduler, executor
fired = asyncio.Event()
async def fake_execute(wf, triggered_by="schedule", scheduled_for=None):
from backend.apps.workflows.models import WorkflowRun
run = WorkflowRun(
workflow_id=wf.id, status="success",
started_at=datetime.now(timezone.utc),
finished_at=datetime.now(timezone.utc), triggered_by=triggered_by,
)
storage.record_run(run)
fired.set()
return run
monkeypatch.setattr(executor, "execute", fake_execute)
await scheduler.start()
try:
wf = _make_wf()
wf.next_run_at = datetime.now(timezone.utc) - timedelta(seconds=1)
storage.save_workflow(wf)
scheduler.kick()
# Without kick(), the loop would sleep up to 60s before checking
# the freshly-saved workflow. With kick, it should fire fast.
await asyncio.wait_for(fired.wait(), timeout=3.0)
finally:
await scheduler.stop()
backend/tests/test_service_legacy.py
+3 -3
View File
@@ -45,7 +45,7 @@ def install_sync_sink():
cs = body.get("client_state") or {}
payload = body.get("d") or body.get("payload") or {}
# Infer a synthetic kind from payload shape — same dispatch logic
# Infer a synthetic kind from payload shape; same dispatch logic
# as the cloud uses in production.
if "status" in payload and "messages" in payload:
status = payload.get("status", "unknown")
@@ -147,7 +147,7 @@ def manager():
# ---------------------------------------------------------------------------
# 1. record() — legacy shim correctness
# 1. record(); legacy shim correctness
# ---------------------------------------------------------------------------
class TestRecordBasics:
@@ -175,7 +175,7 @@ class TestRecordBasics:
# ---------------------------------------------------------------------------
# 2. Multi-message session — close fires exactly once
# 2. Multi-message session; close fires exactly once
# ---------------------------------------------------------------------------
class TestMultiMessageSession:
backend/tests/test_v2_invariants.py
+29 -193
View File
@@ -36,10 +36,6 @@ _TMPROOT = tempfile.mkdtemp(prefix="openswarm-v2-invariants-")
os.environ.setdefault("OPENSWARM_DATA_DIR", _TMPROOT)
# ---------------------------------------------------------------------------
# Fixture: build a fake ToolDefinition without touching disk.
# ---------------------------------------------------------------------------
def _fake_tool(
name: str,
*,
@@ -60,13 +56,10 @@ def _fake_tool(
)
# ===========================================================================
# Group A — MCP activation gate (the non-bypassable ToolSearch invariant)
# ===========================================================================
# The product invariant: NO MCP tool is callable until the model has
# explicitly searched + activated the server, and the user has approved
# the activation. The gate lives at the dispatch layer in
# `_build_mcp_servers` — even if the prompt rules are ignored, the SDK
# `_build_mcp_servers`; even if the prompt rules are ignored, the SDK
# never sees the unactivated server.
@@ -82,7 +75,6 @@ async def test_gate_blocks_when_active_mcps_empty():
with patch("backend.apps.agents.agent_manager.load_all_tools", return_value=fake_tools), \
patch("backend.apps.agents.agent_manager.refresh_google_token", new=AsyncMock(return_value=True)):
mgr = AgentManager()
# allowed_tools includes mcp:Gmail, but active_mcps is empty
result = await mgr._build_mcp_servers(
allowed_tools=["mcp:Gmail", "mcp:Slack", "mcp:Notion"],
active_mcps=[],
@@ -185,11 +177,9 @@ async def test_gate_stress_random_activations():
fake_tools = [_fake_tool(raw_names[i]) for i in connected_idx]
connected_sanitized = [server_pool[i] for i in connected_idx]
# active set is a random subset of connected
active_n = random.randint(0, len(connected_sanitized))
active = random.sample(connected_sanitized, active_n)
# allowed_tools mirrors raw names of connected
allowed = [f"mcp:{raw_names[i]}" for i in connected_idx]
with patch("backend.apps.agents.agent_manager.load_all_tools", return_value=fake_tools), \
@@ -202,7 +192,6 @@ async def test_gate_stress_random_activations():
active_mcps=active,
)
keys = set(result.keys())
# MUST: keys ⊆ active ∩ connected
allowed_set = set(active) & set(connected_sanitized)
assert keys.issubset(allowed_set), (
f"GATE BREACH: {keys - allowed_set} leaked through "
@@ -210,14 +199,6 @@ async def test_gate_stress_random_activations():
)
# ===========================================================================
# Group B — needs_fresh_session soft-restart
# ===========================================================================
# When MCPActivate fires mid-session, the bundled CLI doesn't re-read
# mcp_servers from a fork. We force a fresh sdk_session_id so the new
# server's tools actually reach the model.
def test_needs_fresh_session_field_default_false():
"""Brand-new sessions must default needs_fresh_session=False."""
from backend.apps.agents.models import AgentSession
@@ -239,7 +220,7 @@ def test_needs_fresh_session_serializes_round_trip():
def test_legacy_session_json_loads_without_field():
"""Old session JSONs predate the field — Pydantic must fill in default."""
"""Old session JSONs predate the field; Pydantic must fill in default."""
from backend.apps.agents.models import AgentSession
legacy = {
"id": "old", "name": "legacy", "model": "sonnet", "mode": "agent",
@@ -247,7 +228,6 @@ def test_legacy_session_json_loads_without_field():
}
s = AgentSession.model_validate(legacy)
assert s.needs_fresh_session is False
# extras silently absorbed → can't be a regression hazard
legacy_with_ghost = {**legacy, "answer_tokens": 999, "thought_signature": "abc=="}
s2 = AgentSession.model_validate(legacy_with_ghost)
assert s2.id == "old"
@@ -256,10 +236,8 @@ def test_legacy_session_json_loads_without_field():
def test_mcp_activate_sets_fresh_session_when_history_exists():
"""The gate logic at main.py: if sdk_session_id exists, set needs_fresh_session=True."""
from backend.apps.agents.models import AgentSession
# Mid-session: sdk already locked in
s = AgentSession(id="mid", name="t", model="sonnet", mode="agent")
s.sdk_session_id = "claude-session-existing"
# Simulate the gate handler logic
if s.sdk_session_id:
s.needs_fresh_session = True
assert s.needs_fresh_session is True
@@ -269,7 +247,6 @@ def test_mcp_activate_skips_fresh_session_on_first_turn():
"""First-turn activation: no sdk_session_id yet, so needs_fresh_session stays False."""
from backend.apps.agents.models import AgentSession
s = AgentSession(id="fresh", name="t", model="sonnet", mode="agent")
# No sdk_session_id yet
if s.sdk_session_id:
s.needs_fresh_session = True
assert s.needs_fresh_session is False
@@ -285,9 +262,6 @@ def test_active_mcps_append_idempotent():
assert s.active_mcps.count("gmail") == 1
# ===========================================================================
# Group C — Pydantic Message backward compat (no ghost fields, legacy loads)
# ===========================================================================
def test_message_no_ghost_fields():
@@ -300,7 +274,7 @@ def test_message_no_ghost_fields():
def test_message_legacy_payload_with_ghost_fields_still_loads():
"""Old session JSONs may carry the deleted fields — Pydantic must ignore them."""
"""Old session JSONs may carry the deleted fields; Pydantic must ignore them."""
from backend.apps.agents.models import Message
legacy = {
"id": "m1",
@@ -312,10 +286,8 @@ def test_message_legacy_payload_with_ghost_fields_still_loads():
"input_tokens": 1234,
}
m = Message.model_validate(legacy)
# Fields that survived are preserved
assert m.tool_count == 3
assert m.input_tokens == 1234
# Ghost fields don't blow up + don't leak into re-dump
redumped = m.model_dump(mode="json")
assert "answer_tokens" not in redumped
assert "thought_signature" not in redumped
@@ -358,9 +330,6 @@ def test_message_round_trip_50_iterations():
assert m2.elapsed_ms == m.elapsed_ms
# ===========================================================================
# Group D — resolve_aux_model Gemini route (the gemini-3.1-flash-lite-preview fix)
# ===========================================================================
@pytest.mark.asyncio
@@ -464,11 +433,10 @@ async def test_resolve_aux_model_openrouter_primary_prefers_or():
@pytest.mark.asyncio
async def test_resolve_aux_model_openrouter_priority_after_subs():
"""In the default cascade (no primary_api), Claude/Codex/Gemini subs
win over OR — OR is metered while subs are sub-covered free."""
win over OR; OR is metered while subs are sub-covered free."""
from backend.apps.agents.providers import registry
from backend.apps.settings.models import AppSettings
settings = AppSettings()
# Both Codex and OR connected — Codex (free via sub) should win.
with patch("backend.apps.nine_router.is_running", return_value=True), \
patch("backend.apps.nine_router.get_providers",
new=AsyncMock(return_value=[
@@ -479,14 +447,6 @@ async def test_resolve_aux_model_openrouter_priority_after_subs():
assert model_id == "cx/gpt-5.4-mini", f"got {model_id}"
# ===========================================================================
# Group E — 9Router-streamed 401 detection
# ===========================================================================
# 9Router sometimes returns upstream auth failures AS the assistant's
# reply text, not as an exception. We detect the pattern in the stream
# handler to substitute a friendly bubble.
def test_router_auth_pattern_codex():
"""The pattern detector at agent_manager.py:2841-2846."""
text = (
@@ -520,7 +480,7 @@ def test_router_auth_pattern_does_not_falsely_match_normal_text():
"Here are your recent emails: ...",
"I found 3 results for your search.",
"Sorry, I don't have access to that file.",
"401 Unauthorized — wait this is a code example I'm explaining", # tricky
"401 Unauthorized; wait this is a code example I'm explaining", # tricky
]
for text in benign_replies:
lower = text.lower()
@@ -537,7 +497,6 @@ def test_is_auth_error_classifier():
"""The classifier at agent_manager.py:_is_auth_error covers many shapes."""
from backend.apps.agents.agent_manager import _is_auth_error
# Real shapes that must be caught
matches = [
Exception("Error 401: invalid_api_key"),
Exception("Got 403 from upstream"),
@@ -550,7 +509,6 @@ def test_is_auth_error_classifier():
for e in matches:
assert _is_auth_error(e), f"should match: {e}"
# Non-auth errors must not match
non_matches = [
Exception("Connection timeout"),
Exception("Rate limit exceeded"),
@@ -569,14 +527,6 @@ def test_is_auth_error_with_stderr_tail():
assert _is_auth_error(e, extra_text=stderr)
# ===========================================================================
# Group F — MCP_SERVER_BRAND coverage
# ===========================================================================
# Every server slug we surface to the user via MCPSearch / connected_servers
# should have a brand entry, otherwise the UI falls back to the kebab-case
# id ("microsoft-365" instead of "Microsoft 365").
def test_mcp_brand_covers_curated_servers():
"""Every curated server slug must already be in canonical sanitized form."""
curated = {
@@ -627,19 +577,13 @@ def test_sanitize_server_name_strips_special_chars():
assert _sanitize_server_name("a__b") == "a-b"
# ===========================================================================
# Group G — mcp_meta_server activation backend handler
# ===========================================================================
def test_mcp_activate_handler_unknown_server():
"""Unknown server name → status='unknown_server' with the valid list."""
# We test the response shape independently of the FastAPI plumbing.
# The handler is a closure inside main.py:mcp_meta_handler, so we
# instead exercise the contract: invalid name surfaces alternatives.
from backend.apps.tools_lib.tools_lib import _sanitize_server_name
valid = {"gmail", "slack", "google-workspace"}
requested = "Gmail" # raw, needs sanitize
requested = "Gmail"
sanitized = _sanitize_server_name(requested)
if sanitized in valid:
status = "would_activate"
@@ -649,7 +593,7 @@ def test_mcp_activate_handler_unknown_server():
def test_active_mcps_persistence_on_session():
"""active_mcps survives session.model_dump() round-trip — critical for resume."""
"""active_mcps survives session.model_dump() round-trip; critical for resume."""
from backend.apps.agents.models import AgentSession
s = AgentSession(id="x", name="t", model="sonnet", mode="agent")
s.active_mcps = ["gmail", "slack"]
@@ -658,9 +602,6 @@ def test_active_mcps_persistence_on_session():
assert rehydrated.active_mcps == ["gmail", "slack"]
# ===========================================================================
# Group H — long-context error classifier
# ===========================================================================
def test_long_context_pattern_caught():
@@ -690,10 +631,7 @@ def test_transient_capacity_patterns():
]
for t in transients:
assert _TRANSIENT_CAPACITY_PATTERNS.search(t), f"transient missed: {t!r}"
# Importantly: must NOT also match non-transient (no double-classification)
# except for the fuzzy edge cases. Spot-check a couple:
if "429" in t and "rate_limit" in t.lower():
# rate_limit_error is transient; non-transient should not match this exact text
assert not _NON_TRANSIENT_PATTERNS.search(t)
@@ -703,9 +641,6 @@ def test_long_context_does_not_match_normal_429():
assert not _NON_TRANSIENT_PATTERNS.search("Error 429: rate_limit_error")
# ===========================================================================
# Group I — Mode reconciliation (regression guard)
# ===========================================================================
def test_chat_mode_not_in_builtins():
@@ -726,9 +661,6 @@ def test_active_mcps_default_factory_creates_new_list():
assert s2.active_mcps == [], "active_mcps must not share state across sessions"
# ===========================================================================
# Group J — Concurrent gate stress (real production risk: simultaneous turns)
# ===========================================================================
@pytest.mark.asyncio
@@ -752,9 +684,6 @@ async def test_concurrent_gate_calls_isolated():
assert set(empty.keys()) == set()
# ===========================================================================
# Group K — pending_continuation auto-restart
# ===========================================================================
def test_pending_continuation_default_false():
@@ -776,7 +705,7 @@ def test_pending_continuation_serializes():
def test_compact_threshold_default():
"""compact_threshold_pct default of 0.65 — drift here breaks Phase 2 compaction."""
"""compact_threshold_pct default of 0.65; drift here breaks Phase 2 compaction."""
from backend.apps.agents.models import AgentSession
s = AgentSession(id="x", name="t", model="sonnet", mode="agent")
assert s.compact_threshold_pct == 0.65
@@ -784,13 +713,6 @@ def test_compact_threshold_default():
assert s.context_window == 200_000
# ===========================================================================
# Group L — Sentence-case display (the parseMcpToolName fix)
# ===========================================================================
# This is technically a frontend behavior, but we mirror the rule in
# Python so the backend's MCPSearch results don't leak Title Case either.
def test_sentence_case_rule():
"""Mirror of the JS _humanizeName: first word capitalized, rest lower."""
def sentence_case(name: str) -> str:
@@ -807,9 +729,6 @@ def test_sentence_case_rule():
assert sentence_case(raw) == expected
# ===========================================================================
# Group M — Bash command verb extraction (frontend logic, mirrored)
# ===========================================================================
def test_bash_verb_extraction_strips_env_prefix():
@@ -848,9 +767,6 @@ def test_bash_command_detail_path_basename():
assert basename(raw) == expected
# ===========================================================================
# Group N — Pydantic AppSettings invariants
# ===========================================================================
def test_app_settings_defaults():
@@ -874,9 +790,6 @@ def test_custom_provider_round_trip():
assert s2.custom_providers[0].name == "MyCorp"
# ===========================================================================
# Group O — Tool gate stress with denied permissions
# ===========================================================================
@pytest.mark.asyncio
@@ -887,7 +800,6 @@ async def test_gate_partially_denied_tool_blocked():
"_tool_descriptions": {"send_email": "Send email"},
"send_email": "deny",
})
# Build a minimal class that has the perms_dict shape _is_fully_denied expects
assert _is_fully_denied(fake) in (True, False)
@@ -903,13 +815,9 @@ async def test_gate_handles_missing_refresh_token_gracefully():
allowed_tools=["mcp:MyApiTool"],
active_mcps=["myapitool"],
)
# It should be present (configured + activated + not denied)
assert "myapitool" in result
# ===========================================================================
# Group P — resolve_aux_model failover logic
# ===========================================================================
@pytest.mark.asyncio
@@ -922,10 +830,9 @@ async def test_aux_failover_anthropic_to_codex():
settings.openswarm_proxy_url = "https://api.openswarm.test"
with patch("backend.apps.nine_router.is_running", return_value=True), \
patch("backend.apps.nine_router.get_providers",
new=AsyncMock(return_value=[])): # nothing connected
# primary_api=codex but codex not connected → cascade to Pro/anthropic
new=AsyncMock(return_value=[])):
model_id, base = await registry.resolve_aux_model(settings, primary_api="codex")
assert "haiku" in model_id # fallthrough hit Anthropic Pro path
assert "haiku" in model_id
assert base == "https://api.openswarm.test"
@@ -953,14 +860,10 @@ async def test_aux_returns_sonnet_when_preferred_tier_set():
assert "sonnet" in model_id
# ===========================================================================
# Group Q — get_api_type / model id resolution
# ===========================================================================
def test_get_api_type_openai():
from backend.apps.agents.providers.registry import get_api_type
# gpt-5.4 maps to codex (the OpenAI-via-Codex-subscription api family)
api = get_api_type("gpt-5.4")
assert api in ("openai", "codex"), f"unexpected: {api}"
@@ -977,9 +880,6 @@ def test_find_builtin_model_returns_dict_for_known():
assert sonnet.get("api") == "anthropic"
# ===========================================================================
# Group R — context window
# ===========================================================================
def test_get_context_window_known_model():
@@ -994,11 +894,6 @@ def test_get_context_window_unknown_returns_default():
assert cw == 128_000
# ---------------------------------------------------------------------------
# Custom OpenAI-compatible providers (Ollama Cloud, Together, etc.)
# ---------------------------------------------------------------------------
def test_custom_provider_value_synthesises_route_api_entry():
"""`custom/<slug>/<bare>` picker values must synthesise a route='api',
api='custom' entry whose model_id is the 9Router routing string
@@ -1044,7 +939,6 @@ def test_custom_provider_lookup_finds_entry_by_slug():
assert cp is not None and cp.name == "Ollama Cloud"
cp2 = _find_custom_provider_for_value(s, "custom/together-ai/meta-llama/llama-3-70b")
assert cp2 is not None and cp2.name == "Together AI"
# Unknown slug → None.
assert _find_custom_provider_for_value(s, "custom/nonexistent/whatever") is None
@@ -1066,7 +960,7 @@ def test_get_context_window_custom_provider_value_format():
def test_custom_provider_slug_is_url_safe():
"""The slug must be alnum-and-dash only — it's used both as the 9Router
"""The slug must be alnum-and-dash only; it's used both as the 9Router
prefix and as a URL path segment. Spaces, slashes, and special chars
must all be folded to dashes."""
from backend.apps.agents.providers.registry import _custom_provider_slug_for_lookup
@@ -1079,11 +973,8 @@ def test_custom_provider_slug_is_url_safe():
def test_custom_provider_slug_unicode_collapses_safely():
"""Unicode names are folded to ASCII-safe dashes; emojis/accents drop."""
from backend.apps.agents.providers.registry import _custom_provider_slug_for_lookup
# Accented chars get stripped (regex is [a-zA-Z0-9-] only).
assert _custom_provider_slug_for_lookup("Tögether AI 🚀") == "t-gether-ai"
# Pure-emoji name → fallback "custom".
assert _custom_provider_slug_for_lookup("🚀💎") == "custom"
# Trailing/leading dashes get stripped.
assert _custom_provider_slug_for_lookup("---weird---") == "weird"
@@ -1097,7 +988,6 @@ def test_custom_provider_slug_does_not_collide_with_routing_prefixes():
assert entry is not None
routed = entry["model_id"]
assert routed == "cp-cc/whatever"
# cp-cc is NOT cc/ — startswith check would have to match the exact slash.
assert not routed.startswith(("cc/", "cx/", "gc/", "ag/", "gemini/", "openrouter/"))
@@ -1117,7 +1007,6 @@ def test_custom_provider_models_with_special_chars():
for v in cases:
e = _find_builtin_model(v)
assert e is not None, f"failed: {v}"
# Bare-model portion is everything after first slash after the slug.
rest = v[len("custom/"):]
slug, _, bare = rest.partition("/")
assert e["model_id"] == f"cp-{slug}/{bare}", f"bad routing for {v}: {e['model_id']}"
@@ -1125,7 +1014,7 @@ def test_custom_provider_models_with_special_chars():
def test_custom_provider_value_with_invalid_format_returns_none():
"""Malformed picker values (no slug, no model) must not synthesise a
bogus entry — they should miss _find_builtin_model entirely so the
bogus entry; they should miss _find_builtin_model entirely so the
dispatch loop falls through to the 'unknown model' branch."""
from backend.apps.agents.providers.registry import _find_builtin_model
assert _find_builtin_model("custom/") is None
@@ -1134,7 +1023,7 @@ def test_custom_provider_value_with_invalid_format_returns_none():
def test_custom_provider_get_api_type_returns_custom():
"""get_api_type drives the dispatch branch in agent_manager.py — must
"""get_api_type drives the dispatch branch in agent_manager.py; must
return 'custom' (not 'anthropic' default fallback) for a custom value."""
from backend.apps.agents.providers.registry import get_api_type
assert get_api_type("custom/ollama/gpt-oss:120b") == "custom"
@@ -1188,7 +1077,7 @@ def test_custom_provider_get_anthropic_client_routes_cp_to_9router():
def test_custom_provider_two_providers_get_distinct_slugs():
"""Two custom providers with different display names must produce
two different slugs / routing prefixes — otherwise 9Router will route
two different slugs / routing prefixes; otherwise 9Router will route
both to whichever connection was created last."""
from backend.apps.agents.providers.registry import _custom_provider_slug_for_lookup
a = _custom_provider_slug_for_lookup("Ollama Cloud")
@@ -1203,7 +1092,7 @@ def test_custom_provider_slug_collision_after_sanitize():
The dedupe-by-name UI check guards against same-string entries; this
test just documents that post-slug collisions DO collide and the
UI-level uniqueness check (in Settings.tsx) is the right enforcement
layer — backend resolution would always pick the first match."""
layer; backend resolution would always pick the first match."""
from backend.apps.agents.providers.registry import _custom_provider_slug_for_lookup
assert _custom_provider_slug_for_lookup("Ollama Cloud") == \
_custom_provider_slug_for_lookup("ollama-cloud") == \
@@ -1221,22 +1110,18 @@ def test_list_models_includes_complete_custom_providers_excludes_incomplete():
from unittest.mock import patch
cfg = AppSettings(custom_providers=[
# Complete — should appear.
CustomProvider(
name="Ollama Cloud", base_url="https://ollama.com/v1", api_key="x",
models=[{"value": "gpt-oss:120b", "label": "gpt-oss:120b"}],
),
# Empty base_url — should NOT appear.
CustomProvider(
name="Broken", base_url="", api_key="y",
models=[{"value": "model-a", "label": "model-a"}],
),
# No models — should NOT appear.
CustomProvider(
name="Empty", base_url="https://example.com/v1", api_key="z",
models=[],
),
# Empty name — should NOT appear.
CustomProvider(
name="", base_url="https://example.com/v1", api_key="z",
models=[{"value": "x", "label": "x"}],
@@ -1252,7 +1137,6 @@ def test_list_models_includes_complete_custom_providers_excludes_incomplete():
assert len(groups["Ollama Cloud"]) == 1
assert groups["Ollama Cloud"][0]["value"] == "custom/ollama-cloud/gpt-oss:120b"
assert groups["Ollama Cloud"][0]["billing_kind"] == "api_key"
# None of the incomplete entries' names create a group.
assert "Broken" not in groups
assert "Empty" not in groups
@@ -1319,7 +1203,7 @@ def test_custom_provider_context_window_falls_back_to_default():
def test_custom_provider_resolve_aux_model_unaffected():
"""resolve_aux_model is the one-shot LLM call path. Custom providers
are NOT in its decision tree — Haiku/9Router/OR fallbacks should still
are NOT in its decision tree; Haiku/9Router/OR fallbacks should still
fire. Custom providers are deliberately not used for aux because we
don't know if they support tool calling well enough."""
import asyncio
@@ -1329,7 +1213,6 @@ def test_custom_provider_resolve_aux_model_unaffected():
anthropic_api_key="sk-ant-test",
custom_providers=[CustomProvider(name="Foo", base_url="https://x/v1", api_key="k")],
)
# Should pick Anthropic Haiku, not anything custom.
rid, base = asyncio.run(resolve_aux_model(s, preferred_tier="haiku"))
assert "haiku" in rid.lower()
assert not rid.startswith("cp-")
@@ -1347,20 +1230,15 @@ def test_custom_provider_with_very_long_name_still_works():
assert entry["model_id"] == f"cp-{slug}/some-model"
# ===========================================================================
# 9Router sync stress tests — async, mocked HTTP layer
# ===========================================================================
def _make_mock_9router(initial_nodes=None, initial_conns=None, fail_endpoints=None):
"""Build a mock httpx.AsyncClient that simulates 9Router's HTTP API.
Tracks state across requests so we can assert idempotency.
Returns (mock_client_class, state_dict) — state_dict is mutated by calls."""
Returns (mock_client_class, state_dict); state_dict is mutated by calls."""
from unittest.mock import AsyncMock, MagicMock
state = {
"nodes": list(initial_nodes or []),
"connections": list(initial_conns or []),
"calls": [], # list of (method, url, json) tuples
"calls": [],
"next_id": 1,
}
fail = fail_endpoints or set()
@@ -1404,7 +1282,6 @@ def _make_mock_9router(initial_nodes=None, initial_conns=None, fail_endpoints=No
async def _put(url, json=None, **kw):
state["calls"].append(("PUT", url, json))
# /api/provider-nodes/<id>
for n in state["nodes"]:
if url.endswith(f"/provider-nodes/{n['id']}"):
n.update(json or {})
@@ -1424,7 +1301,6 @@ def _make_mock_9router(initial_nodes=None, initial_conns=None, fail_endpoints=No
for n in list(state["nodes"]):
if url.endswith(f"/provider-nodes/{n['id']}"):
state["nodes"].remove(n)
# Cascade-delete connections.
state["connections"] = [
c for c in state["connections"] if c.get("provider") != n["id"]
]
@@ -1461,7 +1337,6 @@ def test_sync_custom_providers_silently_noop_when_9router_down():
from backend.apps.settings.models import CustomProvider
with upatch("backend.apps.nine_router.is_running", return_value=False):
# Should not raise even with malformed/empty input.
asyncio.run(sync_custom_providers([]))
asyncio.run(sync_custom_providers([
CustomProvider(name="X", base_url="https://x/v1", api_key="k"),
@@ -1483,7 +1358,6 @@ def test_sync_custom_providers_creates_node_and_connection_for_new_provider():
api_key="key1", models=[]),
]))
# Should have POSTed exactly one node and one connection.
posts = [c for c in state["calls"] if c[0] == "POST"]
assert len(posts) == 2, f"expected 2 POSTs, got {len(posts)}: {posts}"
node_post = next(c for c in posts if "/provider-nodes" in c[1])
@@ -1530,20 +1404,18 @@ def test_sync_custom_providers_updates_existing_node_in_place():
asyncio.run(sync_custom_providers([
CustomProvider(
name="Together AI",
base_url="https://api.together.xyz/v1", # unchanged URL
api_key="new-key", # changed key
base_url="https://api.together.xyz/v1",
api_key="new-key",
models=[],
),
]))
# Should PUT the node, PATCH the connection. NO new POSTs.
posts = [c for c in state["calls"] if c[0] == "POST"]
puts = [c for c in state["calls"] if c[0] == "PUT"]
patches = [c for c in state["calls"] if c[0] == "PATCH"]
assert posts == [], f"expected no new nodes/conns, got {posts}"
assert len(puts) >= 1, f"expected node PUT, got {puts}"
assert len(patches) >= 1, f"expected conn PATCH, got {patches}"
# And the apiKey should be the new one in the patched payload.
assert patches[0][2]["apiKey"] == "new-key"
@@ -1562,10 +1434,9 @@ def test_sync_custom_providers_deletes_orphaned_managed_nodes():
"prefix": "cp-oldprovider",
"type": "openai-compatible",
},
# An UNMANAGED node — should never be deleted.
{
"id": "node-user-created",
"name": "Manual Setup", # no suffix
"name": "Manual Setup",
"prefix": "manual",
"type": "openai-compatible",
},
@@ -1574,7 +1445,7 @@ def test_sync_custom_providers_deletes_orphaned_managed_nodes():
with upatch("backend.apps.nine_router.is_running", return_value=True), \
upatch("backend.apps.nine_router.httpx.AsyncClient", MockClient), \
upatch("backend.apps.nine_router.get_providers", new=lambda: _async_return([])):
asyncio.run(sync_custom_providers([])) # empty list → delete all managed
asyncio.run(sync_custom_providers([]))
deletes = [c for c in state["calls"] if c[0] == "DELETE"]
deleted_urls = [c[1] for c in deletes]
@@ -1608,7 +1479,7 @@ def test_sync_custom_providers_skips_incomplete_entries():
def test_sync_custom_providers_handles_node_post_failure_without_crashing():
"""If 9Router rejects the node POST (e.g. duplicate prefix), don't
crash the whole sync — log and move on to the next provider."""
crash the whole sync; log and move on to the next provider."""
import asyncio
from unittest.mock import patch as upatch
from backend.apps.nine_router import sync_custom_providers
@@ -1618,7 +1489,6 @@ def test_sync_custom_providers_handles_node_post_failure_without_crashing():
with upatch("backend.apps.nine_router.is_running", return_value=True), \
upatch("backend.apps.nine_router.httpx.AsyncClient", MockClient), \
upatch("backend.apps.nine_router.get_providers", new=lambda: _async_return([])):
# Should NOT raise.
asyncio.run(sync_custom_providers([
CustomProvider(name="A", base_url="https://a/v1", api_key="k1"),
CustomProvider(name="B", base_url="https://b/v1", api_key="k2"),
@@ -1643,7 +1513,6 @@ def test_sync_custom_providers_three_distinct_providers_create_three_nodes():
CustomProvider(name="Groq", base_url="https://api.groq.com/openai/v1", api_key="k3"),
]))
# Should have POSTed 3 nodes + 3 connections = 6 POSTs.
posts = [c for c in state["calls"] if c[0] == "POST"]
assert len(posts) == 6, f"expected 6 POSTs (3 nodes + 3 conns), got {len(posts)}"
@@ -1661,15 +1530,11 @@ def _async_return(value):
return _f()
# ===========================================================================
# Group S — calculate_cost regression tests
# ===========================================================================
def test_calculate_cost_anthropic_sonnet():
"""Sonnet $3/M input + $15/M output."""
from backend.apps.agents.providers.registry import calculate_cost
# 1M input, 1M output → $18 expected (3 + 15)
cost = calculate_cost("Anthropic", "sonnet", 1_000_000, 1_000_000)
assert 17 <= cost <= 19
@@ -1686,9 +1551,6 @@ def test_calculate_cost_unknown_model_returns_zero():
assert cost == 0.0
# ===========================================================================
# Group T — Mode definitions
# ===========================================================================
def test_agent_mode_no_explicit_tools():
@@ -1719,9 +1581,6 @@ def test_view_builder_mode_has_default_folder():
assert vb.default_folder is not None
# ===========================================================================
# Group U — Stress: gate handles 100 sequential calls without state leak
# ===========================================================================
@pytest.mark.asyncio
@@ -1740,9 +1599,6 @@ async def test_gate_100_sequential_calls_no_leak():
f"iteration {i}: expected {set(active)}, got {set(result.keys())}"
# ===========================================================================
# Group V — Discord shim entrypoint sanity
# ===========================================================================
def test_discord_shim_main_callable():
@@ -1753,13 +1609,9 @@ def test_discord_shim_main_callable():
def test_discord_shim_package_importable():
import backend.apps.discord_mcp_shim
# Empty __init__ now; just confirm the package imports without error
assert backend.apps.discord_mcp_shim is not None
# ===========================================================================
# Group W — Tools/web.py (live MCP for DDG search)
# ===========================================================================
def test_web_tools_classes_inherit_basetool():
@@ -1783,9 +1635,6 @@ def test_web_fetch_tool_has_name_and_schema():
assert isinstance(tool.get_schema(), dict)
# ===========================================================================
# Group X — ToolGroupMeta + caching
# ===========================================================================
def test_tool_group_meta_round_trip():
@@ -1804,9 +1653,6 @@ def test_tool_group_meta_default_is_refined_false():
assert m.is_refined is False
# ===========================================================================
# Group Y — MessageBranch invariants
# ===========================================================================
def test_session_has_main_branch_by_default():
@@ -1826,19 +1672,16 @@ def test_branch_serialization():
assert s2.branches["alt"].parent_branch_id == "main"
# ===========================================================================
# Group Z — End-to-end: realistic session lifecycle
# ===========================================================================
@pytest.mark.asyncio
async def test_e2e_session_lifecycle_with_mcp_activation():
"""
Walk a session through the realistic flow:
1. Fresh session (active_mcps empty) — gate blocks all MCPs
2. MCPActivate('gmail') — set fresh_session, append to active_mcps
3. Continue turn — gate now passes gmail through
4. Persist & re-load — state survives
1. Fresh session (active_mcps empty); gate blocks all MCPs
2. MCPActivate('gmail'); set fresh_session, append to active_mcps
3. Continue turn; gate now passes gmail through
4. Persist & re-load; state survives
"""
from backend.apps.agents.agent_manager import AgentManager
from backend.apps.agents.models import AgentSession
@@ -1848,21 +1691,18 @@ async def test_e2e_session_lifecycle_with_mcp_activation():
mgr = AgentManager()
s = AgentSession(id="e2e", name="End-to-end", model="sonnet", mode="agent")
# Step 1: fresh, gate blocks everything
result = await mgr._build_mcp_servers(
allowed_tools=["mcp:Gmail", "mcp:Slack"],
active_mcps=s.active_mcps,
)
assert result == {}
# Step 2: simulate MCPActivate
s.active_mcps.append("gmail")
s.sdk_session_id = "claude-existing"
if s.sdk_session_id:
s.needs_fresh_session = True
s.pending_continuation = True
# Step 3: continuation turn — gate passes gmail
result = await mgr._build_mcp_servers(
allowed_tools=["mcp:Gmail", "mcp:Slack"],
active_mcps=s.active_mcps,
@@ -1870,7 +1710,6 @@ async def test_e2e_session_lifecycle_with_mcp_activation():
assert "gmail" in result
assert "slack" not in result
# Step 4: persist + reload
dumped = json.dumps(s.model_dump(mode="json"))
s2 = AgentSession.model_validate(json.loads(dumped))
assert s2.active_mcps == ["gmail"]
@@ -1919,7 +1758,7 @@ def test_session_agent_active_ms_round_trip():
def test_session_agent_active_ms_accumulates_via_dict_update():
"""Simulates two turns adding to the bucket — the production accumulator
"""Simulates two turns adding to the bucket; the production accumulator
pattern in agent_manager._on_result."""
from backend.apps.agents.models import AgentSession
s = AgentSession(name="t", model="sonnet", mode="agent")
@@ -1932,14 +1771,11 @@ def test_session_agent_active_ms_accumulates_via_dict_update():
def test_session_time_per_model_records_switch():
"""Simulates a model switch mid-session — each model accumulates its
"""Simulates a model switch mid-session; each model accumulates its
own bucket."""
from backend.apps.agents.models import AgentSession
s = AgentSession(name="t", model="haiku", mode="agent")
# Turn 1 on haiku
s.time_per_model[s.model] = int(s.time_per_model.get(s.model, 0)) + 1200
# User switches to sonnet
s.model = "sonnet"
# Turn 2 on sonnet
s.time_per_model[s.model] = int(s.time_per_model.get(s.model, 0)) + 8400
assert s.time_per_model == {"haiku": 1200, "sonnet": 8400}
backend/tests/test_workflows_semantics.py
+430
View File
@@ -0,0 +1,430 @@
"""Backend semantics tests for the scheduled-tasks fix.
Covers:
- DST-safe wall-clock math (spring forward + fall back) via zoneinfo
- End conditions (ends_at + max_runs) auto-disable the schedule
- Cost cap skips fires with a clear error
- Freeze-default on for new scheduled non-source-session creates
- Audit log captures field diffs
- /workflows/active surfaces in-process running runs
- Legacy timezone="local" coerced in memory at load
- Storage paused flag round-trips
- Month math no longer clamps to day 28
- Server-side escalation kicks tasks (and ack cancels them)
Run:
pip install -r backend/requirements.txt -r backend/requirements-dev.txt
cd backend && python -m pytest tests/test_workflows_semantics.py -v
"""
from __future__ import annotations
import asyncio
import json
import os
import shutil
import tempfile
from datetime import datetime, timedelta, timezone
from zoneinfo import ZoneInfo
import pytest
@pytest.fixture(autouse=True)
def isolated_data_dir(monkeypatch, tmp_path):
"""Point storage at a fresh tmpdir per test so we never touch a real
install's workflows data. Reloads in-process module state so each test
starts with empty caches."""
from backend.apps.workflows import storage as _storage
from backend.apps.workflows import escalation as _escalation
monkeypatch.setattr(_storage, "DATA_DIR", str(tmp_path / "workflows"))
monkeypatch.setattr(_storage, "RUNS_DIR", str(tmp_path / "workflows" / "runs"))
monkeypatch.setattr(_storage, "PAUSED_FILE", str(tmp_path / "workflows" / "paused.json"))
monkeypatch.setattr(_storage, "_workflow_cache", {})
monkeypatch.setattr(_storage, "_runs_cache", {})
monkeypatch.setattr(_storage, "_cache_loaded", False)
monkeypatch.setattr(_storage, "_paused", False)
# Reset escalation registry between tests.
_escalation._tasks.clear()
_escalation._state.clear()
# Also clear audit dir reference; audit.py reads DATA_DIR at import via
# module-level expression, so reach in and override the AUDIT_DIR too.
from backend.apps.workflows import audit as _audit
monkeypatch.setattr(_audit, "AUDIT_DIR", str(tmp_path / "workflows" / "audit"))
yield
def _make_wf(**overrides):
from backend.apps.workflows.models import Workflow, ScheduleConfig, WorkflowStep
base = dict(
title="t",
steps=[WorkflowStep(text="hi")],
schedule=ScheduleConfig(enabled=True, repeat_unit="day", repeat_every=1, hour=9, minute=0, timezone="America/Los_Angeles"),
)
base.update(overrides)
return Workflow(**base)
# --- DST tests ---------------------------------------------------------------
def test_dst_spring_forward_weekly():
"""A 2:30am LA weekly Sunday schedule lands on 3:30am LA on the spring-
forward Sunday (2025-03-09) because the wall clock skips 02:30."""
from backend.apps.workflows.scheduler import _next_fire_after
from backend.apps.workflows.models import ScheduleConfig
tz = ZoneInfo("America/Los_Angeles")
sched = ScheduleConfig(enabled=True, repeat_unit="week", repeat_every=1, on_days=[0], hour=2, minute=30, timezone="America/Los_Angeles")
# Saturday 2025-03-08 23:00 LA, asking "what's the next Sunday 2:30?"
ref_local = datetime(2025, 3, 8, 23, 0, tzinfo=tz)
nxt = _next_fire_after(sched, ref_local.astimezone(timezone.utc))
assert nxt is not None
nxt_local = nxt.astimezone(tz)
# 02:30 wall-clock on the spring-forward day doesn't exist; zoneinfo
# resolves it forward to 03:30. The point is the *date* lands on the
# 9th, not the 8th and not the 16th.
assert nxt_local.date() == datetime(2025, 3, 9).date()
assert nxt_local.hour in (2, 3)
def test_dst_fall_back_no_double_fire():
"""A 9am LA daily schedule should fire exactly once on the fall-back day
(2025-11-02) and the next fire is the 3rd, not the 2nd again."""
from backend.apps.workflows.scheduler import _next_fire_after
from backend.apps.workflows.models import ScheduleConfig
tz = ZoneInfo("America/Los_Angeles")
sched = ScheduleConfig(enabled=True, repeat_unit="day", repeat_every=1, hour=9, minute=0, timezone="America/Los_Angeles")
ref_local = datetime(2025, 11, 1, 23, 0, tzinfo=tz)
nxt = _next_fire_after(sched, ref_local.astimezone(timezone.utc))
assert nxt.astimezone(tz).date() == datetime(2025, 11, 2).date()
# After firing on the 2nd, the next fire should be the 3rd, not a
# second 2nd from the duplicated hour.
after = _next_fire_after(sched, nxt)
assert after.astimezone(tz).date() == datetime(2025, 11, 3).date()
# --- End condition tests -----------------------------------------------------
def test_max_runs_disables_schedule():
from backend.apps.workflows import storage, scheduler
wf = _make_wf()
wf.schedule.max_runs = 2
wf.schedule.runs_count = 2
wf.next_run_at = datetime.now(timezone.utc) - timedelta(minutes=1)
storage.save_workflow(wf)
asyncio.new_event_loop().run_until_complete(scheduler._tick())
after = storage.get_workflow(wf.id)
assert after.schedule.enabled is False
assert after.next_run_at is None
def test_ends_at_disables_schedule():
from backend.apps.workflows import storage, scheduler
wf = _make_wf()
wf.schedule.ends_at = datetime.now(timezone.utc) - timedelta(days=1)
wf.next_run_at = datetime.now(timezone.utc) - timedelta(minutes=1)
storage.save_workflow(wf)
asyncio.new_event_loop().run_until_complete(scheduler._tick())
after = storage.get_workflow(wf.id)
assert after.schedule.enabled is False
# --- Month-day-31 (formerly clamped to 28) -----------------------------------
def test_month_repeat_no_longer_clamps_to_28():
"""An every-month schedule starting on March 31 should next fire on
April 30 (last day of April), then May 31, then June 30."""
from backend.apps.workflows.scheduler import _next_fire_after
from backend.apps.workflows.models import ScheduleConfig
tz = ZoneInfo("America/Los_Angeles")
sched = ScheduleConfig(enabled=True, repeat_unit="month", repeat_every=1, hour=9, minute=0, timezone="America/Los_Angeles")
ref_local = datetime(2025, 3, 31, 10, 0, tzinfo=tz) # past 9am on the 31st
nxt = _next_fire_after(sched, ref_local.astimezone(timezone.utc))
assert nxt.astimezone(tz).date() == datetime(2025, 4, 30).date()
# --- Cost cap ----------------------------------------------------------------
def test_cost_cap_skips_with_clear_error(monkeypatch):
from backend.apps.workflows import storage, executor
from backend.apps.workflows.models import WorkflowRun
wf = _make_wf()
wf.cost_cap_usd_monthly = 1.0
storage.save_workflow(wf)
storage.record_run(WorkflowRun(workflow_id=wf.id, status="success", cost_usd=0.6, started_at=datetime.now(timezone.utc), finished_at=datetime.now(timezone.utc)))
storage.record_run(WorkflowRun(workflow_id=wf.id, status="success", cost_usd=0.6, started_at=datetime.now(timezone.utc), finished_at=datetime.now(timezone.utc)))
async def fake_launch(*a, **k):
raise AssertionError("agent_manager should not be reached when cost-capped")
# Patch agent_manager.launch_agent so we'd fail loudly if the cap
# didn't short-circuit before launch.
from backend.apps.agents import agent_manager
monkeypatch.setattr(agent_manager.agent_manager, "launch_agent", fake_launch)
run = asyncio.new_event_loop().run_until_complete(executor.execute(wf, triggered_by="schedule"))
assert run.status == "skipped"
assert "Monthly cost cap reached" in (run.error or "")
# --- Freeze-default for scheduled non-source-session creates ----------------
def test_freeze_defaults_on_for_scheduled_create():
"""POST /workflows/create with schedule.enabled=true and no source
session should flip actions.freeze=True to keep blast radius small."""
from backend.apps.workflows.workflows import create_workflow
from backend.apps.workflows.models import WorkflowCreate, ScheduleConfig, ActionsConfig
body = WorkflowCreate(
title="scheduled",
schedule=ScheduleConfig(enabled=True, repeat_unit="day", repeat_every=1, hour=9, minute=0),
actions=ActionsConfig(freeze=False, configured_sets=[]),
)
result = asyncio.new_event_loop().run_until_complete(create_workflow(body))
assert result["actions"]["freeze"] is True
def test_freeze_not_forced_when_source_session_present():
"""Source-session creates inherit the chat's choices; we don't override."""
from backend.apps.workflows.workflows import create_workflow
from backend.apps.workflows.models import WorkflowCreate, ScheduleConfig, ActionsConfig
body = WorkflowCreate(
title="from chat",
source_session_id="sess-1",
schedule=ScheduleConfig(enabled=True, repeat_unit="day", repeat_every=1, hour=9, minute=0),
actions=ActionsConfig(freeze=False, configured_sets=[]),
)
result = asyncio.new_event_loop().run_until_complete(create_workflow(body))
assert result["actions"]["freeze"] is False
# --- Audit log ---------------------------------------------------------------
def test_audit_log_records_title_change():
from backend.apps.workflows import audit
audit.log_change("wf-1", "user", {"title": "old"}, {"title": "new"})
entries = audit.read_tail("wf-1", limit=10)
assert len(entries) == 1
diff = entries[0]["diff"]
assert diff["title"]["before"] == "old"
assert diff["title"]["after"] == "new"
def test_audit_log_no_op_when_unchanged():
from backend.apps.workflows import audit
audit.log_change("wf-2", "user", {"title": "same"}, {"title": "same"})
assert audit.read_tail("wf-2") == []
# --- /workflows/active -------------------------------------------------------
def test_list_active_reflects_running_map():
from backend.apps.workflows import storage, executor, scheduler
wf = _make_wf(title="active-test")
storage.save_workflow(wf)
from backend.apps.workflows.models import WorkflowRun
run = WorkflowRun(workflow_id=wf.id, status="running")
storage.record_run(run)
executor._running[wf.id] = run.id
try:
active = scheduler.list_active()
assert len(active) == 1
assert active[0]["workflow_id"] == wf.id
assert active[0]["title"] == "active-test"
finally:
executor._running.pop(wf.id, None)
# --- Legacy tz coercion ------------------------------------------------------
def test_legacy_timezone_coerced_on_load(monkeypatch):
from backend.apps.workflows import storage
storage._ensure_dirs()
wf_id = "legacy-wf"
legacy_blob = {
"id": wf_id,
"title": "legacy",
"schedule": {
"enabled": False, "repeat_every": 1, "repeat_unit": "week",
"on_days": [], "hour": 9, "minute": 0, "timezone": "local",
"on_missed": "skip", "ends_at": None, "max_runs": None, "runs_count": 0,
},
}
with open(os.path.join(storage.DATA_DIR, f"{wf_id}.json"), "w") as f:
json.dump(legacy_blob, f)
monkeypatch.setenv("OPENSWARM_TIMEZONE", "America/Los_Angeles")
monkeypatch.setattr(storage, "_cache_loaded", False)
loaded = storage.get_workflow(wf_id)
assert loaded is not None
# In-memory should be the host zone, not "local".
assert loaded.schedule.timezone == "America/Los_Angeles"
# On-disk file should be unchanged (still "local") so we don't churn
# mtime on every restart.
with open(os.path.join(storage.DATA_DIR, f"{wf_id}.json")) as f:
on_disk = json.load(f)
assert on_disk["schedule"]["timezone"] == "local"
# --- Paused flag -------------------------------------------------------------
def test_paused_flag_persists_and_blocks_tick():
from backend.apps.workflows import storage, scheduler
wf = _make_wf()
wf.next_run_at = datetime.now(timezone.utc) - timedelta(minutes=1)
storage.save_workflow(wf)
storage.set_paused(True)
# Reload simulates a backend restart.
storage._cache_loaded = False
assert storage.get_paused() is True
# Tick must not advance next_run_at when paused.
before = storage.get_workflow(wf.id).next_run_at
asyncio.new_event_loop().run_until_complete(scheduler._tick())
after = storage.get_workflow(wf.id).next_run_at
assert before == after
# --- Escalation --------------------------------------------------------------
def test_escalation_schedules_and_ack_cancels():
from backend.apps.workflows import escalation
from backend.apps.workflows.models import Workflow, PermissionTier, WorkflowRun, ScheduleConfig
async def runner():
wf = Workflow(title="t", permissions=[
PermissionTier(kind="notify"),
PermissionTier(kind="text", after_minutes=60, phone="+15551234567"),
])
run = WorkflowRun(workflow_id=wf.id, status="success")
escalation.schedule(wf, run)
# State should be present immediately.
await asyncio.sleep(0.01)
assert escalation.status(run.id) is not None
# Ack cancels.
assert escalation.cancel(run.id) is True
await asyncio.sleep(0.01)
assert escalation.status(run.id) is None
asyncio.new_event_loop().run_until_complete(runner())
def test_executor_merge_does_not_clobber_concurrent_patch():
"""Executor's final save must NOT overwrite unrelated fields that
were PATCHed while the run was in flight. We simulate this by
capturing a wf, mutating storage's record directly (acting as the
PATCH that landed mid-run), then asking the executor's persist
helper to flush its run-side bookkeeping. The patched fields must
survive.
"""
from backend.apps.workflows import storage, executor
from datetime import datetime
wf = _make_wf(title="t-orig")
storage.save_workflow(wf)
# Simulate a user PATCH mid-run.
storage._workflow_cache[wf.id].title = "t-patched"
storage._workflow_cache[wf.id].description = "patched while running"
storage.save_workflow(storage._workflow_cache[wf.id])
# Executor uses the stale `wf` it captured before the patch. With
# the merge helper, the patched fields must remain.
executor._persist_run_fields(wf, {
"last_run_at": datetime.now(),
"last_run_status": "success",
})
after = storage.get_workflow(wf.id)
assert after.title == "t-patched", "title clobbered by executor"
assert after.description == "patched while running", "description clobbered"
assert after.last_run_status == "success"
def test_executor_delete_during_run_does_not_resurrect():
"""If the workflow was deleted mid-run, executor's persist must
silently no-op so the deleted record isn't re-written."""
from backend.apps.workflows import storage, executor
from datetime import datetime
wf = _make_wf(title="doomed")
storage.save_workflow(wf)
storage.delete_workflow(wf.id)
executor._persist_run_fields(wf, {
"last_run_at": datetime.now(),
"last_run_status": "success",
}, schedule_runs_count_delta=1)
assert storage.get_workflow(wf.id) is None
def test_patch_if_match_rejects_stale_write():
"""A PATCH with a stale If-Match must return 409. Without If-Match,
the request still succeeds (legacy clients keep working until they
roll out the header)."""
from backend.apps.workflows.workflows import update_workflow
from backend.apps.workflows.models import WorkflowUpdate
from backend.apps.workflows import storage
from fastapi import HTTPException
wf = _make_wf(title="optimistic-test")
storage.save_workflow(wf)
stale = "1999-01-01T00:00:00"
async def runner():
# Stale If-Match → 409.
try:
await update_workflow(wf.id, WorkflowUpdate(title="x"), if_match=stale)
return "no exception"
except HTTPException as he:
return he.status_code
code = asyncio.new_event_loop().run_until_complete(runner())
assert code == 409, f"stale If-Match should 409, got {code}"
# Fresh If-Match → 200.
fresh = storage.get_workflow(wf.id)
fresh_stamp = fresh.updated_at.isoformat()
async def runner_ok():
return await update_workflow(wf.id, WorkflowUpdate(title="y"), if_match=fresh_stamp)
result = asyncio.new_event_loop().run_until_complete(runner_ok())
assert result["title"] == "y"
# Missing If-Match → legacy path still works.
async def runner_legacy():
return await update_workflow(wf.id, WorkflowUpdate(title="z"), if_match=None)
result = asyncio.new_event_loop().run_until_complete(runner_legacy())
assert result["title"] == "z"
def test_killed_by_restart_message_is_friendly():
"""stuck-run reaper writes a user-facing string, not internal jargon."""
from backend.apps.workflows import storage, scheduler
from backend.apps.workflows.models import WorkflowRun
wf = _make_wf()
storage.save_workflow(wf)
storage.record_run(WorkflowRun(workflow_id=wf.id, status="running"))
scheduler._mark_stuck_runs_failed()
runs = storage.list_runs(wf.id, limit=10)
assert any(r.status == "failure" and "OpenSwarm closed" in (r.error or "") for r in runs)
assert not any("Killed by restart" in (r.error or "") for r in runs)
def test_run_endpoint_surfaces_skipped_status():
"""POST /workflows/{id}/run returns the skipped status + error when
a cost-cap or in-flight collision short-circuits the run."""
from backend.apps.workflows.workflows import run_workflow_now
from backend.apps.workflows import storage
from backend.apps.workflows.models import WorkflowRun
from datetime import datetime, timezone
wf = _make_wf(title="cap-immediate")
wf.cost_cap_usd_monthly = 0.01
storage.save_workflow(wf)
# Burn the cap with a single $5 historical run.
storage.record_run(WorkflowRun(workflow_id=wf.id, status="success", cost_usd=5.0,
started_at=datetime.now(timezone.utc),
finished_at=datetime.now(timezone.utc)))
async def runner():
return await run_workflow_now(wf.id)
res = asyncio.new_event_loop().run_until_complete(runner())
assert res.get("status") == "skipped"
assert "cost cap" in (res.get("error") or "").lower()
def test_escalation_noop_for_single_tier():
from backend.apps.workflows import escalation
from backend.apps.workflows.models import Workflow, PermissionTier, WorkflowRun
wf = Workflow(title="t", permissions=[PermissionTier(kind="notify")])
run = WorkflowRun(workflow_id=wf.id, status="success")
escalation.schedule(wf, run)
assert escalation.status(run.id) is None
debugger/debug.py
-5
View File
@@ -14,11 +14,8 @@ def debug(*args, mode:str='debug', override_max_chars:bool=False):
calling_file_name = os.path.basename(code.co_filename)
if calling_function_name == "<module>":
calling_function_name = calling_file_name
# Retrieve the file path of the calling function
file_path = os.path.abspath(code.co_filename)
# print(f"FILE PATH: {file_path}")
t_color, t_is_on, t_emoji = Debugleton().find_file_info(file_path)
# print(f"DEBUGGING: {t_color}, {t_is_on}")
max_chars = 3000
with open(code.co_filename, 'r', encoding='utf-8') as f:
@@ -44,7 +41,6 @@ def debug(*args, mode:str='debug', override_max_chars:bool=False):
arg_value = arg_value[:int(max_chars/2)] + "...\n..." + arg_value[arg_len-int(max_chars/2):]
function_print_str = calling_function_name if 'self' not in frame.f_locals else f'{frame.f_locals["self"].__class__.__name__}.{calling_function_name}'
# color = COLORS.get(function_print_str, white)
color = hex_to_rgb(t_color)
if arg_is_text:
print_str = f"{t_emoji}{rgb_to_ansi(color)}{indent_str}[{function_print_str}] : {bold_and_italicize_text(arg_value)}\033[0m"
@@ -52,6 +48,5 @@ def debug(*args, mode:str='debug', override_max_chars:bool=False):
print_str = f"{t_emoji}{rgb_to_ansi(color)}{indent_str}[{function_print_str}] : {arg_name} = {arg_value}\033[0m"
if t_is_on: log_config.debug_custom(print_str, mode)
# Assign the function to the module's __call__ attribute
import sys
sys.modules[__name__] = debug
debugger/debugger_backend/DebugFile.py
+2 -6
View File
@@ -13,9 +13,7 @@ class DebugFile(File):
self.directory = directory # Reference to parent directory
def to_dict(self):
"""
Converts the DebugFile object to a dictionary format.
"""
"""Convert the DebugFile to a dict."""
return {
"name": os.path.basename(self.filename),
"color": self.color,
@@ -26,9 +24,7 @@ class DebugFile(File):
@classmethod
def from_dict(cls, file_dict, directory):
"""
Creates a DebugFile object from a dictionary loaded from JSON.
"""
"""Build a DebugFile from a JSON-loaded dict."""
filename = os.path.join(directory.path, file_dict["name"])
return cls(
filename=filename,
debugger/debugger_backend/Debugleton.py
+1 -14
View File
@@ -16,7 +16,6 @@ class Debugleton:
sync_lock: threading.Lock
def __new__(cls, *args, **kwargs):
# Double-checked locking for thread-safe singleton creation
if cls._instance is None:
with cls._lock:
if cls._instance is None:
@@ -32,27 +31,18 @@ class Debugleton:
print("\033[38;5;120m|\t...Project Scanned\t|\033[0m")
print("\033[38;5;120m|\tDEBUGLETON INIT DONE\t|\033[0m")
print("\033[38;5;120m---------------------------------\n\033[0m")
# else: print("DEBUGLETON Already initialized INNER")
# else: print("DEBUGLETON Already initialized OUTER")
return cls._instance
def sync_to_saved(self, is_first_sync=False):
# print(f"[sync_to_saved]: START")
if not is_first_sync: self.sync_lock.acquire()
# print(f"[sync_to_saved]: Acquired sync lock")
self.dir = update_debug_toggles(save_to_file=False)
# print(f"Synced to saved dir: {self.dir}")
self.abspaths, self.instances = self.dir.get_ordered_abspaths_and_instances()
# print(f"Synced to abspaths: {self.abspaths}")
with open(NEEDS_RESYNC_FILE, 'w') as f:
f.write('0')
if not is_first_sync: self.sync_lock.release()
# print(f"[sync_to_saved]: Released sync lock")
# print(f"[sync_to_saved]: END")
def needs_resync(self):
# print(f"[needs_resync]: START")
num_tries = 0
while self.is_syncing():
print(f"Waiting for Debugleton to sync... ({num_tries})")
@@ -67,8 +57,6 @@ class Debugleton:
""")
with open(NEEDS_RESYNC_FILE, 'r') as f:
does_need_resync = True if f.read().strip() == '1' else False
# if does_need_resync: print("Resyncing Debugleton...")
# print(f"[needs_resync]: END")
return does_need_resync
def is_syncing(self):
@@ -76,7 +64,6 @@ class Debugleton:
def find_file_info(self, filepath: str):
filepath = filepath.lower()
# print(f"Finding file info for {filepath}")
if self.needs_resync():
self.sync_to_saved()
try:
debugger/debugger_backend/Directory.py
+12 -52
View File
@@ -7,11 +7,10 @@ from debugger_backend.DEFAULTS import DEFAULT_COLOR, DEFAULT_TOGGLED, DEFAULT_SE
from debugger_backend.path_mngr import get_abspath, get_root_rel_path
class Directory:
def __init__(self, path, color=DEFAULT_COLOR, is_toggled=DEFAULT_TOGGLED,
def __init__(self, path, color=DEFAULT_COLOR, is_toggled=DEFAULT_TOGGLED,
set_manually=DEFAULT_SET_MANUALLY, emoji=DEFAULT_EMOJI):
self.path = path
# print(f"Directory init: {self.path}")
self.children = [] # Can contain DebugFile or other Directory objects
self.children = []
self.color = color
self.is_toggled = is_toggled
self.set_manually = set_manually
@@ -24,39 +23,26 @@ class Directory:
return get_abspath(self.path)
def add_child(self, child):
"""
Adds a child to the directory (either a DebugFile or another Directory).
"""
"""Append a child DebugFile/Directory."""
self.children.append(child)
def get_ordered_abspaths_and_instances(self):
# print("[get_ordered_abspaths]: START")
curr_file_path = os.path.abspath(__file__)
root_dir = os.path.dirname(os.path.dirname(os.path.dirname(curr_file_path)))
# print(f"[get_ordered_abspaths]: Curr path: {curr_file_path}")
# print(f"[get_ordered_abspaths]: Dir path: {root_dir}")
def construct_ordered_abspaths(dir: Directory, ordered_abspaths: list):
dir_path = dir.path
full_path = os.path.join(root_dir, dir_path)
ordered_abspaths.append({"abspath": full_path, "instance": dir})
# print(f"\t[construct_ordered_abspaths]: Full path: {full_path}")
for child in dir.children:
child_abspath = os.path.join(root_dir, child.path).lower()
if os.path.isdir(child_abspath):
construct_ordered_abspaths(child, ordered_abspaths)
elif os.path.isfile(child_abspath):
# print(f"\t[construct_ordered_abspaths]: Child is file: {child_abspath}")
ordered_abspaths.append({"abspath": child_abspath, "instance": child})
else:
print(f"\033[38;5;120mEntry is non existent: {child_abspath}\033[0m")
# print(f"\t[construct_ordered_abspaths]: Finished for dir: {full_path}")
# print(f"\t[construct_ordered_abspaths]: RETURNING FROM DIR: {full_path}")
return ordered_abspaths
ordered_abspaths_and_instances = construct_ordered_abspaths(self, [])
# print("[get_ordered_abspaths]: Finished getting ordered abspaths and instances")
# for abspath_and_instance in ordered_abspaths_and_instances:
# abspath = abspath_and_instance["abspath"]
# print(f"\t[get_ordered_abspaths]: Abspath: {abspath}")
ordered_abspaths = [abspath_and_instance["abspath"] for abspath_and_instance in ordered_abspaths_and_instances]
ordered_instances = [abspath_and_instance["instance"] for abspath_and_instance in ordered_abspaths_and_instances]
return ordered_abspaths, ordered_instances
@@ -65,17 +51,12 @@ class Directory:
def build_structure(self):
print("[build_structure]: START")
root_dir = self.get_abspath()
# print(f"[build_structure]: Root dir: {root_dir}")
excluded_dirs = [".venv", "debugger", "node_modules", ".git", "__pycache__"]
project_structure = []
def construct_project_structure(dir_path: str, parent_dir: Directory):
# print(f"[build_structure]: Scanning dir: {dir_path}")
with os.scandir(dir_path) as it:
for entry in it:
# print(f"[build_structure]: Entry: {entry.path}")
if any(excluded_dir in entry.path for excluded_dir in excluded_dirs):
# print(f"[build_structure]: Excluding {entry.path}")
continue
root_rel_path = get_root_rel_path(entry.path)
if entry.is_dir():
@@ -88,16 +69,12 @@ class Directory:
parent_dir.add_child(debug_file)
else:
continue
construct_project_structure(root_dir, self)
# [print(f"[build_structure]: {file}") for file in project_structure]
# print(f"[build_structure]: END")
construct_project_structure(root_dir, self)
return
def to_dict(self):
"""
Converts the Directory object to a dictionary format, recursively.
"""
"""Recursively convert the Directory to a dict."""
return {
"name": os.path.basename(self.path),
"color": self.color,
@@ -108,22 +85,14 @@ class Directory:
}
def prune_empty(self):
# Recursively prune empty directories
# Base case) if the current directory has no children, return
# Recursive case) for each of the directories in the current directory, call prune_empty
# then remove the directory from the children of the current directory if it has no children
for child in self.children[:]:
if isinstance(child, Directory):
# Recursively prune empty subdirectories
child.prune_empty()
# If the subdirectory is empty after pruning, remove it
if len(child.children) == 0:
self.children.remove(child)
def propagate_toggled_state(self):
"""
Propagates the toggled state down the hierarchy.
"""
"""Propagate the toggled state down the hierarchy."""
for child in self.children:
if isinstance(child, DebugFile) and not child.set_manually:
child.is_toggled = self.is_toggled
@@ -132,9 +101,7 @@ class Directory:
child.propagate_toggled_state()
def propagate_color(self, parent_color=DEFAULT_COLOR):
"""
Propagates the color from parent to children.
"""
"""Propagate color from parent to children."""
if self.color == DEFAULT_COLOR:
self.color = lighten_color(parent_color)
for child in self.children:
@@ -144,9 +111,7 @@ class Directory:
child.propagate_color(self.color)
def load_from_json(self, json_data):
"""
Loads a directory structure from a JSON file into this Directory instance.
"""
"""Load a directory structure from JSON into this Directory."""
for item in json_data:
if 'children' in item:
subdir = Directory(
@@ -160,7 +125,6 @@ class Directory:
subdir.load_from_json(item['children'])
self.add_child(subdir)
else:
# debug_file = DebugFile.from_dict(item, self)
debug_file = DebugFile(
filename=item['name'],
path=os.path.join(self.path, item['name']),
@@ -173,9 +137,7 @@ class Directory:
self.add_child(debug_file)
def reset_colors(self):
"""
Resets the color of all DebugFile and Directory objects in this directory structure to the default color.
"""
"""Reset every nested color to the default."""
self.color = DEFAULT_COLOR
for child in self.children:
if isinstance(child, DebugFile):
@@ -185,9 +147,7 @@ class Directory:
def lighten_color(color, amount=0.1):
"""
Lightens the given color by the specified amount.
"""
"""Lighten the given color by amount."""
try:
color = color.lstrip('#')
r, g, b = int(color[:2], 16), int(color[2:4], 16), int(color[4:6], 16)
debugger/debugger_backend/File.py
+1 -4
View File
@@ -10,9 +10,7 @@ class File:
return get_abspath(self.path)
def calls_debug_function(self):
"""
Checks if the file calls the debug function.
"""
"""True if the file contains a debug() call."""
full_path = self.get_abspath()
if not full_path.endswith('.py') or full_path.endswith('.pyc'):
@@ -25,5 +23,4 @@ class File:
except (UnicodeDecodeError, FileNotFoundError) as e:
print(f"Error reading file {full_path}")
result = False
# print(f"??calls_debug_function?? {result}")
return result
debugger/debugger_backend/color_adjuster.py
+3 -6
View File
@@ -1,9 +1,9 @@
import colorsys
def adjust_brightness(color, brightness_factor):
hls = colorsys.rgb_to_hls(*[x/255.0 for x in color]) # Convert RGB to HLS
hls = (hls[0], max(0, min(1, hls[1] + brightness_factor)), hls[2]) # Adjust lightness
rgb = [int(x*255.0) for x in colorsys.hls_to_rgb(*hls)] # Convert back to RGB
hls = colorsys.rgb_to_hls(*[x/255.0 for x in color])
hls = (hls[0], max(0, min(1, hls[1] + brightness_factor)), hls[2])
rgb = [int(x*255.0) for x in colorsys.hls_to_rgb(*hls)]
return rgb
@@ -14,8 +14,5 @@ def bold_and_italicize_text(text):
return f"\033[1m\033[3m{text}\033[0m"
def hex_to_rgb(hex_code):
# Remove the '#' symbol if it exists
hex_code = hex_code.lstrip('#')
# Convert the hex code to RGB
return tuple(int(hex_code[i:i+2], 16) for i in (0, 2, 4))
debugger/debugger_backend/debug_arg_parser.py
-2
View File
@@ -2,7 +2,6 @@
def is_fstring(arg_name):
if not isinstance(arg_name, str):
return False
# print(f"arg_name: {arg_name}")
fstring_start_values = ["f'", "f\""]
num_start_matches = sum(arg_name.startswith(start_value) for start_value in fstring_start_values)
conditions = [num_start_matches == 1]
@@ -12,7 +11,6 @@ def is_text(arg_value, arg_name):
arg_is_text = isinstance(arg_value, str) and len(arg_name) > 2 and arg_name[1:len(arg_name)-1] == arg_value and not arg_name.endswith(")")
if not arg_is_text:
arg_is_text = is_fstring(arg_name)
# print(f"is_text: {arg_is_text}")
return arg_is_text
def is_error(arg_value, arg_name):
debugger/debugger_backend/debugger_server.py
-5
View File
@@ -12,10 +12,8 @@ CORS(app)
def api_get_structure():
print("GET /get_structure")
scanned_dir=update_debug_toggles(save_to_file=True)
# print("\n\nPS scanned_dir: ", scanned_dir)
output = dir_to_output_format(scanned_dir)
output = json.dumps(output, ensure_ascii=False, indent=4)
# print("output: ", output)
return Response(output, mimetype='application/json')
@app.route('/push_structure', methods=['POST'])
@@ -23,7 +21,6 @@ def api_push_structure():
print("POST /push_structure")
data = request.get_json()
data = data['projectStructure']
# print(data)
with open(DEBUG_TOGGLE_FILE, 'w', encoding='utf-8') as file:
json.dump(data, file, indent=4)
with open(NEEDS_RESYNC_FILE, 'w') as f:
@@ -35,10 +32,8 @@ def api_reset_color():
print("POST /reset_color")
scanned_dir=update_debug_toggles(save_to_file=False)
scanned_dir.reset_colors()
# print("RS: scanned_dir: ", scanned_dir)
output = dir_to_output_format(scanned_dir)
output = json.dumps(output, ensure_ascii=False, indent=4)
# print("RS: output: ", output)
return Response(output, mimetype='application/json')
debugger/debugger_backend/log_config.py
+1 -3
View File
@@ -19,12 +19,11 @@ class LogConfig:
for name, level in self.MODES.items():
logging.addLevelName(level, name.upper())
self.logger = logging.getLogger('custom_logger')
self.logger.propagate = False # Prevent log propagation
self.logger.propagate = False
handler = logging.StreamHandler()
formatter = logging.Formatter('%(message)s')
handler.setFormatter(formatter)
# Remove existing handlers to prevent duplicate logging
if self.logger.hasHandlers():
self.logger.handlers.clear()
@@ -39,7 +38,6 @@ class LogConfig:
def set_debug_mode(self, mode):
current_mode = get_log_mode()
# print(f"Setting debug mode from {current_mode} -> to {mode}")
if mode not in self.MODES: raise ValueError(f"Invalid mode: {mode}")
set_log_mode(mode)
self.logger.setLevel(self.MODES[mode])
debugger/debugger_backend/log_mode.py
+1 -2
View File
@@ -1,6 +1,5 @@
import os
# LOG_MODE_FILE = 'debugger/log_mode.txt'
LOG_MODE_FILE = os.path.join(os.path.dirname(__file__), 'log_mode.txt')
def set_log_mode(mode):
with open(LOG_MODE_FILE, 'w') as f:
@@ -10,4 +9,4 @@ def get_log_mode():
if os.path.exists(LOG_MODE_FILE):
with open(LOG_MODE_FILE, 'r') as f:
return f.read().strip()
return 'all' # Default to 'all' if the file doesn't exist
return 'all'
debugger/debugger_backend/project_scanner.py
+11 -50
View File
@@ -8,16 +8,9 @@ from debugger_backend.DebugFile import DebugFile
from collections import OrderedDict
def merge_directories(json_dir: Directory, scanned_dir: Directory):
"""
Merges two Directory instances: one loaded from JSON (json_dir) and one built from scanning (scanned_dir).
The values from json_dir take precedence where attributes overlap.
It matches based on full directory and file structure, not just file names.
"""
# print(f"Merging JSON_DIR: {json_dir.path}\n with SCAN_DIR: {scanned_dir.path}")
"""Merge json_dir into scanned_dir; json values win on overlap, matched by full path."""
json_abspaths, json_instances = json_dir.get_ordered_abspaths_and_instances()
# print(f"json_abspaths: {json_abspaths}")
scanned_abspaths, scanned_instances = scanned_dir.get_ordered_abspaths_and_instances()
# print(f"scanned_abspaths: {scanned_abspaths}")
def find_matching_in_structure(scanned_child: Union[DebugFile, Directory], json_dir: Directory):
assert json_dir in json_instances, f"JSON_DIR: {json_dir.path} not in json_instances"
@@ -28,32 +21,26 @@ def merge_directories(json_dir: Directory, scanned_dir: Directory):
try:
json_id = json_abspaths.index(scanned_abspath)
json_instance = json_instances[json_id]
# print(f"Match found: {scanned_child.path} == {json_instance.path}")
except ValueError:
# print(f"SCANNED_ABSPATH: {scanned_abspath} not in JSON_ABSPATHS")
pass
return json_instance
def construct_merged_dir(json_dir: Directory, scanned_dir: Directory):
for scanned_child in scanned_dir.children:
# Use the new recursive function to find the corresponding child in the JSON directory structure
matching_json_child = find_matching_in_structure(scanned_child, json_dir)
if isinstance(scanned_child, DebugFile) and matching_json_child:
# Merge attributes from the JSON-loaded structure
scanned_child.color = matching_json_child.color
scanned_child.is_toggled = matching_json_child.is_toggled
scanned_child.set_manually = matching_json_child.set_manually
scanned_child.emoji = matching_json_child.emoji
elif isinstance(scanned_child, Directory) and matching_json_child:
# Merge directory attributes
scanned_child.color = matching_json_child.color
scanned_child.is_toggled = matching_json_child.is_toggled
scanned_child.set_manually = matching_json_child.set_manually
scanned_child.emoji = matching_json_child.emoji
# Recursively merge the subdirectories
construct_merged_dir(matching_json_child, scanned_child)
else:
scanned_child.color = DEFAULT_COLOR
@@ -65,7 +52,6 @@ def merge_directories(json_dir: Directory, scanned_dir: Directory):
def update_debug_toggles(save_to_file=True) -> Directory:
# print(f"[update_debug_toggles]: START")
json_loaded_dir = None
if os.path.exists(TOGGLE_FILE):
with open(TOGGLE_FILE, 'r', encoding='utf-8') as file:
@@ -78,65 +64,40 @@ def update_debug_toggles(save_to_file=True) -> Directory:
set_manually=json_data[0].get('set_manually', DEFAULT_SET_MANUALLY),
emoji=json_data[0].get('emoji', DEFAULT_EMOJI)
)
# print(f"Root: {json_loaded_dir}")
# print("Json Children 1:")
# [print(child.path) for child in json_loaded_dir.children]
json_loaded_dir.load_from_json(json_data[0]['children']) # Assuming the root is in json_data[0]
# print("Json Children 2:")
# [print(child.path) for child in json_loaded_dir.children]
json_loaded_dir.load_from_json(json_data[0]['children'])
except json.JSONDecodeError:
ValueError("Error: JSON file could not be decoded.")
else:
print("No JSON file found")
# 1. Create a directory structure from the filesystem scan
# print("Scanning directory...")
scanned_dir = Directory(path="",
color=json_loaded_dir.color if json_loaded_dir else DEFAULT_COLOR,
is_toggled=json_loaded_dir.is_toggled if json_loaded_dir else DEFAULT_TOGGLED,
scanned_dir = Directory(path="",
color=json_loaded_dir.color if json_loaded_dir else DEFAULT_COLOR,
is_toggled=json_loaded_dir.is_toggled if json_loaded_dir else DEFAULT_TOGGLED,
set_manually=json_loaded_dir.set_manually if json_loaded_dir else DEFAULT_SET_MANUALLY,
emoji=json_loaded_dir.emoji if json_loaded_dir else DEFAULT_EMOJI
)
# print(f"\n\nNum Children 1: {len(scanned_dir.children)}")
# [print(child.path) for child in scanned_dir.children]
scanned_dir.build_structure()
# print(f"\n\nNum Children 2: {len(scanned_dir.children)}")
# [print(child.path) for child in scanned_dir.children]
scanned_dir.prune_empty()
# print(f"\n\nNum Children 3: {len(scanned_dir.children)}")
# [print(child.path) for child in scanned_dir.children]
# print("1.1 Merged Dir First Child: ", scanned_dir.children[0])
# 4. Propagate the toggled state and color through the merged structure
scanned_dir.propagate_toggled_state()
# print(f"\n\nNum Children 4: {len(scanned_dir.children)}")
# [print(child.path) for child in scanned_dir.children]
# 3. Merge the two directory structures
if json_loaded_dir:
merge_directories(json_loaded_dir, scanned_dir)
# print(f"\n\nNum Children 5: {len(scanned_dir.children)}")
scanned_dir.propagate_color()
output = dir_to_output_format(scanned_dir)
# print(f"\n\nNum Children 6: {len(scanned_dir.children)}")
# 5. Write the updated structure back to the JSON file
if save_to_file:
with open(TOGGLE_FILE, 'w', encoding='utf-8') as file:
json.dump(output, file, ensure_ascii=False, indent=4)
# print(f"[update_debug_toggles]: END")
return scanned_dir
def dir_to_output_format(input_dir):
root_node = {
"name": "root",
"color": input_dir.color, # Use input_dir's color
"is_toggled": input_dir.is_toggled, # Use input_dir's toggled state
"set_manually": input_dir.set_manually, # Use input_dir's set_manually
"emoji": input_dir.emoji, # Use input_dir's emoji
"color": input_dir.color,
"is_toggled": input_dir.is_toggled,
"set_manually": input_dir.set_manually,
"emoji": input_dir.emoji,
"children": input_dir.to_dict()["children"]
}
return [ordered(root_node)]
debugger/setup.py
+1 -4
View File
@@ -1,9 +1,6 @@
from setuptools import setup, find_packages
# `py_modules` exposes BOTH `debug` (legacy import name used by OpenSwarm's
# own backend) and `swarm_debug` (the import name the webapp-template
# scaffold uses, matching the published-package convention `swarm-debug`).
# The `swarm_debug` module is a thin re-export of `debug` — see swarm_debug.py.
# Exposes both `debug` (legacy) and `swarm_debug` (webapp-template convention; thin re-export).
setup(
name="debug",
version="0.1",
debugger/swarm_debug.py
+2 -16
View File
@@ -1,21 +1,7 @@
"""Module alias — exposes the `debug()` function under the `swarm_debug`
name so code that does `from swarm_debug import debug` resolves to the
same OpenSwarm-bundled package that the legacy `import debug` path
already serves.
"""Re-exports debug() under swarm_debug; debug.py swaps sys.modules to the function so from-imports there don't work."""
`debug.py` ends with `sys.modules[__name__] = debug`, which replaces the
module object with the bare function. That trick lets OpenSwarm's own
code write `import debug; debug(x)` (the imported name binds to the
function directly), but it means `from debug import debug` doesn't work
(you can't attribute-walk a function). This shim captures the function
via `import debug` (which now binds to the function thanks to the
sys.modules swap) and re-exports it as a normal module attribute, so
the more conventional `from swarm_debug import debug` pattern works.
"""
import debug as _debug # noqa: F401
import debug as _debug # noqa: F401 — `_debug` is actually the function
# Re-export as a module attribute so `from swarm_debug import debug` resolves.
debug = _debug
__all__ = ["debug"]
electron/affiliateTracking.js
+5 -50
View File
@@ -1,23 +1,4 @@
// Affiliate / referral install tracking on the desktop side.
//
// On first launch the app opens https://openswarm.com/welcome?app_install_id=…
// in the user's default browser and polls the cloud's /api/install/lookup
// endpoint until a referral binding shows up (or we time out). The browser
// page is what actually performs the bind: it reads the install_token that
// the landing page stashed in localStorage / cookie when the user clicked
// Download, and POSTs it to the cloud paired with our app_install_id.
//
// State lives in `<userData>/install.json`. The shape:
// {
// app_install_id: "uuid", // generated once per install
// first_launch_at: 1700000000000, // unix ms; presence = "this isn't first launch"
// ref: "haik" | null, // populated once lookup succeeds
// ref_bound_at: 1700000000000 | null,
// attempts: 0 // last polling attempt count, for debugging
// }
//
// Skipped entirely in dev unless OPENSWARM_AFFILIATE_FORCE=1 is set, so
// `bash run.sh` doesn't pop a browser tab on every restart.
// First-launch affiliate ref capture: opens welcome page, polls cloud lookup, persists to install.json.
const fs = require("fs");
const path = require("path");
@@ -26,13 +7,7 @@ const crypto = require("crypto");
const DEFAULT_LANDING_URL = "https://openswarm.com";
const DEFAULT_CLOUD_URL = "https://api.openswarm.com";
// Polling: 12 attempts, 5s apart = 60s window. Generous enough for the user
// to actually click through the welcome page; small enough that a stuck
// poll doesn't sit around all day. The page itself is fast (single POST)
// so most binds land in the first one or two ticks.
//
// Both knobs are overridable via env so tests can drive a 200ms × 5
// poll window instead of 60s.
// 12 attempts * 5s = 60s window; env-overridable for tests.
const POLL_INTERVAL_MS = Number(process.env.OPENSWARM_AFFILIATE_POLL_INTERVAL_MS) || 5000;
const POLL_MAX_ATTEMPTS = Number(process.env.OPENSWARM_AFFILIATE_POLL_MAX_ATTEMPTS) || 12;
@@ -54,9 +29,7 @@ function writeState(userDataDir, state) {
const p = getStateFilePath(userDataDir);
try {
fs.mkdirSync(path.dirname(p), { recursive: true });
// Atomic-ish write: temp file + rename. Avoids leaving a half-written
// install.json if the process is killed mid-write (which would brick
// first-launch detection on the next start).
// Atomic write so kill mid-write doesn't brick first-launch detection.
const tmp = p + ".tmp";
fs.writeFileSync(tmp, JSON.stringify(state, null, 2), "utf8");
fs.renameSync(tmp, p);
@@ -74,8 +47,6 @@ function urlsFromEnv() {
async function pollLookupOnce(cloudUrl, appInstallId) {
const url = `${cloudUrl}/api/install/lookup?app_install_id=${encodeURIComponent(appInstallId)}`;
// Node 18+ ships global fetch; Electron 40 is on a Chromium that has it.
// Defensive timeout via AbortSignal.timeout (Node 17+).
const controller = new AbortController();
const t = setTimeout(() => controller.abort(), 5000);
try {
@@ -114,32 +85,20 @@ async function pollUntilBound({ cloudUrl, appInstallId, userDataDir }) {
return null;
}
// Public entry: call once from app.whenReady() after backend is up. Safe to
// call on every launch — internal first-launch check makes subsequent calls
// a no-op. `shell` is electron's shell module, passed in to avoid this
// module needing to require electron at the top (keeps it test-friendly).
/** Run once from app.whenReady(); idempotent across launches. */
async function maybeRunFirstLaunchHandshake({ shell, userDataDir, isDev, isPackaged }) {
// Skip in dev to avoid spawning a browser tab on every `bash run.sh`.
// OPENSWARM_AFFILIATE_FORCE=1 lets us actually exercise the flow against
// a local landing page + local cloud during integration testing.
if (isDev && process.env.OPENSWARM_AFFILIATE_FORCE !== "1") {
return;
}
const state = readState(userDataDir);
if (state.first_launch_at) {
// Returning launch. If we never managed to bind a ref, optionally try
// again — but only for a short grace window after the original launch
// (24h) so we don't pop a browser tab on someone who's been using the
// app for a month.
// Re-poll on returning launches only within a 24h grace; don't spam old installs.
const ageMs = Date.now() - Number(state.first_launch_at || 0);
const stillInGracePeriod = Number.isFinite(ageMs) && ageMs >= 0 && ageMs < 24 * 60 * 60 * 1000;
if (state.ref || !stillInGracePeriod || !state.app_install_id) {
return;
}
// Within grace window and still no ref — silently re-poll (no second
// browser pop-up) in case the user hasn't completed the welcome page
// handshake yet.
pollUntilBound({
cloudUrl: urlsFromEnv().cloudUrl,
appInstallId: state.app_install_id,
@@ -148,7 +107,6 @@ async function maybeRunFirstLaunchHandshake({ shell, userDataDir, isDev, isPacka
return;
}
// First launch.
const appInstallId = crypto.randomUUID();
const now = Date.now();
const fresh = {
@@ -172,8 +130,6 @@ async function maybeRunFirstLaunchHandshake({ shell, userDataDir, isDev, isPacka
console.warn("[affiliate] failed to open welcome URL:", err && err.message);
}
// Fire-and-forget the polling loop. We intentionally don't await it from
// app.whenReady() so backend / window startup stays unblocked.
pollUntilBound({ cloudUrl, appInstallId, userDataDir }).catch((err) => {
console.warn("[affiliate] poll loop crashed:", err && err.message);
});
@@ -181,7 +137,6 @@ async function maybeRunFirstLaunchHandshake({ shell, userDataDir, isDev, isPacka
module.exports = {
maybeRunFirstLaunchHandshake,
// Exported for tests + IPC handlers.
_readState: readState,
_writeState: writeState,
_getStateFilePath: getStateFilePath,
electron/affiliateTracking.test.js
+4 -54
View File
@@ -1,22 +1,4 @@
// End-to-end tests for the desktop-side affiliate / referral handshake.
//
// We stand up an in-process HTTP server that implements the same contract
// as openswarm-cloud's /api/install/{mint,bind,lookup} endpoints (in-memory
// state, no SQLite). The Electron module's polling code talks to this
// server over real fetch over real loopback TCP, which is as realistic as
// it gets without booting the actual cloud Hono app.
//
// We then drive both halves of the flow:
// * The "user clicks Download on the landing page" half: mint() to get an
// install_token, stash it where the test's "welcome page" simulator can
// find it.
// * The "user installs the app" half: maybeRunFirstLaunchHandshake() with
// a fake shell that captures the welcome URL, then we simulate the
// welcome page by calling /api/install/bind from the test before the
// poll loop times out.
//
// Polling cadence is squeezed via env vars (OPENSWARM_AFFILIATE_POLL_*) so
// the suite finishes in milliseconds instead of seconds.
// E2E tests for the desktop affiliate handshake against an in-process mock cloud.
const test = require("node:test");
const assert = require("node:assert/strict");
@@ -26,17 +8,13 @@ const os = require("node:os");
const http = require("node:http");
const crypto = require("node:crypto");
// Force the tracking module to use tight polling well before requiring it,
// because the constants are read at module-load time.
// Polling envs must be set before require: constants read at module load.
process.env.OPENSWARM_AFFILIATE_POLL_INTERVAL_MS = "20";
process.env.OPENSWARM_AFFILIATE_POLL_MAX_ATTEMPTS = "30";
const affiliateTracking = require("./affiliateTracking");
// --- in-memory mock cloud --------------------------------------------------
function makeMockCloud() {
// Mirrors the install_tokens table.
const tokens = new Map();
const server = http.createServer((req, res) => {
@@ -120,8 +98,6 @@ function makeMockCloud() {
});
}
// --- fake shell ------------------------------------------------------------
function makeFakeShell() {
const opened = [];
return {
@@ -133,8 +109,6 @@ function makeFakeShell() {
};
}
// --- temp-dir helper -------------------------------------------------------
function makeTempUserDataDir() {
const dir = fs.mkdtempSync(path.join(os.tmpdir(), "openswarm-affiliate-test-"));
return dir;
@@ -148,7 +122,6 @@ function delay(ms) {
return new Promise((r) => setTimeout(r, ms));
}
// Prefer the actual install_token = call the bind endpoint with it.
async function simulateWelcomePageBind(cloudUrl, installToken, appInstallId) {
const res = await fetch(`${cloudUrl}/api/install/bind`, {
method: "POST",
@@ -173,10 +146,6 @@ function appInstallIdFromWelcomeUrl(url) {
return u.searchParams.get("app_install_id");
}
// ---------------------------------------------------------------------------
// Tests
// ---------------------------------------------------------------------------
test("first launch: opens welcome URL and binds ref via poll loop", async () => {
const cloud = await makeMockCloud();
try {
@@ -186,11 +155,8 @@ test("first launch: opens welcome URL and binds ref via poll loop", async () =>
process.env.OPENSWARM_AFFILIATE_LANDING_URL = "https://landing.test";
process.env.OPENSWARM_AFFILIATE_CLOUD_URL = cloud.url;
// 1. Pre-mint a token at the cloud as if the user had clicked Download
// on the landing page.
const installToken = await mintTokenFromCloud(cloud.url, "haik-test");
// 2. Run the desktop's first-launch handshake.
await affiliateTracking.maybeRunFirstLaunchHandshake({
shell,
userDataDir,
@@ -198,8 +164,6 @@ test("first launch: opens welcome URL and binds ref via poll loop", async () =>
isPackaged: true,
});
// 3. The shell should have been told to open the welcome URL with the
// freshly generated app_install_id.
assert.equal(shell.opened.length, 1, "exactly one browser open");
assert.ok(
shell.opened[0].startsWith("https://landing.test/welcome?app_install_id="),
@@ -209,20 +173,17 @@ test("first launch: opens welcome URL and binds ref via poll loop", async () =>
const appInstallId = appInstallIdFromWelcomeUrl(shell.opened[0]);
assert.ok(appInstallId && appInstallId.length > 8, "app_install_id present in URL");
// 4. install.json on disk now has the app_install_id but no ref yet.
const stateFile = path.join(userDataDir, "install.json");
let state = readJson(stateFile);
assert.equal(state.app_install_id, appInstallId);
assert.equal(state.ref, null);
assert.ok(state.first_launch_at > 0);
// 5. Simulate the welcome page completing the bind.
const bindResult = await simulateWelcomePageBind(cloud.url, installToken, appInstallId);
assert.equal(bindResult.status, 200);
assert.equal(bindResult.body.ref, "haik-test");
// 6. Wait for the poll loop to pick up the bind. Poll cadence is
// 20ms × 30 attempts = ~600ms upper bound; we wait up to 1s.
// Poll budget: 20ms * 30 attempts ~= 600ms; wait up to 1s.
let final = null;
for (let i = 0; i < 50; i++) {
await delay(50);
@@ -243,8 +204,6 @@ test("returning launch: no-op when ref already bound", async () => {
const shell = makeFakeShell();
process.env.OPENSWARM_AFFILIATE_CLOUD_URL = cloud.url;
// Seed install.json as if first launch already happened and a ref
// was bound a few minutes ago.
fs.writeFileSync(
path.join(userDataDir, "install.json"),
JSON.stringify({
@@ -278,8 +237,6 @@ test("returning launch within grace window: silent re-poll, no second browser po
const shell = makeFakeShell();
process.env.OPENSWARM_AFFILIATE_CLOUD_URL = cloud.url;
// Pre-mint a token + seed install.json as if first launch happened
// but the user never completed the welcome page handshake yet.
const appInstallId = "grace-app-install-id-abcdef0123";
fs.writeFileSync(
path.join(userDataDir, "install.json"),
@@ -293,7 +250,6 @@ test("returning launch within grace window: silent re-poll, no second browser po
);
const installToken = await mintTokenFromCloud(cloud.url, "grace-ref");
// Simulate a late welcome bind (user finally clicked through).
await simulateWelcomePageBind(cloud.url, installToken, appInstallId);
await affiliateTracking.maybeRunFirstLaunchHandshake({
@@ -303,10 +259,8 @@ test("returning launch within grace window: silent re-poll, no second browser po
isPackaged: true,
});
// Specifically NO browser pop-up the second time around.
assert.equal(shell.opened.length, 0, "no second browser open");
// Wait for the silent re-poll to pick up the bind.
const stateFile = path.join(userDataDir, "install.json");
let state = null;
for (let i = 0; i < 50; i++) {
@@ -346,7 +300,6 @@ test("returning launch outside grace window: skipped entirely", async () => {
});
assert.equal(shell.opened.length, 0, "no browser open after grace window");
// Give the poll loop time to NOT run.
await delay(200);
const state = readJson(path.join(userDataDir, "install.json"));
assert.equal(state.ref, null, "no ref bound");
@@ -392,7 +345,6 @@ test("dev mode: skipped unless OPENSWARM_AFFILIATE_FORCE=1", async () => {
test("install.json write is atomic-ish (temp + rename)", async () => {
const userDataDir = makeTempUserDataDir();
affiliateTracking._writeState(userDataDir, { app_install_id: "atomic-test-1234567890", ref: "x" });
// After write, the temp file shouldn't be left behind.
const files = fs.readdirSync(userDataDir);
assert.ok(files.includes("install.json"));
assert.ok(!files.some((f) => f.endsWith(".tmp")), "no leftover temp file");
@@ -412,10 +364,8 @@ test("readState returns {} when install.json is corrupt", () => {
});
test("poll loop respects max attempts and gives up", async () => {
// No cloud server at all — every poll attempt fails (ECONNREFUSED).
const userDataDir = makeTempUserDataDir();
const shell = makeFakeShell();
// Point at a port nothing's listening on.
process.env.OPENSWARM_AFFILIATE_CLOUD_URL = "http://127.0.0.1:1";
await affiliateTracking.maybeRunFirstLaunchHandshake({
@@ -425,7 +375,7 @@ test("poll loop respects max attempts and gives up", async () => {
isPackaged: true,
});
// Wait long enough for all attempts to fail. 20ms × 30 = 600ms.
// 20ms * 30 = 600ms upper bound; wait 900ms.
await delay(900);
const state = readJson(path.join(userDataDir, "install.json"));
assert.equal(state.ref, null, "no ref after exhausted polls");
electron/assets/tray-idle.ico
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 4.2 KiB

electron/assets/tray-idle.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 119 B

electron/assets/tray-idle@2x.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 165 B

electron/assets/tray-paused.ico
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 2.0 KiB

electron/assets/tray-paused.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 92 B

electron/assets/tray-paused@2x.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 115 B

electron/assets/tray-running.ico
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 8.0 KiB

electron/assets/tray-running.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 143 B

electron/assets/tray-running@2x.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 238 B

electron/main.js
+192 -356
View File
File diff suppressed because it is too large Load Diff
electron/package-lock.json
Generated
+2 -2
View File
@@ -1,12 +1,12 @@
{
"name": "openswarm",
"version": "1.0.33",
"version": "1.0.36",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "openswarm",
"version": "1.0.33",
"version": "1.0.36",
"hasInstallScript": true,
"dependencies": {
"electron-updater": "^6.3.0",
electron/package.json
+11 -3
View File
@@ -1,6 +1,6 @@
{
"name": "openswarm",
"version": "1.0.33",
"version": "1.0.36",
"description": "OpenSwarm — AI Agent Orchestrator",
"main": "main.js",
"scripts": {
@@ -28,6 +28,7 @@
"build": {
"appId": "com.clusterlabs.openswarm",
"productName": "OpenSwarm",
"electronLanguages": ["en"],
"electronDownload": {
"mirror": "https://github.com/castlabs/electron-releases/releases/download/v"
},
@@ -132,8 +133,15 @@
]
},
{
"from": "build-staging/node",
"to": "node",
"from": "build-staging/node/${arch}",
"to": "node/${arch}",
"filter": [
"**/*"
]
},
{
"from": "build-staging/uv-bin/${arch}",
"to": "backend/uv-bin",
"filter": [
"**/*"
]

Some files were not shown because too many files have changed in this diff Show More