[eric] router: a completed subscription connect bounces 9Router so its dispatch state rebuilds; new subs no longer dead until app restart (ENG-315)

This commit is contained in:
ciregenz
2026-08-16 08:11:09 -07:00
parent a81958acb9
commit b9afcae6ac
4 changed files with 87 additions and 0 deletions
+3
View File
@@ -534,6 +534,9 @@ async def subscriptions_poll(body: dict):
p_sync(load_settings().model_dump())
from backend.apps.subscription.free_trial import clear_free_trial_on_connect
await clear_free_trial_on_connect()
# Background so the UI's "Connected" lands instantly; the bounce takes ~5-10s (ENG-315).
from backend.apps.nine_router.bounce_after_connect import bounce_router_after_connect
asyncio.create_task(bounce_router_after_connect(provider))
return result
except Exception as e:
raise HTTPException(status_code=500, detail=str(e))
+1
View File
@@ -91,6 +91,7 @@ P_RELEASES: List[ReleaseNote] = [
"Starting a new chat no longer makes another agent's revealed subagents disappear from the canvas. Their cards were being cleaned up as strays by the same pass that places the new one.",
"Workflow run history shows each run's workflow name instead of the word \"Workflow\" on every row. The name now travels with the run, so it survives renames and deleted workflows.",
"An expired provider login heals itself mid-chat: the first failure rebuilds the connection and retries your message with zero clicks, and only a second failure asks you to reconnect. It used to take six manual steps every time a token aged out.",
"A newly connected ChatGPT or Gemini subscription works immediately. The routing layer restarts itself the moment a connect completes, so new subscriptions no longer sit dead behind rate-limit errors until you restart the app.",
"Heavy sessions no longer vanish without a trace. When memory climbs past the safe line the app now sheds weight itself: preview thumbnails pause and refetchable caches drop, instead of growing until the operating system kills it mid-task.",
],
),
@@ -0,0 +1,37 @@
"""Restart 9Router right after a subscription connect completes (ENG-315).
A freshly connected subscription can sit dead until OpenSwarm restarts, dying on rate-limit-shaped
errors. Every layer WE own reads live (picker payload, provider env, route resolution: all
verified no-restart on packaged exp.9), so the stale state is inside the router process itself:
0.3.60 stamps failing connections with `modelLock_<model>` cooldowns + `testStatus:"unavailable"`
in its in-memory DB, and dispatch skips locked connections, which reads as a rate limit to the
caller. A router restart re-reads db.json and was measured to land every arm in a known-good state
within ~10s, which is exactly why "restart OpenSwarm" cures the reported bug: the app restart's
only relevant side effect IS the router restart. So do just that part, at the connect chokepoint.
Safe mid-session: the CLI retries provider 5xx itself (10 attempts / 30s), and the kill-drill on
the packaged build showed an in-flight agent surviving a router bounce with the session resuming.
"""
import asyncio
import logging
from typeguard import typechecked
from backend.apps.nine_router import process as p_router
logger = logging.getLogger(__name__)
@typechecked
async def bounce_router_after_connect(provider: str) -> bool:
"""Sequential stop -> ensure_running. Never two live routers (a parallel pair once rotated one
token family to death); never raises, a failed bounce leaves the watchdog to revive."""
try:
logger.info(f"bouncing 9Router after {provider} connect so its dispatch state is rebuilt")
p_router.stop()
await asyncio.sleep(0.5)
await p_router.ensure_running()
return p_router.is_running()
except Exception:
logger.warning("post-connect router bounce failed; watchdog will revive", exc_info=True)
return False
@@ -0,0 +1,46 @@
"""A freshly connected subscription must not stay dead until an app restart (ENG-315).
Live evidence on packaged exp.9: with the router knowing the sub and the app booted without it,
picker payload, fresh GPT session, and a pre-connect session switched to GPT all worked with zero
restart, so every layer we own reads live. The remaining stale layer is the router process itself
(0.3.60 stamps modelLock cooldowns + testStatus into its in-memory DB and skips locked connections
at dispatch), and a router restart was the measured heal. These pin that connect-success actually
schedules that heal, sequentially, and that a failed bounce cannot take the poll route down.
"""
import asyncio
import inspect
import pytest
from backend.apps.nine_router import bounce_after_connect as b
@pytest.mark.asyncio
async def test_bounce_is_sequential_stop_then_start(monkeypatch):
order = []
monkeypatch.setattr(b.p_router, "stop", lambda: order.append("stop"))
async def p_start():
order.append("start")
monkeypatch.setattr(b.p_router, "ensure_running", p_start)
monkeypatch.setattr(b.p_router, "is_running", lambda: True)
assert await b.bounce_router_after_connect("codex") is True
assert order == ["stop", "start"], "two live routers once rotated a token family to death; stop must fully precede start"
@pytest.mark.asyncio
async def test_a_failed_bounce_never_raises(monkeypatch):
def p_boom():
raise RuntimeError("router dir vanished")
monkeypatch.setattr(b.p_router, "stop", p_boom)
assert await b.bounce_router_after_connect("codex") is False
def test_poll_success_schedules_the_bounce():
# Wire-check both directions: the heal exists AND the connect chokepoint calls it (ENG-284 rule).
from backend.apps.agents import agents as agents_mod
src = inspect.getsource(agents_mod)
poll = src[src.index("async def subscriptions_poll"):src.index("async def subscriptions_exchange")]
assert "bounce_router_after_connect" in poll, "an unheal-ed connect is the whole ENG-315 bug"
assert 'result.get("success")' in poll.split("bounce_router_after_connect")[0], "the bounce must be gated on OAuth success, never on every poll tick"
assert "create_task" in poll, "the UI's Connected flash must not wait ~10s on the bounce"