The SSE route authenticated through CurrentUser, whose session dependency is
only released when the response ends — so every open stream left one connection
idle in transaction for as long as the tab stayed open. The engine was created
with no pool arguments at all, so the ceiling was SQLAlchemy's default 5+10:
fifteen tabs and every DB endpoint blocked for pool_timeout and then 500'd.
Reproduced exactly, and both pinned backends' last query was get_current_user's
SELECT users.
StreamUser opens and closes its own short session, so a stream holds nothing
pooled — 200 concurrent streams now report checked_out 1.
The pool is configured rather than defaulted (10+20, pre-ping, 30m recycle,
fail-fast timeout) and carries an application_name, because a blank one is why
this needed a live reproduction to diagnose. idle_in_transaction_session_timeout
is the backstop: any future leak from any path is reclaimed by Postgres instead
of accumulating, and pre-ping makes the next checkout succeed on a fresh socket.
A stream also no longer outlives its token. It re-checks expiry and revocation
on each heartbeat — JWT plus Redis, no DB — and emits `event: unauthorized`, so
logout actually ends the feed and the client refreshes at once instead of
burning three failed reconnects.
Worker pools are sized per prefork child (8 x 15 was over max_connections) and
the inherited engine is disposed on fork so a child never shares a parent socket.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015biqVXTng75nnfKXMqm61H
- Implemented TargetEnrichmentStage for enriching targets with DNS, WHOIS, and BGP context.
- Added VhostStage for virtual host discovery using host-header fuzzing.
- Introduced WafDetectStage for fingerprinting web application firewalls.
- Created configuration files for each new stage with relevant parameters.
- Updated Dockerfile to copy stages instead of engines.
- Enhanced existing tools and services for better integration with new stages.
- Refactored task management in Celery to accommodate new stages and improve error handling.
