- Store each run's row, first-seen and retired counts (scan_deltas, scan_retired), valid while
the revision stamp matches scan_revisions; statement triggers on the seven result tables bump
it. Finalize warms, a beat backfill fills the rest, a live run answers from its last count and
queues a refresh. /scans 1.5 s -> 43 ms, dashboard overview 5-10 s -> 0.9 s.
- lead_cache holds a live scope for 20 s with a stale copy, and every build is single-flight;
search pages opt out.
- Leads: lowercase subdomain names (CHECK) and LIKE for host:, a guard for empty JSON arrays,
branches as MATERIALIZED CTEs read to the cap, every count in one statement.
- Addresses and services: the derived CTE as one grouped pass, rows and count in one execution
(page_rows), GIN lookups for page hosts, facets through grouping sets.
- Correlation carriers counted by EXISTS; takeover latest-sighting in SQL, no silent cap.
- Screenshot backfill partial index, nine unused low-cardinality indexes dropped, hostname
helpers memoised, vhost names normalised.
Every place that compares a stored address as INET guards the cast
with a character test first, so a value that is not an address reads
as NULL instead of raising 22P02 — except the IPs ordering, which cast
straight to INET. The guard is written once as predicates.inet_of and
the four copies of its regex are gone: the host, ip, software and
vulnerability compilers, and now the sort.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DafWxfoWZXVDgr9KroJBDZ
Every result and management table gains multi-row selection and the floating
action bar targets and scans already had: copy, export selection, delete, and
the dimension's own actions. Rescan and triage move off the inline strip, which
now only offers the rescan that acts on the whole query.
Delete is dimension-local. A row takes only what cannot stand without it: a web
asset its stored responses, a secret its sightings. Endpoints, services and
findings on a deleted host are kept, so every count stays equal to its table.
POST /surface/delete is the one route and the scope is a WHERE, so an id from
another scan deletes nothing. Vulnerabilities are addressable by template_id so
the Issues view deletes a whole check. Exposures dismiss rather than delete: an
exposure row is a web asset.
Selection export runs through the export pipeline. Every surface filter gains
ids, capped by MAX_SELECTED_ROWS.
Secrets and software become exportable, which fixes two breaks: a bundle export
raised KeyError on the secrets dimension, and mcp read a SURFACE_COLUMNS entry
for secrets that did not exist.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XyDpdVXL7YVsr5D7BVj2zt
The export engine, with no surface on it yet. Given a dimension, a scope
and the filter body a result table POSTs to search, it writes the same
rows to csv, json or plain text.
It selects only the columns the file will carry, so a vulnerability
export does not drag a stored request and response through memory to
throw them away, and it streams with yield_per, so a 50,000 row export
holds one chunk at a time. Statement timeout is the export's own: a
background job does not read under the api's twenty second ceiling.
Two columns a table shows are joined per chunk rather than per row —
the ports behind a host's addresses, and the hosts behind an address —
because #523 and #703 ask for those by name. The four rollup counts the
api computes per page are left out rather than written blank.
A cell opening with = + - @ tab or return is a formula to Excel and
Sheets, and page titles and banners are written by the target, so every
cell is prefixed and stripped of control characters. CSV carries a BOM
or Excel reads utf-8 as latin-1.
Proven against a live scan and in tests: for every dimension, filtered
or not, the export writes exactly as many rows as the table reports.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LYZfaNjq3Ym8GPvWYuHisS
The five search services build their statement with pure, session-free
helpers — _context, _apply_filter, _order, _scoped — and then enrich the
page asynchronously. Only the enrichment needs the api. Moving the
builders to shared/services/surface_query lets the worker compile the
same filtered set the table shows, which an export has to do and could
not reach from worker/app.
Web assets gains the _scoped it never had: its base statement was
inlined in search, leads, seeds and renders, so a predicate could drift
between the rows a query returns and the rows its leads count. The four
now read one definition.
No behaviour change. Every route over the moved builders — search,
leads, groups, renders and facets across all five dimensions — was
exercised against a live scan.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LYZfaNjq3Ym8GPvWYuHisS
Yogesh Ojha