Files
rengine/worker
Yogesh OjhaandClaude Opus 5 2b433a4e1e perf(software): the backfill converges and match_scan merges instead of rebuilding
http_assets.software was never declared on the model, so the components the
probe parsed were dropped at insert and the five-minute backfill became the
only writer. Its pending test was an empty list beside a tech, and a
versionless technology yields an empty list, so those rows were queued for
ever. Each tick rebuilt the scan with a delete and a reinsert, and a tick
outran its own interval, so eight ran at once, deadlocked on the same scan
and held every worker slot.

The column is three-state: NULL is not evaluated, [] is evaluated with
nothing versioned. The probe writes it at insert. The backfill reads NULL
alone under an instance lock and matches only when it wrote a component.
match_scan builds the scan's rows in a temp table, applies confidence and
rank there, and merges them by fingerprint under a per-scan advisory lock, so
a row whose columns did not move is not written and finalize, the nightly
rematch and the backfill queue rather than deadlock. Backfill beat entries
expire after one interval. software_cves carries the result tables'
autovacuum thresholds. The dashboard store loads its cells beside the
overview rather than after it.

A full rematch of 441,044 rows now writes none of them. The dashboard
overview went from 13.7 s to 0.8 s and its software count from 13.6 s to
22 ms with no heap fetches.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CjhsosXbGgSkBU5z5yPYiz
2026-09-15 17:22:35 +05:30
..
2026-02-04 20:05:28 +05:30
2026-02-04 20:05:28 +05:30