urllib.parse.SplitResult.port raises ValueError when the authority
carries a non-numeric port; nine call sites read it unguarded, so one
malformed URL (a stray quote, a client typo) turned into an unhandled
exception in the middle of a scan, a report, an interest rule or a
proxy dial. shared/utils/net.py:url_port makes the read itself the
failure boundary; unreadable_port names the bogus text so the naabu
and banner proxy paths can report what they refused rather than
retrying against a host string that dropped its port silently.
shared/definitions/ports.py:SCHEME_PORTS replaces four private copies
of the same http/https default-port map (endpoints.py, endpoint_judge,
scan_surface/normalize, nuclei's parser, dalfox's parser).
Also renames live_hosts to answered_hosts in the report source and the
interest context, and lifts the asset query's inline is:web predicate
into predicates.answered() — the existing distinction between is:live
(2xx) and is:web (answered at all) was already the rule; live_hosts
was the one name still claiming the stronger one.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VckaDx6mBB4TsG76u61gvv
- a mail host row skips the zone checks (DNSSEC, CAA) its parent answers
for, and the panel shows no zone facts on it
- inheritance reads only v=DMARC1 records at _dmarc; a stray TXT no longer
blocks it, and a parent that never answered leaves DMARC unjudged
- the MX sweep asks mail-named and short names first, capped at
MX_SWEEP_CAP, retries names that gave no answer once, and reports names
not asked or still silent as partial
- zones and mail hosts are counted apart in the cell, the panel, the report
and the MCP summary
- the posture summary loads on the table's first activation and on a scan
switch; a pending-only summary still shows the control
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KAP7PmMKSfNAFse3s2y2Xj
SPF, DMARC, DKIM, MTA-STS, TLS-RPT, DNSSEC and CAA checked per registrable
domain by a new dns_posture stage, stored in domain_posture and folded onto
every subdomains row under the zone, so posture:<check> is a host filter whose
count equals the rows it opens.
- shared/definitions/domain_posture.py mirrored by config/domain-posture.ts
- dnsx for TXT/MX/CAA; a stdlib DO-bit query for DNSSEC (dnsx has none);
the MTA-STS policy fetched over HTTPS, so the stage is touches_target with
passive_capable
- DMARC inheritance through sp, SPF lookups counted through includes in one
batch per depth, null MX as a positive fact, DKIM stated as probed
- an unanswered zone has no verdict and marks the run partial; a capped
lookup count is not a pass
- grammar field, facet, group dimension, overview panel, target and dashboard
cell, web asset sheet section, report section, compare signal
posture_weakened, digest line, dashboard spoofable signal over the rows,
MCP tool domain_posture
- finalize re-folds so rows written after the stage carry the zone's checks
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KAP7PmMKSfNAFse3s2y2Xj
Seventh surface dimension. Passive response mining for credentials,
tokens, keys and contacts already stored in http_assets bodies and
headers — no request is ever sent, no key is tested for validity.
- shared/definitions/secrets.py: 44 detectors across cloud, code,
payments, messaging, AI, auth, data, platform and contact groups
- shared/services/secret_mining/: detector sweep, the only writer of
secrets/secret_sightings/secret_coverage, fingerprint + context
- stages/secret_mining/: one stage, one knob, on by default, off
under passive intensity via touches_target=False
- shared/services/asset_query/secret_compiler.py + the SECRET_QUERY
grammar: is:exposed / is:new / group: / secret: / state: search
- api/app/services/secret.py + api/app/api/v1/secrets.py: search,
facets, coverage, groups, detail
- worker/app/tasks/secrets.py: backfill beat for scans mined before
the stage existed
- Frontend: secrets table and detail sheet built to match the scans
table (plain values, soft state badges, no explanatory copy),
own query bar, coverage strip, quick filters
- Wired into every SURFACE_ORDER fan-out (surface scope, dashboard,
reports, rescan, scan compare, notes, MCP)
- Wired into scan compare (SPECS/COMPARE_KEYS/WATCHED_FIELDS) and
notes (ASSET_IDENTITY)
Values are stored and shown plain, no encryption, no masking, no
reveal endpoint, no audit trail — a stored value is already exposed
the moment the response was read.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XpxhGvFNFCcEEf4gZykvka
A full static review of the 3.0 tree, run as two fan-out passes with every
finding verified against the running stack rather than taken on inspection.
Correctness and security
- instance settings encrypted the mask over the stored AI key: `!= MASK` never
matched `••••••••abcd`, so saving settings destroyed the key and `test_ai`
posted the mask as a bearer token
- the MFA token was replayable for its full five-minute life; the gate now
checks revocation and consumes the token after a successful exchange
- logout revoked nothing for a bearer client while answering "Logged out"
- `POST /api-keys/{id}/test` returned the plaintext key inside httpx's error
string, which embeds the whole URL
- an interest rule could be edited or deleted through another project's id
- redis ran the celery broker under `allkeys-lru`, free to evict a queued task
or a chord counter; now `noeviction`
- nuclei's stored command carried the OAST token and every auth header
- a finalize step could poison the session and leave a scan RUNNING for ever
- `is:issue` raised TypeError on every search
- offset paging repeated and skipped rows in four dimensions
- a cancelled run kept a stale `scans.*_found` rollup
- every PDF report raised `KeyError: 'software'`
- `version_key` truncated a wide numeric run and inverted release ordering
- psycopg raises a NUL as a bare ValueError, above StatementError, so every
savepoint guard was blind to the case it was written for
Performance
- endpoints.host had no trigram index: 35.9ms seq scan -> 0.389ms
- free text compiled to one OR, not the prescribed UNION ALL: 132ms -> 41ms
- `/http-assets/summary` and `/ips/summary` materialised whole tables
- the cross-link `ip` kind ran an estate query per page and drew nothing
Frontend
- `$effect` loaders that read the state they write, an unbounded request loop
on a failing endpoint
- error states that rendered as empty states
- 2FA backup codes were unusable while the copy said they worked
- a "Scan history" dialog that always reported none
- ~400 duplicated lines across five filter bars, now one component
Build and tooling
- every scanner installed at @latest; all 13 pinned to the measured versions
- CI built `web/`, gone since 2.x, with tags that cannot be valid
- the prettier pre-commit hook discarded its filenames and always passed
- the test suite destroyed itself when run twice
Also: subdomains from public code, via a pinned github-subdomains source and a
GitHub API key that subfinder receives too; sublist3r and oneforall removed
after measuring one unique host in 89 across four targets.
Verified: 853 tests, ruff clean over eleven packages, svelte-check 0 errors,
the count-is-a-promise gate at 131 checks and 0 mismatches, and 143 GET routes
with no 5xx. Four migrations, each with an exercised downgrade.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DVkdx5ynZ6aof5kLFczmhX
Repo-wide sweep of every user-facing string: frontend components, routes,
config and stores; API details and route docs; shared definitions and
models; stage config help; report prose; MCP, toolbox, interest and
connector text; notification and worker messages.
Descriptions that explained, reassured, justified or restated their title
are deleted. Confirm dialogs name what is removed and stop. Failed actions
read "<Noun> not <verbed>", successes "<Noun> <verbed>". No reader address,
no product-as-subject, no em-dashes or parentheticals in prose.
Mirrors updated on both sides: EPSS band help, compare refusal reasons,
interest labels. Four tests asserting on old wording follow the new text.
The rules are condensed into CLAUDE.md under Writing.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SK5o8xR96cbSnAfjYSTPSe
Seventeen hardening checks evaluated inside the write from response headers,
cookies, CSP and CORS: HSTS, HTTPS redirect, framing, nosniff, CSP presence,
enforcement, inline and wildcard script sources, cookie Secure and HttpOnly,
session cookie caching, CORS origin and credentials, server and runtime
disclosure, Referrer-Policy. Each check is pass, fail or not applicable, so
every count is measured against the hosts the check applies to.
Stored per asset on http_assets and folded as a union onto the host row by one
SQL statement shared by the stage and a beat backfill that evaluates responses
stored before the checks existed. Cookies are parsed from the Set-Cookie lines
of the raw header; httpx joins repeated headers with commas and cookie expiry
dates contain commas.
Surfaces: hygiene: query field with warning, info, any and none, nine counted
leads, group-by and facet, a Web hygiene panel on the scan Overview and the
project-wide Web assets page, a host table column and hover line, a sheet
section with an HTTP Hygiene pane, a compare watched field and a report
section. No score and no vulnerability rows.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JvutSbg8hHxUMCSTMpQctW
The comment rule in CLAUDE.md is one line maximum, never narration: no
multi-line blocks, no module-header explainers, no sentences that explain or
justify the code to the reader. The tree had drifted a long way from it, and
most of the drift is mine.
Removed:
232 lines of multi-line comment blocks
203 single-line comments that argued instead of stating
154 comments joined by so/because/otherwise/rather-than
143 narrating comments in the frontend
112 test docstrings (the test name already says what it asserts)
25 test module docstrings and 25 decorative section dividers
Collapsed:
324 multi-line docstrings to one factual line
271 one-line docstrings that had a justifying clause bolted on
Kept: functional directives, section dividers, short field labels, alembic
revision headers, and the eight FastAPI route docstrings that render in /docs.
Where a fact was load-bearing it is in CLAUDE.md, which is where it belongs.
284 python tests and 34 frontend tests pass, ruff and ruff-format clean across
ten packages, svelte-check 0 errors, prettier and eslint clean, app.main imports.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013dKUV96WkQQETZBMoAG98C
Three narratives and the services lede were written for an internet-
facing estate and stated it as fact. A report on 192.168.0.0/24 led with
"Administrative and data services are reachable from the internet" and
described the NAS and the router as services "of a kind that normally
sits on a private network" — which is where they were. A deliverable
that misstates where a service is reachable from is not usable evidence.
The vantage is now read off the addresses that answered, once, on the
report source: an estate whose every address is private was reached from
inside, and the attack paths, the executive summary and the services
section all say so.
The opening sentence also lowercased the dimension labels, so it read
"the run produced web assets, endpoints, services and ips". It uses the
prose nouns the surface vocabulary already carries.
A telnet port answers with protocol negotiation and no text, and every
unprintable byte was rendered as a dot, so the banner stored for it was
twelve dots — printed in the report under "Software". A response with no
readable character is no longer recorded as a banner.
The CDN coverage note printed on scans with no CDN address.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K6LcH8uUddjJbKrahf34ys
reNgine 2 could produce a PDF. This produces a deliverable, and it is built
the way scan engines are built: sections are plug-and-play directories under
reports/, each with a pydantic config model that IS the source of truth. That
model feeds GET /reports/catalog, the builder renders its own form from it, and
the API validates against it. Adding a section is a directory. Scan Engine is to
stages what Report Template is to sections.
WeasyPrint 69 renders it, chosen over Chromium and Typst after measuring what
each can actually do. Only real CSS Paged Media gives a table of contents with
true page numbers, running headers, a PDF outline and footnotes; SysReptor
reaches the same verdict and pipes Chromium's HTML into WeasyPrint for exactly
that reason. reNgine drops the Chromium hop because it templates in Jinja and
draws its charts as SVG in Python, so a browser buys nothing and costs 400 MB
per report. Typst was rejected because a user-uploadable theme would then have
to be Typst code rather than CSS.
A theme is data. shared/definitions/report_theme.py is the schema, six themes
ship as YAML, and a user uploads their own; there is no markup in a theme, so
there is no injection surface. Fonts are vendored so an offline instance still
renders in Latin, Greek and Cyrillic.
Everything the document claims is computed once, in reports/analysis/engine.py:
a posture score whose every deduction is named, weaknesses ranked by the signals
behind the rank rather than severity alone, attack paths that chain observations
into a route, a remediation plan ordered by risk removed, per-dimension change
against the previous run, and caveats that state what was never assessed. Two
narrators read that one brief. Without AI the report still argues its case; with
AI the prose is better and the model never sees a row, only the brief, capped and
cached per check. An AI failure falls back and the report still completes.
Generation is a job. POST /reports queues, the worker walks progress over SSE,
and four formats come out of one render: PDF, self-contained HTML, Markdown and
JSON. The estimate reads each section's own limits, so the page count it promises
is close to the one it delivers.
AI moves out of Settings into its own top-level page: connection, a model per
tier, what it may be used for, what it has cost, and the narrative cache.
Verified on a 1,379-host gov.cy scan: 124 pages in 19.5s across 19 sections,
all six themes, and a broken key still producing a complete report.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017f2Mf2HGwFaTRnui4UxpRd
