A target carries a list of host names, addresses and URLs an operator already
holds. Every scan of that target starts from them, so a list earned elsewhere
no longer has to be re-pasted or re-discovered.
Each pasted line is classified by its own shape and scope-checked against the
target; a refusal names the line and the reason. Seeds enter a run through the
asset_seed stage, which resolves the names it has no answer for and materialises
what they resolve to. Stages the engine has switched off still do not run.
Seeds no longer force a focused run: scope reads the new seed_only, which only
rescan, watch and connector runs set, so a seeded census scan still covers its
target. Seed count is a material run facet, so Compare states it before any
delta it would otherwise explain.
Two stages share the subdomains table for a run: discovery's clear no longer
drops rows asset_seed wrote, and the run's seeded hosts join its merge so the
imported source survives a name discovery also finds.
Closes#1538
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LYZfaNjq3Ym8GPvWYuHisS
Three holes in target validation, found by adding them all as targets.
A host name was stored as typed, so example.com, EXAMPLE.com and
Example.Com were three targets in one project and the unique constraint
never saw a collision. Host names now case-fold; a URL folds its scheme
and host and keeps its path, since a path is case-sensitive. The
migration folds existing rows and leaves a row whose folded value is
already taken for the operator to merge.
validate_asn was ^AS\d+$ with no bound, so AS0 and
AS99999999999999999999 were accepted and then failed enrichment with an
RDAP error nobody can act on. It now takes 1 to 4294967295.
0.0.0.0/0 and ::/0 were accepted, and expand_network samples a netblock
target evenly, so the scan would have swept the whole internet from this
instance's address. Loopback, link-local, multicast, reserved space and
the default route are refused; RFC1918 stays, corporate estates are the
reason it exists. Each refusal says which bound it crossed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XxW9yfg1SEchtg6HLGHRkK
Repo-wide sweep of every user-facing string: frontend components, routes,
config and stores; API details and route docs; shared definitions and
models; stage config help; report prose; MCP, toolbox, interest and
connector text; notification and worker messages.
Descriptions that explained, reassured, justified or restated their title
are deleted. Confirm dialogs name what is removed and stop. Failed actions
read "<Noun> not <verbed>", successes "<Noun> <verbed>". No reader address,
no product-as-subject, no em-dashes or parentheticals in prose.
Mirrors updated on both sides: EPSS band help, compare refusal reasons,
interest labels. Four tests asserting on old wording follow the new text.
The rules are condensed into CLAUDE.md under Writing.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SK5o8xR96cbSnAfjYSTPSe
The comment rule in CLAUDE.md is one line maximum, never narration: no
multi-line blocks, no module-header explainers, no sentences that explain or
justify the code to the reader. The tree had drifted a long way from it, and
most of the drift is mine.
Removed:
232 lines of multi-line comment blocks
203 single-line comments that argued instead of stating
154 comments joined by so/because/otherwise/rather-than
143 narrating comments in the frontend
112 test docstrings (the test name already says what it asserts)
25 test module docstrings and 25 decorative section dividers
Collapsed:
324 multi-line docstrings to one factual line
271 one-line docstrings that had a justifying clause bolted on
Kept: functional directives, section dividers, short field labels, alembic
revision headers, and the eight FastAPI route docstrings that render in /docs.
Where a fact was load-bearing it is in CLAUDE.md, which is where it belongs.
284 python tests and 34 frontend tests pass, ruff and ruff-format clean across
ten packages, svelte-check 0 errors, prettier and eslint clean, app.main imports.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013dKUV96WkQQETZBMoAG98C
Naming an engine or a scan context with more than 200 characters reached
Postgres and came back as a 500 — the column is bounded, the request
model was not. A name of spaces was accepted outright by engines,
contexts, schedules and projects, and a project created that way got an
empty slug, which is the segment its own URL is built from.
Proxies and notification channels already ran their names through
clean_name; every other named entity now does the same, on update as
well as on create, where none of them did. A rejected name says whether
it was empty or too long instead of failing at the database.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K6LcH8uUddjJbKrahf34ys
Adding a target failed on the forms people actually paste. A value with
surrounding whitespace, a trailing slash, or a trailing root dot was
rejected outright, and "*.example.com" — how every bug bounty programme
writes a domain and its subdomains — was rejected too, though it names
exactly what a domain target already scans.
In the other direction any scheme validators.url accepts was taken as a
URL target, so ftp://x.com was stored, and the host was later read off
it as "ftp" and looked up in the registry under that name. A URL target
is now http or https, since that is what reNgine can scan.
A target value is normalised once, and that is what gets stored, so the
value on the page is the value that was validated.
The rejection said "Invalid target ID" for a value that is not an ID. It
now names the value and the accepted forms, in the same words the launch
dialog already uses. The two places that really were about a malformed
identifier keep saying so.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K6LcH8uUddjJbKrahf34ys
- Added `clean_name` function to `validation.py` for name validation with max length.
- Simplified docstrings in `dnsx` client, parser, and service files for clarity.
- Enhanced error handling in `ViewDNSService` to prevent disabling keys on rate limits.
- Improved WHOIS service to handle concurrent inserts gracefully with integrity checks.
- Updated `Settings` class in worker config to inherit from `BaseAppSettings`.
- Added new dependencies in `pyproject.toml` for `apprise` and `cryptography`.
- Added models for WHOIS responses including WhoisDomainResponse, WhoisIPResponse, and WhoisASNResponse.
- Created a parser for WHOIS data to convert raw responses into structured models.
- Developed an RDAPProvider class to handle RDAP lookups using the whoisit library.
- Implemented a WhoisService class for performing WHOIS lookups with optional database caching.
- Added methods for finding correlations between WHOIS records based on registrant, registrar, network, country, and nameservers.
- Updated Dockerfile to include tools directory and added rengine-tools dependency in pyproject.toml.
