Files
rengine/shared/definitions/interest.py
T
Yogesh OjhaandClaude Opus 5 0b853e04b6 chore: production-readiness review across the tree
A full static review of the 3.0 tree, run as two fan-out passes with every
finding verified against the running stack rather than taken on inspection.

Correctness and security
- instance settings encrypted the mask over the stored AI key: `!= MASK` never
  matched `••••••••abcd`, so saving settings destroyed the key and `test_ai`
  posted the mask as a bearer token
- the MFA token was replayable for its full five-minute life; the gate now
  checks revocation and consumes the token after a successful exchange
- logout revoked nothing for a bearer client while answering "Logged out"
- `POST /api-keys/{id}/test` returned the plaintext key inside httpx's error
  string, which embeds the whole URL
- an interest rule could be edited or deleted through another project's id
- redis ran the celery broker under `allkeys-lru`, free to evict a queued task
  or a chord counter; now `noeviction`
- nuclei's stored command carried the OAST token and every auth header
- a finalize step could poison the session and leave a scan RUNNING for ever
- `is:issue` raised TypeError on every search
- offset paging repeated and skipped rows in four dimensions
- a cancelled run kept a stale `scans.*_found` rollup
- every PDF report raised `KeyError: 'software'`
- `version_key` truncated a wide numeric run and inverted release ordering
- psycopg raises a NUL as a bare ValueError, above StatementError, so every
  savepoint guard was blind to the case it was written for

Performance
- endpoints.host had no trigram index: 35.9ms seq scan -> 0.389ms
- free text compiled to one OR, not the prescribed UNION ALL: 132ms -> 41ms
- `/http-assets/summary` and `/ips/summary` materialised whole tables
- the cross-link `ip` kind ran an estate query per page and drew nothing

Frontend
- `$effect` loaders that read the state they write, an unbounded request loop
  on a failing endpoint
- error states that rendered as empty states
- 2FA backup codes were unusable while the copy said they worked
- a "Scan history" dialog that always reported none
- ~400 duplicated lines across five filter bars, now one component

Build and tooling
- every scanner installed at @latest; all 13 pinned to the measured versions
- CI built `web/`, gone since 2.x, with tags that cannot be valid
- the prettier pre-commit hook discarded its filenames and always passed
- the test suite destroyed itself when run twice

Also: subdomains from public code, via a pinned github-subdomains source and a
GitHub API key that subfinder receives too; sublist3r and oneforall removed
after measuring one unique host in 89 across four targets.

Verified: 853 tests, ruff clean over eleven packages, svelte-check 0 errors,
the count-is-a-promise gate at 131 checks and 0 mismatches, and 143 GET routes
with no 5xx. Four migrations, each with an exercised downgrade.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DVkdx5ynZ6aof5kLFczmhX
2026-09-13 18:03:29 +05:30

293 lines
7.6 KiB
Python

"""What makes an asset an exposure, and how much each reason weighs."""
from __future__ import annotations
from dataclasses import dataclass
from enum import StrEnum
MAX_SCORE = 100
MAX_SIGNALS_PER_HOST = 8
MAX_RULES = 200
MAX_KEYWORDS = 120
MAX_RULE_NAME = 80
MAX_REASON = 400
MAX_EVIDENCE = 300
MIN_ESTATE_FOR_RARITY = 25
RARE_SHARE = 0.05
RARE_MAX_HOSTS = 3
EDGE_MAJORITY = 0.7
class InterestSource(StrEnum):
KEYWORD = "keyword"
RULE = "rule"
CORRELATION = "correlation"
AI = "ai"
SOURCE_LABELS: dict[str, str] = {
InterestSource.KEYWORD.value: "Keywords",
InterestSource.RULE.value: "Rule",
InterestSource.CORRELATION.value: "Correlation",
InterestSource.AI.value: "AI",
}
SOURCE_HELP: dict[str, str] = {
InterestSource.KEYWORD.value: "Matched a configured keyword.",
InterestSource.RULE.value: "Matched a saved query.",
InterestSource.CORRELATION.value: "Stands out from the rest of this estate.",
InterestSource.AI.value: "A judgement written by a model, not an observation.",
}
JUDGEMENT_SOURCES: frozenset[str] = frozenset({InterestSource.AI.value})
class InterestKind(StrEnum):
ADMIN_INTERFACE = "admin_interface"
DEVELOPER_TOOLING = "developer_tooling"
REMOTE_ACCESS = "remote_access"
BUSINESS_SYSTEM = "business_system"
NON_PRODUCTION = "non_production"
LEGACY = "legacy"
INTERNAL_NAMING = "internal_naming"
NO_AUTHENTICATION = "no_authentication"
EXPOSED_CONTENT = "exposed_content"
DIAGNOSTIC = "diagnostic"
SENSITIVE_SERVICE = "sensitive_service"
EXPLOITED_SOFTWARE = "exploited_software"
CERTIFICATE_ANOMALY = "certificate_anomaly"
TAKEOVER_RISK = "takeover_risk"
NETWORK_OUTLIER = "network_outlier"
RARE_TECHNOLOGY = "rare_technology"
RARE_IDENTITY = "rare_identity"
UNPROTECTED_EDGE = "unprotected_edge"
NEWLY_APPEARED = "newly_appeared"
OTHER = "other"
@dataclass(frozen=True)
class KindSpec:
key: str
label: str
help: str
weight: int
tone: str
TONE_WARNING = "warning"
TONE_INFO = "info"
TONE_NEUTRAL = "neutral"
KINDS: tuple[KindSpec, ...] = (
KindSpec(
InterestKind.ADMIN_INTERFACE.value,
"Administrative interface",
"A control panel or management console reachable from the internet.",
30,
TONE_WARNING,
),
KindSpec(
InterestKind.DEVELOPER_TOOLING.value,
"Developer tooling",
"Build, source or observability tooling answering on a public hostname.",
28,
TONE_WARNING,
),
KindSpec(
InterestKind.REMOTE_ACCESS.value,
"Remote access",
"A VPN, gateway or remote desktop portal.",
28,
TONE_WARNING,
),
KindSpec(
InterestKind.BUSINESS_SYSTEM.value,
"Business system",
"An ERP, finance or HR application exposed to the internet.",
24,
TONE_WARNING,
),
KindSpec(
InterestKind.NON_PRODUCTION.value,
"Non-production",
"Named as a staging, test or sandbox deployment.",
20,
TONE_INFO,
),
KindSpec(
InterestKind.LEGACY.value,
"Legacy",
"Named as old, retired or superseded and answering.",
22,
TONE_INFO,
),
KindSpec(
InterestKind.INTERNAL_NAMING.value,
"Internal naming",
"Carries internal or private naming on a publicly resolvable host.",
18,
TONE_INFO,
),
KindSpec(
InterestKind.NO_AUTHENTICATION.value,
"No authentication",
"Answered without authentication where authentication is expected.",
26,
TONE_WARNING,
),
KindSpec(
InterestKind.EXPOSED_CONTENT.value,
"Exposed content",
"Serves a directory listing, an archive or a sensitive file.",
30,
TONE_WARNING,
),
KindSpec(
InterestKind.DIAGNOSTIC.value,
"Diagnostic output",
"Returns stack traces, debug pages or environment detail.",
26,
TONE_WARNING,
),
KindSpec(
InterestKind.EXPLOITED_SOFTWARE.value,
"Known exploited software",
"Runs a product on the CISA Known Exploited Vulnerabilities list. "
"The version was not identified.",
28,
TONE_WARNING,
),
KindSpec(
InterestKind.SENSITIVE_SERVICE.value,
"Sensitive service",
"An administrative or database port is open on this host.",
26,
TONE_WARNING,
),
KindSpec(
InterestKind.CERTIFICATE_ANOMALY.value,
"Certificate anomaly",
"The certificate is expired, self-signed or names an unexpected subject.",
14,
TONE_INFO,
),
KindSpec(
InterestKind.TAKEOVER_RISK.value,
"Takeover risk",
"A CNAME points at a name that does not answer.",
30,
TONE_WARNING,
),
KindSpec(
InterestKind.NETWORK_OUTLIER.value,
"Network outlier",
"Sits on a network almost nothing else in this estate uses.",
18,
TONE_INFO,
),
KindSpec(
InterestKind.RARE_TECHNOLOGY.value,
"Rare technology",
"Runs software almost nothing else in this estate runs.",
16,
TONE_INFO,
),
KindSpec(
InterestKind.RARE_IDENTITY.value,
"Distinct application",
"Serves an icon or page shared by almost nothing else in this estate.",
14,
TONE_INFO,
),
KindSpec(
InterestKind.UNPROTECTED_EDGE.value,
"Outside the edge",
"Answers directly while the rest of this estate sits behind a CDN.",
20,
TONE_INFO,
),
KindSpec(
InterestKind.NEWLY_APPEARED.value,
"New",
"Absent from the previous scan of this target.",
10,
TONE_NEUTRAL,
),
KindSpec(
InterestKind.OTHER.value,
"Other reason",
"Flagged without a more specific reason.",
12,
TONE_NEUTRAL,
),
)
KIND_BY_KEY: dict[str, KindSpec] = {k.key: k for k in KINDS}
KIND_KEYS: tuple[str, ...] = tuple(k.key for k in KINDS)
KIND_LABELS: dict[str, str] = {k.key: k.label for k in KINDS}
KIND_WEIGHTS: dict[str, int] = {k.key: k.weight for k in KINDS}
def kind_weight(kind: str) -> int:
spec = KIND_BY_KEY.get(kind)
return spec.weight if spec else KIND_BY_KEY[InterestKind.OTHER.value].weight
def kind_label(kind: str) -> str:
spec = KIND_BY_KEY.get(kind)
return spec.label if spec else kind.replace("_", " ").capitalize()
def coerce_kind(value: str | None) -> str:
raw = (value or "").strip().lower().replace("-", "_").replace(" ", "_")
return raw if raw in KIND_BY_KEY else InterestKind.OTHER.value
class InterestBand(StrEnum):
NOTABLE = "notable"
HIGH = "high"
CRITICAL = "critical"
BAND_ORDER: tuple[str, ...] = (
InterestBand.CRITICAL.value,
InterestBand.HIGH.value,
InterestBand.NOTABLE.value,
)
BAND_LABELS: dict[str, str] = {
InterestBand.CRITICAL.value: "Critical",
InterestBand.HIGH.value: "High",
InterestBand.NOTABLE.value: "Notable",
}
BAND_TONES: dict[str, str] = {
InterestBand.CRITICAL.value: "destructive",
InterestBand.HIGH.value: TONE_WARNING,
InterestBand.NOTABLE.value: TONE_INFO,
}
# score at or above which a host enters the band
BAND_FLOOR: dict[str, int] = {
InterestBand.CRITICAL.value: 55,
InterestBand.HIGH.value: 30,
InterestBand.NOTABLE.value: 1,
}
class RuleMode(StrEnum):
KEYWORD = "keyword"
QUERY = "query"
RULE_MODE_LABELS: dict[str, str] = {
RuleMode.KEYWORD.value: "Keywords",
RuleMode.QUERY.value: "Query",
}
KEYWORD_FIELD_LABELS: dict[str, str] = {
"host": "Hostname",
"title": "Page title",
}