Commit Graph
4 Commits
Author SHA1 Message Date
Yogesh OjhaandClaude Opus 5 d41a66deb9 refactor(query): one builder for a query the parser refused
Eleven call sites across six services each spelled the same
QueryError(message=..., hint=..., start=..., end=...) so the UI could
underline the query in place. errors.py already owned the other half
of that vocabulary, query_error_for(DBAPIError); syntax_error() is its
counterpart and the services call it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DafWxfoWZXVDgr9KroJBDZ
2026-09-16 08:45:15 +05:30
Yogesh OjhaandClaude Fable 5.1 0a69f3201c copy: state the fact, delete the explanation
Repo-wide sweep of every user-facing string: frontend components, routes,
config and stores; API details and route docs; shared definitions and
models; stage config help; report prose; MCP, toolbox, interest and
connector text; notification and worker messages.

Descriptions that explained, reassured, justified or restated their title
are deleted. Confirm dialogs name what is removed and stop. Failed actions
read "<Noun> not <verbed>", successes "<Noun> <verbed>". No reader address,
no product-as-subject, no em-dashes or parentheticals in prose.

Mirrors updated on both sides: EPSS band help, compare refusal reasons,
interest labels. Four tests asserting on old wording follow the new text.
The rules are condensed into CLAUDE.md under Writing.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SK5o8xR96cbSnAfjYSTPSe
2026-09-12 11:35:07 +05:30
Yogesh OjhaandClaude Opus 5 2cb8d73280 style: strip explanatory comments across the codebase
The comment rule in CLAUDE.md is one line maximum, never narration: no
multi-line blocks, no module-header explainers, no sentences that explain or
justify the code to the reader. The tree had drifted a long way from it, and
most of the drift is mine.

Removed:
  232  lines of multi-line comment blocks
  203  single-line comments that argued instead of stating
  154  comments joined by so/because/otherwise/rather-than
  143  narrating comments in the frontend
  112  test docstrings (the test name already says what it asserts)
   25  test module docstrings and 25 decorative section dividers

Collapsed:
  324  multi-line docstrings to one factual line
  271  one-line docstrings that had a justifying clause bolted on

Kept: functional directives, section dividers, short field labels, alembic
revision headers, and the eight FastAPI route docstrings that render in /docs.
Where a fact was load-bearing it is in CLAUDE.md, which is where it belongs.

284 python tests and 34 frontend tests pass, ruff and ruff-format clean across
ten packages, svelte-check 0 errors, prettier and eslint clean, app.main imports.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013dKUV96WkQQETZBMoAG98C
2026-09-11 14:11:23 +05:30
Yogesh OjhaandClaude Opus 5 cb1412ce90 feat(interest): a scan can say which twenty of five thousand hosts to open first
reNgine 2 answered this with one global keyword list matched by `icontains`, and
rebuilt that query once per table row — `get_is_interesting` was a
SerializerMethodField, so a 5,000-host page ran the whole unindexable OR-chain
5,000 times. It was also dead until the user saved the settings form once, since
`get_interesting_subdomains` returns none() with no custom_type row.

The rule survives; everything around it is rebuilt. An interest rule IS a saved
HOST_QUERY query, so there is no second matcher to keep in sync, and the keyword
box compiles to `host:[…] or title:[…]` behind the same grammar. Signals persist
to interest_signals, roll up in SQL onto subdomains.interest_score/band/kinds so
sorting and `is:interesting` stay index-backed, and the scan carries a signature
of the rules it was labelled against — a stale read refreshes in the background,
so editing a keyword still re-labels every past scan without a read-time query.

interest/ is a top-level package mounted into api and worker like stages/ and
reports/, with providers auto-discovered from disk. Three ship:

  rules        the keyword lists and 19 saved queries
  correlation  what stands out from the estate, which no keyword can see
  ai           a judgement, labelled as one everywhere it appears

Correlation is the half a rule cannot express: an ASN carrying under 5% of the
surface, software or a favicon on three hosts or fewer, the one asset answering
directly while the rest sits behind a CDN. All gated on a minimum estate size,
because rarity means nothing without a norm. On gov.cy: 226 flagged of 1,379 in
0.4s, 29 of them critical.

AI sees hostname, status, page title and technology for hosts that answered, and
nothing else — never a response body — deduped to three samples per identical
response shape. It may answer only in a closed schema, a host it was not given is
dropped rather than stored, and its sentence must state what was observed, never
what is exploitable. Page titles are attacker-controlled, so they travel as JSON
data with strip_control and a cap, never interpolated into the instruction, and
nothing it returns triggers an action. Every AI signal stores its model and
prompt version, and the UI never lets it read as an observation.

Then the judgement graduates: suggest.py shows the model what it flagged and asks
for the query that would catch it again without a model. Each proposal is
compiled, counted against the scan and dropped if it errors, matches nothing or
already exists; nothing is stored until a person presses Add, after which it runs
on every keyless install with no cost.

Alerts are delta-only — hosts the target has never flagged before, and only those
in a high band or matched by a rule the user asked to be notified about.

Four bugs found on the way. `dr` matched drones.gov.cy, so shipped hostname rules
are anchored on label boundaries and false positives went from 6 hosts to 0.
predicates.interest_signal aliased InterestSignal while its condition used the
unaliased class, uncorrelating the EXISTS so every interest: field returned the
unfiltered total. The correlation aggregates needed explicit output aliases or a
bare `t` collided with the CTE and printed a UUID inside a sentence. And dismiss
removed the signals while leaving the denormalised rank, so the row stayed in the
list.

The asset query compiler moves to shared/ because the worker has no api/ and has
to compile a rule at finalize; api keeps a shim that aliases every submodule into
sys.modules, so no existing import changes.

The rule library lives at /interest, not in the Arsenal. The Arsenal is tools.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NVh1CGtjDA33fupXjLGbk2
2026-09-06 19:30:04 +05:30