Commit Graph
5 Commits
Author SHA1 Message Date
Yogesh Ojha a548afffa2 feat(settings): tables that save on change, Users, sheets on their pages
Settings has six tabs: General, API keys, Proxies, Notifications, AI and
Users. Each tab is one card whose controls save on change; list tabs are
tables with a row menu and their Add action in the tab row. A non-admin
sees General and AI, read-only.

Out-of-band testing is the Callback server sheet on the Arsenal Nuclei
tab, with a Need a callback count that filters the library to the same
rows. Bounty Hub settings are a sheet on the Bounty Hub page. The old
settings routes redirect to both.

- API keys store the last test result on the row and clear it when the
  key changes. Providers report whether a test exists. The usage count
  is gone: only ViewDNS ever incremented it.
- Notification channels store the last delivery, written by the
  notifier on its own connection. A channel offers the seven event
  types the product emits, program changes on by default, and levels
  named All events, Warning and critical, Critical only.
- A default proxy applies to a scan launched without a context and is
  preselected for new contexts. A pool hands each scan one endpoint,
  proxy URLs bracket IPv6 hosts, and deleting a proxy clears it from
  its contexts.
- Users can be added with a role, disabled and promoted. An account
  cannot change itself, and one that owns records answers 409.
- Retention windows are validated and include keep forever. The
  instance name cannot be blank and names every page title.
- The AI list drops the two switches nothing read; each feature states
  what it sends. The out-of-band Off help says the checks are skipped.

Connectors, Bounty Hub and Remote control lose their extra page
padding. Migration 5d2e9a8c1b47 adds the test and delivery columns and
drops proxies.mode.

Claude-Session: https://claude.ai/code/session_01RVAHQxZ7JAkoEiTCDiHYue
2026-09-27 10:01:19 +05:30
Yogesh OjhaandClaude Opus 5 0b853e04b6 chore: production-readiness review across the tree
A full static review of the 3.0 tree, run as two fan-out passes with every
finding verified against the running stack rather than taken on inspection.

Correctness and security
- instance settings encrypted the mask over the stored AI key: `!= MASK` never
  matched `••••••••abcd`, so saving settings destroyed the key and `test_ai`
  posted the mask as a bearer token
- the MFA token was replayable for its full five-minute life; the gate now
  checks revocation and consumes the token after a successful exchange
- logout revoked nothing for a bearer client while answering "Logged out"
- `POST /api-keys/{id}/test` returned the plaintext key inside httpx's error
  string, which embeds the whole URL
- an interest rule could be edited or deleted through another project's id
- redis ran the celery broker under `allkeys-lru`, free to evict a queued task
  or a chord counter; now `noeviction`
- nuclei's stored command carried the OAST token and every auth header
- a finalize step could poison the session and leave a scan RUNNING for ever
- `is:issue` raised TypeError on every search
- offset paging repeated and skipped rows in four dimensions
- a cancelled run kept a stale `scans.*_found` rollup
- every PDF report raised `KeyError: 'software'`
- `version_key` truncated a wide numeric run and inverted release ordering
- psycopg raises a NUL as a bare ValueError, above StatementError, so every
  savepoint guard was blind to the case it was written for

Performance
- endpoints.host had no trigram index: 35.9ms seq scan -> 0.389ms
- free text compiled to one OR, not the prescribed UNION ALL: 132ms -> 41ms
- `/http-assets/summary` and `/ips/summary` materialised whole tables
- the cross-link `ip` kind ran an estate query per page and drew nothing

Frontend
- `$effect` loaders that read the state they write, an unbounded request loop
  on a failing endpoint
- error states that rendered as empty states
- 2FA backup codes were unusable while the copy said they worked
- a "Scan history" dialog that always reported none
- ~400 duplicated lines across five filter bars, now one component

Build and tooling
- every scanner installed at @latest; all 13 pinned to the measured versions
- CI built `web/`, gone since 2.x, with tags that cannot be valid
- the prettier pre-commit hook discarded its filenames and always passed
- the test suite destroyed itself when run twice

Also: subdomains from public code, via a pinned github-subdomains source and a
GitHub API key that subfinder receives too; sublist3r and oneforall removed
after measuring one unique host in 89 across four targets.

Verified: 853 tests, ruff clean over eleven packages, svelte-check 0 errors,
the count-is-a-promise gate at 131 checks and 0 mismatches, and 143 GET routes
with no 5xx. Four migrations, each with an exercised downgrade.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DVkdx5ynZ6aof5kLFczmhX
2026-09-13 18:03:29 +05:30
Yogesh OjhaandClaude Opus 5 163df52ad5 perf(db): a primary key is already an index, so the second one on id was write cost for nothing
SQLModel emitted index=True beside primary_key=True on every id field, so 41
tables carried ix_<table>_id alongside <table>_pkey: the same single-column btree
twice. 12 MB of duplicate index, and every insert maintaining both — on the hot
path of a scan writing 345,152 endpoints.

The planner had been using them (72,160 scans across the set) simply because
they were there and equivalent; with them gone it uses the primary key, verified
on subdomains: Index Scan using subdomains_pkey.

The migration finds them rather than naming them, because which tables exist
depends on how far an install has migrated and a name already gone must not fail
an upgrade. It drops only a non-unique, single-column btree on id belonging to a
table that has a primary key — no other index can match that shape. There is no
downgrade: the primary key serves every lookup they served.

The 33 models are corrected too, so the next hand-written migration is not
comparing against a schema that would want them back.

Counted honestly: 41, not the 102 an earlier pass reported — that number included
the 61 primary keys themselves.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013dKUV96WkQQETZBMoAG98C
2026-09-10 21:24:43 +05:30
Yogesh OjhaandClaude Opus 5 048abbf3f3 fix(api): a long name crashed the request, and a blank one was accepted
Naming an engine or a scan context with more than 200 characters reached
Postgres and came back as a 500 — the column is bounded, the request
model was not. A name of spaces was accepted outright by engines,
contexts, schedules and projects, and a project created that way got an
empty slug, which is the segment its own URL is built from.

Proxies and notification channels already ran their names through
clean_name; every other named entity now does the same, on update as
well as on create, where none of them did. A rejected name says whether
it was empty or too long instead of failing at the database.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K6LcH8uUddjJbKrahf34ys
2026-09-07 00:12:56 +05:30
Yogesh Ojha 69239333bc Refactor validation and DNS tools, implementation of onboarding wizard
- Added `clean_name` function to `validation.py` for name validation with max length.
- Simplified docstrings in `dnsx` client, parser, and service files for clarity.
- Enhanced error handling in `ViewDNSService` to prevent disabling keys on rate limits.
- Improved WHOIS service to handle concurrent inserts gracefully with integrity checks.
- Updated `Settings` class in worker config to inherit from `BaseAppSettings`.
- Added new dependencies in `pyproject.toml` for `apprise` and `cryptography`.
2026-06-21 21:56:20 +05:30