Commit Graph
5 Commits
Author SHA1 Message Date
Yogesh OjhaandClaude Opus 5 06fa55824d perf(docker): build the worker image once and keep the go cache out of it
Four compose services declared their own build against worker/Dockerfile, so
buildkit shared the solve and then exported and unpacked the same image once per
tag. Compose does not dedupe identical targets, measured on a three-service
probe. Under the containerd snapshotter the unpack is a second full pass over
every byte, as expensive as the export.

The go module and build caches were baked into layers: `go install nuclei` made
a 2.66GB layer for a 190MB binary, and /root/go/pkg/mod (3.4GB) plus
/root/.cache/go-build (3.3GB) were half the 13.1GB image.

- the tools build in a throwaway stage with --mount=type=cache on GOCACHE and
  GOMODCACHE; the runtime stage copies /out/bin alone
- worker-default builds rengine-worker:latest; worker-beat, ct-stream and
  flower reference it. Building one of those alone now needs worker-default
  built first
- the runtime stage drops the go toolchain, build-essential and libpcap-dev;
  uv sync needs no compiler. libpcap0.8t64 stays, naabu dlopens it for SYN so
  ldd shows nothing and the loss would only surface at scan time
- -ldflags=-s trims the binaries 650MB -> 495MB
- only shared/ and tools/ are copied, being file:// path deps; stages/,
  reports/, interest/, toolbox/, mcp/, connectors/ and binaries/ are mounted
- the frontend fetches tech icons above COPY frontend/. ., so a source edit no
  longer re-runs the tarball download
- .dockerignore excludes .git (271MB) and node_modules (320MB); COPY frontend/.
  . had been copying the host's node_modules over the npm ci result

worker image 13.1GB -> 2.41GB, frontend 1.38GB -> 969MB
worker export+unpack 453s across three tags -> 55s across one
editing stages/ or reports/ now rebuilds nothing (1s), frontend/src 4s,
shared/ 19s; cold build 475s -> 414s, still dominated by go compilation

Tool versions diff identical against the old image, naabu SYN proven against a
live listener, 8/8 services up, 853 tests green, ruff clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015fWjAbvFBdmsmVeMcwaaR9
2026-09-13 19:00:40 +05:30
Yogesh OjhaandClaude Opus 5 0b853e04b6 chore: production-readiness review across the tree
A full static review of the 3.0 tree, run as two fan-out passes with every
finding verified against the running stack rather than taken on inspection.

Correctness and security
- instance settings encrypted the mask over the stored AI key: `!= MASK` never
  matched `••••••••abcd`, so saving settings destroyed the key and `test_ai`
  posted the mask as a bearer token
- the MFA token was replayable for its full five-minute life; the gate now
  checks revocation and consumes the token after a successful exchange
- logout revoked nothing for a bearer client while answering "Logged out"
- `POST /api-keys/{id}/test` returned the plaintext key inside httpx's error
  string, which embeds the whole URL
- an interest rule could be edited or deleted through another project's id
- redis ran the celery broker under `allkeys-lru`, free to evict a queued task
  or a chord counter; now `noeviction`
- nuclei's stored command carried the OAST token and every auth header
- a finalize step could poison the session and leave a scan RUNNING for ever
- `is:issue` raised TypeError on every search
- offset paging repeated and skipped rows in four dimensions
- a cancelled run kept a stale `scans.*_found` rollup
- every PDF report raised `KeyError: 'software'`
- `version_key` truncated a wide numeric run and inverted release ordering
- psycopg raises a NUL as a bare ValueError, above StatementError, so every
  savepoint guard was blind to the case it was written for

Performance
- endpoints.host had no trigram index: 35.9ms seq scan -> 0.389ms
- free text compiled to one OR, not the prescribed UNION ALL: 132ms -> 41ms
- `/http-assets/summary` and `/ips/summary` materialised whole tables
- the cross-link `ip` kind ran an estate query per page and drew nothing

Frontend
- `$effect` loaders that read the state they write, an unbounded request loop
  on a failing endpoint
- error states that rendered as empty states
- 2FA backup codes were unusable while the copy said they worked
- a "Scan history" dialog that always reported none
- ~400 duplicated lines across five filter bars, now one component

Build and tooling
- every scanner installed at @latest; all 13 pinned to the measured versions
- CI built `web/`, gone since 2.x, with tags that cannot be valid
- the prettier pre-commit hook discarded its filenames and always passed
- the test suite destroyed itself when run twice

Also: subdomains from public code, via a pinned github-subdomains source and a
GitHub API key that subfinder receives too; sublist3r and oneforall removed
after measuring one unique host in 89 across four targets.

Verified: 853 tests, ruff clean over eleven packages, svelte-check 0 errors,
the count-is-a-promise gate at 131 checks and 0 mismatches, and 143 GET routes
with no 5xx. Four migrations, each with an exercised downgrade.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DVkdx5ynZ6aof5kLFczmhX
2026-09-13 18:03:29 +05:30
Yogesh Ojha 1ff04e641e feat: update Docker configuration to fetch tech icons and adjust static volume 2026-09-03 07:37:11 +05:30
Yogesh Ojha 5c2a0b40fa refactor frontend Docker setup: adjust build context and add version handling 2026-01-26 21:13:56 +05:30
Yogesh Ojha 77e93659d1 add frontend service with Docker configuration 2026-01-24 14:38:46 +05:30