Commit Graph
14 Commits
Author SHA1 Message Date
Yogesh OjhaandClaude Opus 5 0522444245 feat(reports): white-labelling, and a renderer that cannot be made to fetch
Uploaded typefaces and instance-wide branding defaults, which together make a
report genuinely white-labellable: a client's own font, a logo set once, and a
theme that names both.

The security half came first, because the logo field was a live SSRF. It was a
free-text string rendered into an <img src>, and WeasyPrint would have fetched
whatever it pointed at from inside the worker. Nothing fetches now, at four
layers. The load-bearing one is a WeasyPrint url_fetcher that allows data: URIs
and files under three known roots and refuses everything else; it was checked
against cloud metadata, internal https, loopback, /etc/passwd, a UNC host, ftp
and protocol-relative references. Above it sit three input guards: a logo must
be embedded data of a known image type, an uploaded theme may not use @import or
a url() that is not data:, and authored markdown loses any image that is not
embedded. A new feature that lets a report name a resource goes through the
fetcher, not around it.

A typeface is bytes, never a link. The family row records files named by us, so
a hostile filename never reaches disk, and the family directory refuses anything
resolving outside itself. A face is accepted only when the bytes agree: the
signature is sniffed and then fontTools opens it and requires outlines, which is
why fonttools is now an api dependency rather than only a WeasyPrint one. A
5-byte file with a valid TTF header and a 4 KB file of padding are both refused.
The family name is banned from carrying quotes or braces because it is written
into a quoted CSS font-family string.

The standalone HTML export now embeds only the families the theme actually
names, which halves it, and no longer carries file:// references that would have
been dead outside the container.

Instance defaults fill only what a template leaves blank, so setting a logo once
never overrides a per-client deliverable that set its own.

Verified: a real TTF family uploaded and embedded in a 53-page PDF under an
uploaded theme with an uploaded logo, and every probe above refused.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017f2Mf2HGwFaTRnui4UxpRd
2026-09-06 12:53:18 +05:30
Yogesh OjhaandClaude Opus 5 424e6d6e23 feat(reports): a document engine, not an export
reNgine 2 could produce a PDF. This produces a deliverable, and it is built
the way scan engines are built: sections are plug-and-play directories under
reports/, each with a pydantic config model that IS the source of truth. That
model feeds GET /reports/catalog, the builder renders its own form from it, and
the API validates against it. Adding a section is a directory. Scan Engine is to
stages what Report Template is to sections.

WeasyPrint 69 renders it, chosen over Chromium and Typst after measuring what
each can actually do. Only real CSS Paged Media gives a table of contents with
true page numbers, running headers, a PDF outline and footnotes; SysReptor
reaches the same verdict and pipes Chromium's HTML into WeasyPrint for exactly
that reason. reNgine drops the Chromium hop because it templates in Jinja and
draws its charts as SVG in Python, so a browser buys nothing and costs 400 MB
per report. Typst was rejected because a user-uploadable theme would then have
to be Typst code rather than CSS.

A theme is data. shared/definitions/report_theme.py is the schema, six themes
ship as YAML, and a user uploads their own; there is no markup in a theme, so
there is no injection surface. Fonts are vendored so an offline instance still
renders in Latin, Greek and Cyrillic.

Everything the document claims is computed once, in reports/analysis/engine.py:
a posture score whose every deduction is named, weaknesses ranked by the signals
behind the rank rather than severity alone, attack paths that chain observations
into a route, a remediation plan ordered by risk removed, per-dimension change
against the previous run, and caveats that state what was never assessed. Two
narrators read that one brief. Without AI the report still argues its case; with
AI the prose is better and the model never sees a row, only the brief, capped and
cached per check. An AI failure falls back and the report still completes.

Generation is a job. POST /reports queues, the worker walks progress over SSE,
and four formats come out of one render: PDF, self-contained HTML, Markdown and
JSON. The estimate reads each section's own limits, so the page count it promises
is close to the one it delivers.

AI moves out of Settings into its own top-level page: connection, a model per
tier, what it may be used for, what it has cost, and the narrative cache.

Verified on a 1,379-host gov.cy scan: 124 pages in 19.5s across 19 sections,
all six themes, and a broken key still producing a complete report.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017f2Mf2HGwFaTRnui4UxpRd
2026-09-06 11:45:51 +05:30
Yogesh Ojha 69239333bc Refactor validation and DNS tools, implementation of onboarding wizard
- Added `clean_name` function to `validation.py` for name validation with max length.
- Simplified docstrings in `dnsx` client, parser, and service files for clarity.
- Enhanced error handling in `ViewDNSService` to prevent disabling keys on rate limits.
- Improved WHOIS service to handle concurrent inserts gracefully with integrity checks.
- Updated `Settings` class in worker config to inherit from `BaseAppSettings`.
- Added new dependencies in `pyproject.toml` for `apprise` and `cryptography`.
2026-06-21 21:56:20 +05:30
Yogesh OjhaandClaude Opus 4.8 73f460999f feat(automation): scan engines, scan contexts, and engine×context launch
Scan Engine — SvelteFlow capability-flow builder (Discovery→Expansion→Depth, 14 capabilities, data-lineage edges, neutral phase bands, Recon Data terminal), Canvas/YAML dual view, CRUD + duplicate + YAML import/export; backend SQLModel + service + /engines API + migration.

Scan Context — runtime auth/scope/rate profiles (6 auth types resolving to headers server-side; secrets plaintext-at-rest but masked in every read), exclusions/inclusions, per-tool + global rate overrides, thread/timeout multipliers, protocol/redirects; full-page shadcn form + list; CRUD + duplicate; merge-ready. Baseline-compare/new-assets deferred (422).

Launch — merge_engine_context() resolves engine×context per Section 7 (headers union/context-wins, rate ceiling applied after per-tool replace, threads/timeouts ×multiplier, exclusions additive, tool-enable engine-only) into a secret-safe ResolvedScanConfig; /scans/preview dry-run + /scans launch creating a pending Scan record (engine/context name snapshots, execution_config persisted); /automation/scans page + launch modal + execution preview. Worker dispatch stubbed.

Security/QA (multi-agent adversarial review): no secret value in any API body (preview/create/list/read mask ALL header values unconditionally); same-project scoping (404 cross-project); CRLF/control-char rejection on auth + headers + scope lists; rate/thread/timeout clamps; list size caps; ScanCreate extra=forbid; engine deletion preserves scan history (no FK on scans.engine_id). Official shadcn components throughout; monochrome.

Migrations: add_scan_engines → add_scan_contexts → add_scans → drop_scan_engine_fk (applied, JSONB configs).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-31 19:57:04 +05:30
Yogesh Ojha 53a50660f5 Add migration commands to README, update docker-compose for volume mounts, and enhance Target model with WhoisRecord relationship
- Added migration commands to the new dev-README.md for creating and rolling back migrations.
- Updated docker-compose.yml to include additional volume mounts for alembic and build artifacts in development and worker services.
- Enhanced Target model in target.py to include a relationship with WhoisRecord and added WhoisRecordSummary to TargetRead for better data representation.
2026-02-07 21:37:56 +05:30
Yogesh Ojha 7a6f3f8c16 change project structure, add whois celery worker, made redis notif handler sync 2026-02-07 20:42:40 +05:30
Yogesh Ojha 4792f70305 Implement WHOIS service with RDAP provider integration
- Added models for WHOIS responses including WhoisDomainResponse, WhoisIPResponse, and WhoisASNResponse.
- Created a parser for WHOIS data to convert raw responses into structured models.
- Developed an RDAPProvider class to handle RDAP lookups using the whoisit library.
- Implemented a WhoisService class for performing WHOIS lookups with optional database caching.
- Added methods for finding correlations between WHOIS records based on registrant, registrar, network, country, and nameservers.
- Updated Dockerfile to include tools directory and added rengine-tools dependency in pyproject.toml.
2026-02-07 10:43:08 +05:30
Yogesh Ojha 24b6c1ca53 refactor pre-commit configuration and update project descriptions 2026-02-05 09:04:46 +05:30
Yogesh Ojha ecc772eea9 add CSV import functionality for targets and update dependencies 2026-02-05 08:58:49 +05:30
Yogesh Ojha 634926aa4a add celery health endpoints in api 2026-02-04 20:52:22 +05:30
Yogesh Ojha f7c6429d5d add celery 2026-02-04 20:27:02 +05:30
Yogesh Ojha 25df5b62f5 change project structure 2026-02-04 20:05:28 +05:30
Yogesh Ojha f5667e06d8 init fastapi 2026-01-24 11:41:11 +05:30
Yogesh Ojha 13632c0b5d init fastapi 2026-01-24 11:12:43 +05:30