Commit Graph
12 Commits
Author SHA1 Message Date
Yogesh Ojha 69239333bc Refactor validation and DNS tools, implementation of onboarding wizard
- Added `clean_name` function to `validation.py` for name validation with max length.
- Simplified docstrings in `dnsx` client, parser, and service files for clarity.
- Enhanced error handling in `ViewDNSService` to prevent disabling keys on rate limits.
- Improved WHOIS service to handle concurrent inserts gracefully with integrity checks.
- Updated `Settings` class in worker config to inherit from `BaseAppSettings`.
- Added new dependencies in `pyproject.toml` for `apprise` and `cryptography`.
2026-06-21 21:56:20 +05:30
Yogesh OjhaandClaude Opus 4.8 73f460999f feat(automation): scan engines, scan contexts, and engine×context launch
Scan Engine — SvelteFlow capability-flow builder (Discovery→Expansion→Depth, 14 capabilities, data-lineage edges, neutral phase bands, Recon Data terminal), Canvas/YAML dual view, CRUD + duplicate + YAML import/export; backend SQLModel + service + /engines API + migration.

Scan Context — runtime auth/scope/rate profiles (6 auth types resolving to headers server-side; secrets plaintext-at-rest but masked in every read), exclusions/inclusions, per-tool + global rate overrides, thread/timeout multipliers, protocol/redirects; full-page shadcn form + list; CRUD + duplicate; merge-ready. Baseline-compare/new-assets deferred (422).

Launch — merge_engine_context() resolves engine×context per Section 7 (headers union/context-wins, rate ceiling applied after per-tool replace, threads/timeouts ×multiplier, exclusions additive, tool-enable engine-only) into a secret-safe ResolvedScanConfig; /scans/preview dry-run + /scans launch creating a pending Scan record (engine/context name snapshots, execution_config persisted); /automation/scans page + launch modal + execution preview. Worker dispatch stubbed.

Security/QA (multi-agent adversarial review): no secret value in any API body (preview/create/list/read mask ALL header values unconditionally); same-project scoping (404 cross-project); CRLF/control-char rejection on auth + headers + scope lists; rate/thread/timeout clamps; list size caps; ScanCreate extra=forbid; engine deletion preserves scan history (no FK on scans.engine_id). Official shadcn components throughout; monochrome.

Migrations: add_scan_engines → add_scan_contexts → add_scans → drop_scan_engine_fk (applied, JSONB configs).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-31 19:57:04 +05:30
Yogesh Ojha 53a50660f5 Add migration commands to README, update docker-compose for volume mounts, and enhance Target model with WhoisRecord relationship
- Added migration commands to the new dev-README.md for creating and rolling back migrations.
- Updated docker-compose.yml to include additional volume mounts for alembic and build artifacts in development and worker services.
- Enhanced Target model in target.py to include a relationship with WhoisRecord and added WhoisRecordSummary to TargetRead for better data representation.
2026-02-07 21:37:56 +05:30
Yogesh Ojha 7a6f3f8c16 change project structure, add whois celery worker, made redis notif handler sync 2026-02-07 20:42:40 +05:30
Yogesh Ojha 4792f70305 Implement WHOIS service with RDAP provider integration
- Added models for WHOIS responses including WhoisDomainResponse, WhoisIPResponse, and WhoisASNResponse.
- Created a parser for WHOIS data to convert raw responses into structured models.
- Developed an RDAPProvider class to handle RDAP lookups using the whoisit library.
- Implemented a WhoisService class for performing WHOIS lookups with optional database caching.
- Added methods for finding correlations between WHOIS records based on registrant, registrar, network, country, and nameservers.
- Updated Dockerfile to include tools directory and added rengine-tools dependency in pyproject.toml.
2026-02-07 10:43:08 +05:30
Yogesh Ojha 24b6c1ca53 refactor pre-commit configuration and update project descriptions 2026-02-05 09:04:46 +05:30
Yogesh Ojha ecc772eea9 add CSV import functionality for targets and update dependencies 2026-02-05 08:58:49 +05:30
Yogesh Ojha 634926aa4a add celery health endpoints in api 2026-02-04 20:52:22 +05:30
Yogesh Ojha f7c6429d5d add celery 2026-02-04 20:27:02 +05:30
Yogesh Ojha 25df5b62f5 change project structure 2026-02-04 20:05:28 +05:30
Yogesh Ojha f5667e06d8 init fastapi 2026-01-24 11:41:11 +05:30
Yogesh Ojha 13632c0b5d init fastapi 2026-01-24 11:12:43 +05:30