The wordlist field was a free-text absolute path, so the only lists anyone could
actually use were the two baked into the image. Changing that meant a bind mount or a
docker cp — a sysadmin action, not a product one — which in practice meant every
install guessed from the same list. Bruteforce quality is almost entirely wordlist
quality, so the knob was there and nobody could reach it.
Wordlists are now a library, on the same shape as the nuclei check library. Shipped
lists stay in the image and are indexed on read so the picker never starts empty;
uploads land under a `wordlists` volume shared by api and worker. A config field holds
a slug, never a path — `stages/config.py:wordlist(kind)` declares one the way
`threads()` and `rate()` declare theirs, and carries the kind the picker filters on so
a DNS stage is never offered a list of URL paths. `lookup()` still accepts the
absolute paths older engines stored, so an existing engine keeps working.
The UI is the Arsenal's second tab, which is why the Arsenal is now "what your scans
run with" rather than only scanners and their checks: upload, filter by what the words
are, preview the first 200 in the order a scan will read them, delete an upload. Every
list shows the name an engine refers to it by, because that is the thing you type into
a YAML document.
`resolve_path()` refuses anything resolving outside its origin's root and
`delete_custom()` refuses a shipped list — the guard the template library learned the
hard way. A stage that cannot find its list says so and reports PARTIAL rather than
quietly guessing from a different one; that is how the shared volume being absent was
caught during the end-to-end run.
Verified end to end: a seven-word list uploaded through the API, selected by slug,
every passive source disabled — the scan returned six hosts, all attributed to
bruteforce.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DPueZXQMbjLXMokXeA8LFP
A scan no longer needs a saved engine. The launch dialog has two
configuration modes: Scan Engine (default, runs a saved engine exactly
as configured, shown as a pipeline and its YAML) and Custom Scan (pick
the capabilities to run; dependent and supporting stages are included
automatically). Targets are created on launch from typed values, the
scan context is a first-class field, and "Run again" restores the
engine or the selection that ran.
Backend
- scans.engine_id is nullable; an absent engine resolves through
AdHocEngine and the run is named after its capabilities (plan_label)
- ScanCreate/ScanBatchCreate accept target_value(s) (create or reuse via
TargetService.ensure_targets), a per-run intensity, and preview works
for a target that does not exist yet
- every stage declares consumes/produces (AssetKind), group (StageGroup)
and role (capability | support); the registry validates them and the
catalog serves them with groups and seed_produces
- stage_effects walks levels so a producer only feeds later levels and
marks an unfed consumer skipped_no_input
Frontend
- components/scans/launch/: launch-dialog, launch-state, target-picker,
plan-picker (segmented Scan Engine | Custom Scan, pipeline + YAML
Config views, capability pills with full per-stage settings),
stage-pill, stage-config-popover, launch-summary + what-runs,
launch-context-field, launch-context-form, save-engine-dialog
- utilities/launch-plan.ts resolves implied producers and following
support stages; the last engine and last custom selection persist
- quick-scan-fields offers presets and engines in one select; the
command palette offers "Scan <value>"; scan list action is "Run again"
- effect-panel drops its explanatory header; NO_CONTEXT_LABEL is
"No scan context"; launch-modal and execution-preview are removed
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014erb6Jbm6v9sxPgYFBxB7E
The unit of exposure is now a service — (ip, port) — not a bare open port.
Pipeline (EXPANSION): L0 produce hosts, L1 cdn_check + passive_ports,
L2 port_scan, L3 http_probe, L4 screenshot/waf_detect/service_fingerprint.
- cdn_check applies to every target type and records cdn_type, so a domain
scan finally knows which addresses are CDN-fronted. cdn/waf is an edge you
do not own (edge ports only); cloud is your own machine (scanned in full).
- port_scan decides and records a per-address scan_policy (full/web/skip)
honouring the scan context's excluded_ips, and offers port profiles,
exclusions, scan type, a tarpit threshold and an address budget.
- passive_ports reads Shodan internetdb through naabu -passive: keyless,
touches_target=False, so a passive-intensity scan still gets exposure data.
- http_probe seeds its host x port matrix from open ports and folds every
live response back onto the port (is_http, tls, product).
- service_fingerprint names the software behind non-web ports with a pure
Python banner prober (tools/banner/), including a SOCKS5 CONNECT so it
honours the context proxy. naabu -sV is unimplemented.
- shared/definitions/ports.py is the one vocabulary; ip_inventory and
port_inventory are the only writers of ip_addresses/ports.
- open_ports_found is measured from the table (DERIVED_COUNTS): four stages
write ports, so summing stage results double-counted.
- naabu failures no longer report zero open ports as success.
Surfaces: a Services tab (SERVICE_QUERY grammar, 16 fields, 18 counted
leads, 9 group dimensions, detail sheet) built on the shared results shell,
and an outlier-first Exposure panel on the Overview. Every count is a
promise — leads, groups and panel pivots were verified to equal the row
count they land on.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BnC2xMMzV6WQnpDp5zcXoU
- Updated the engine management UI in `+page.svelte` to improve user experience with new controls for filtering stages, toggling side panels, and managing YAML configurations.
- Introduced new components such as `EngineSummaryBar` and `Resizable` for better layout handling.
- Added clipboard functionality for copying YAML configurations and formatting options.
- Enhanced the backend models in `scan_engine.py` to include a new `PreviewResolved` class for better handling of scan context and parameters.
- Updated presets in `presets.py` to clarify descriptions and improve user understanding of stage settings.
- Implemented TargetEnrichmentStage for enriching targets with DNS, WHOIS, and BGP context.
- Added VhostStage for virtual host discovery using host-header fuzzing.
- Introduced WafDetectStage for fingerprinting web application firewalls.
- Created configuration files for each new stage with relevant parameters.
- Updated Dockerfile to copy stages instead of engines.
- Enhanced existing tools and services for better integration with new stages.
- Refactored task management in Celery to accommodate new stages and improve error handling.
- Add aggregate functions for scan status and counts in orchestrator.
- Create ScanEventPublisher for emitting scan-related events.
- Introduce ScanActivityService for managing scan activities.
- Implement ScanCommandRecorder for logging command executions.
- Develop canvas building and finalization logic for scan tasks.
- Refactor scan task to utilize orchestrator functions for running stages and finalizing scans.
- Enhance scan notification system to dispatch relevant events.
- Integrate command recording into CLI tool execution for better tracking.
- Updated the default route for automation to redirect to engines instead of scans.
- Removed the scans page and replaced it with a new scans overview page that lists scan history for projects.
- Introduced a detailed scan view page to display individual scan results and subdomain discoveries.
- Added new enums for scan statuses and subdomain sources to improve code clarity and maintainability.
- Implemented a new scan task in Celery to handle scan execution and logging.
- Enhanced the database models for scans and subdomains to support new features and improve data integrity.
- Updated the worker Dockerfile to include necessary scanning tools and dependencies.
- Added utility functions for domain matching and pattern compilation to streamline scan operations.
