- Implemented TargetEnrichmentStage for enriching targets with DNS, WHOIS, and BGP context.
- Added VhostStage for virtual host discovery using host-header fuzzing.
- Introduced WafDetectStage for fingerprinting web application firewalls.
- Created configuration files for each new stage with relevant parameters.
- Updated Dockerfile to copy stages instead of engines.
- Enhanced existing tools and services for better integration with new stages.
- Refactored task management in Celery to accommodate new stages and improve error handling.
Rebuild the scan detail view into a tabbed results page: Overview
(tech/status/web-server distributions, TLS health, top networks and
services), Web assets (rich httpx table with screenshots, status,
tech, CDN/WAF/TLS-expired badges + a full per-asset detail drawer),
Subdomains (denormalized HTTP summary), Infrastructure (IP-centric
cards correlating open ports and the hosts resolving to each IP), and
Pipeline. Assets correlate by value within the scan (subdomain.name /
http_asset.host / ip), so one IP card surfaces every co-located host.
API: add per-scan http-assets, ports and ips list+summary endpoints
plus services mirroring the subdomains pattern, and serve screenshots
through an authenticated /media/screenshot route; mount scan_media
into the api container so it can read worker-written captures.
Fix two serializers that silently dropped fields: ScanRead omitted
http_assets_found (scan detail and list 500'd), and the subdomain
serializer never emitted the denormalized HTTP summary it stores.
Drop leftover docstrings and banner comments from the httpx client,
parser and field-map.
Rounds out expansion (host discovery, virtual hosts, screenshots, WAF):
- host_discovery: naabu liveness sweep marks IpAddress.is_alive for
ip_range/asn; port_scan then deep-scans only live hosts
- vhost: ffuf Host-header bruteforce (domain, opt-in) -> vhost-only
Subdomains that http_probe then picks up
- screenshot: httpx headless chromium -> HttpAsset.screenshot_path under
a scan_media volume
- waf_detect: wafw00f fingerprints WAFs -> HttpAsset.waf
All config-driven (ExpansionConfig) with proxy / user-agent / headers +
per-tool custom args threaded through. Adds ffuf, chromium and wafw00f to
the worker image and a scan_media volume.
- Added new types and interfaces for scan scheduling in the frontend.
- Created a new Svelte page for managing scheduled scans, including listing, creating, editing, and deleting schedules.
- Implemented backend models and services for scan schedules, including handling different schedule types (one-off, interval, daily, cron).
- Introduced a Celery task to manage the execution of scheduled scans based on their defined timing.
- Enhanced notification system to include alerts for newly discovered subdomains during scans.
- Added utility functions for handling time zones and scheduling logic.
- Updated the default route for automation to redirect to engines instead of scans.
- Removed the scans page and replaced it with a new scans overview page that lists scan history for projects.
- Introduced a detailed scan view page to display individual scan results and subdomain discoveries.
- Added new enums for scan statuses and subdomain sources to improve code clarity and maintainability.
- Implemented a new scan task in Celery to handle scan execution and logging.
- Enhanced the database models for scans and subdomains to support new features and improve data integrity.
- Updated the worker Dockerfile to include necessary scanning tools and dependencies.
- Added utility functions for domain matching and pattern compilation to streamline scan operations.
- Updated project slug generation to ensure uniqueness by checking active projects.
- Modified project retrieval functions to only fetch active projects.
- Introduced unique slug generation for tags associated with specific projects.
- Enhanced user authentication flow with improved error handling and session management.
- Added session expiration handling in the API client and SSE client for better user experience.
- Improved UI components for better accessibility and user feedback, including copy buttons and activity drawers.
- Refactored notification handling to ensure proper session management and state reset on logout.
- Removed unused enrichment components to streamline the codebase.
- Updated notification model to simplify JSONB handling and ensure proper expiration logic.
- Added migration commands to the new dev-README.md for creating and rolling back migrations.
- Updated docker-compose.yml to include additional volume mounts for alembic and build artifacts in development and worker services.
- Enhanced Target model in target.py to include a relationship with WhoisRecord and added WhoisRecordSummary to TargetRead for better data representation.
- Introduced a new endpoint to validate multiple target values in a single request.
- Added a constant for maximum target imports.
- Updated the target validation response model to include the target value.
- Modified Dockerfile CMD for graceful shutdown and reload directory.
- Commented out environment variable in docker-compose for polling.
- Update Dockerfile to copy shared directory and dependency files
- Create new README.md file
- Refactor database session management and security imports
- Implement user authentication routes and user management endpoints
- Add configuration settings for application and database
- Set up logging configuration for backend services
- Update docker-compose for improved service management
* Add django debug toolbar
* Remove duplicate debug toolbar URLs
* Remove orphaned import in startScan
* Add DEBUG config commented in entrypoint
* Install debug toolbar into celery beat
* Create dev docker compose & entrypoints
* Move django toolbar install to dev entrypoints
* Install debug_toolbar only when debug env var is set
* Add dev command to MakeFile
* Use DEBUG from settings.py exclusively
* Add remote debug configuration in dev env
* Add VSCode debug configuration
* Celery log level to 1 when debug is on
* Remove hard coded celery debug port
* Add gunicorn in the production setup
* Add export var for celery log level
* Restrict debug port to 127.0.0.1
* Set remote debug to false by default
* Update doc on debug step for submitting issue
* Create dev docker compose & entrypoints
* Add django debug toolbar
* Remove duplicate debug toolbar URLs
* Remove orphaned import in startScan
* Add DEBUG config commented in entrypoint
* Install debug toolbar into celery beat
* Move django toolbar install to dev entrypoints
* Install debug_toolbar only when debug env var is set
* Add dev command to MakeFile
* Use DEBUG from settings.py exclusively
* Add remote debug configuration in dev env
* Add VSCode debug configuration
* Celery log level to 1 when debug is on
* Remove hard coded celery debug port
* Add gunicorn in the production setup
* Add export var for celery log level
* Restrict debug port to 127.0.0.1
* Set remote debug to false by default
* Update doc on debug step for submitting issue
---------
Co-authored-by: Raynald <raynald@easi-services.fr>
Co-authored-by: Psyray <psyray@users.noreply.github.com>
Yogesh Ojha