Fixes four reported issues.
#321 / #322 — POST /osm/api/runs was unusable when the server was started
with --settings-file. That branch in pkg/cli/root.go loaded the settings via
config.LoadFromFile (a plain YAML unmarshal) and never called ResolvePaths(),
so every derived runtime path stayed empty. An empty WorkflowsPath made
workflow lookups scan "" (reported as "Workflow not found") and made a flow's
relative module refs resolve against the process working directory.
- root.go now resolves paths (and applies env overrides) on that branch
- ResolvePaths backfills any environments.* key a partial settings file omits,
sourced from the new defaultEnvironments() so DefaultConfig and the backfill
can no longer drift
- parser.ErrWorkflowsDirNotConfigured replaces the silent cwd fallback, so all
loader call sites report the misconfiguration instead of a misleading 404
#323 — a run submitted with run_mode:distributed could be lost permanently.
BRPOP is at-most-once: a task popped but not yet recorded in osm:tasks:running
existed nowhere, and the master's recovery sweep only reads that hash.
- workers now claim via BLMOVE onto osm:tasks:processing:{worker_id}
- the claim is acked only once SetTaskRunning succeeds; a failure requeues
instead of executing the task untracked
- recovery on worker startup, on dead workers, and for processing lists left
by workers that are no longer registered
- requires Redis 6.2+
#320 — settings values can now be overridden by OSM_* environment variables so
secrets need not live in osm-settings.yaml. The mapping is derived from the YAML
tags by reflection, so new settings are overridable with no extra code. Applied
in config.Load, hotreload and the --settings-file branch, but deliberately not
in LoadFromFile: `osmedeus config set` round-trips through it and writes back,
which would persist env secrets to disk.
Known gaps: pkg/cli/worker_queue.go still consumes the pending queue with the
at-most-once PopTask, and two distributed e2e tests (TaskSubmission,
FullWorkflow) fail on main independently of these changes.
Org (tenant) layer
- New Org model with org_uuid denormalized onto workspaces, assets,
vulnerabilities and runs so cross-workspace queries need no join
- Automatic attribution via BeforeAppendModel hooks; importers stay org-unaware
- Read semantics: empty org means no filter (backward compatible)
Write semantics: empty org coerced to the default org
- Migration backfills every pre-existing row into the default org
- CLI: osmedeus org create/show/assign/use/rename/delete
- API: /osm/api/orgs CRUD plus ?org= on assets, vulns, runs and workspaces
npm distribution
- npm install -g @j3ssie/osmedeus ships the Go binary through npm
- One npm name with version-suffixed platform builds pulled in as aliased
optionalDependencies, so an install downloads exactly one binary
- Binary ships gzipped and is decompressed on first run into a
version-scoped cache, so an upgrade can never exec a stale binary
- make bump-version is the single source of truth for the version constant
Bundled agent skills
- public/skills/ embedded in the binary, installed via osmedeus skills install
- Filesystem-driven discovery: a new bundle needs no code change
- make sync-skills mirrors bundles out to the standalone skills repo
Platform sub-projects
- Vendor dashboard, registry and workflow under platform/ so they version
with the engine they talk to; make sync-platform publishes them out
- Rebuild the embedded UI in public/ui/
Follow-up cleanup to 94ddad4, no behaviour change.
- export installer.IsGitHubURL and drop pkg/cli's isGitHubURLForFetch, which
was a third substring copy of the same predicate still gating an
Authorization header. internal/functions keeps its own copy: that one also
has to match the SSH form (git@github.com:user/repo.git), which has no
parseable host.
- replace the isTrustedRegistry one-line predicate with the comparison it
wrapped, and extract the fallback-URL rule the two mode handlers had
copy-pasted into displayRegistryURL, so registryPathOrURL is no longer
overwritten mid-function with a different meaning
- drop a no-op int64 conversion in the registry size check
- Emit the raw `threads` value in step command summaries so template variables
like `{{deparosParallel}}` survive until templateEngine.Render resolves them,
instead of silently collapsing to 1 when Int() fails to parse the template
- Only print the Artifacts section in `printResultSummary` when -v/--verbose
is set, removing state/log file clutter from default run output
- Update embedded UI build artifacts under public/ui/_next to the latest
dashboard build (chunk renames, manifest updates, page txt/html refreshes)
- Reorder Migrate() to add columns to existing tables BEFORE creating
indexes, since CREATE TABLE IF NOT EXISTS is a no-op on existing
databases and some indexes (e.g. finding_hash) reference migrated
columns that would otherwise be missing
- Add explanatory comment in internal/database/database.go documenting
why column migrations must precede index creation
- Surface a helpful CLI hint in pkg/cli/root.go pointing users to
'osmedeus db migrate' when startup fails with 'failed to run
migrations', so stale schemas are easier to recover from
- Add db_import_vigolium JS function that routes vigolium JSONL records
by envelope type: http_record -> assets, finding -> vulnerabilities
(deduped on new finding_hash column), skipping scan/oast_interaction
- Add FindingHash field to Vulnerability model with idempotent ALTER
TABLE migration and matching index
- Suppress run errors when using --silent --empty-target placeholder mode
- Retry transient fetchURLContent failures (network errors, 408/429/5xx)
with exponential backoff; leave 4xx untouched so GitHub auth fallback
can engage
- Add unit tests for the vigolium importer (import + idempotency) and
the HTTP retry behavior, plus a vigolium juice-shop sample fixture
- Bump katana, naabu, kingfisher, bearer registry entries and add
vigolium to the direct-fetch registry
- Track tmux sessions in ActiveRun and kill them on run cancel via new
TmuxHooks indirection; expose killed_tmux_sessions in CLI and API responses
- Add ExecuteSSHCommand with remote pidfile + process-group kill watcher so
cancelling a run actually terminates remote scans (not just the local session)
- Route ssh_exec/ssh_rsync/sync_* through the run's cancellable context via
new RunContextHooks
- Switch docker-publish to sequential per-arch buildx builds + imagetools
manifest to avoid OOM on multi-arch builds; add docker-buildx-setup target
- Cross-compile Dockerfile via BUILDPLATFORM/TARGETOS/TARGETARCH and retry
SAST binary installs to survive QEMU-flaky downloads
- Bump version to v5.0.3
Workers now propagate RunUUID from the submitted task so the master
can resolve the correct DB run record and associate step results.
Also increments completed_steps on successful insert and connects
the worker to the database for db_import_* functions.
- Extract KillProcessAndChildren into core.types for reuse across CLI and server handlers
- Replace hardcoded 'claude-code' strings with core.DefaultACPAgent constant
- Precompute absolute allowed paths in ACP client to avoid repeated filepath.Abs calls
- Simplify runQuerySteps to delegate to runQueryTable, reducing code duplication
- Refactor agent_chat concurrency guard using sync.Mutex.TryLock for cleaner code
- Use request context for agent timeout instead of background context
- Add query subcommands (vulns, runs, steps) with JSON output and flexible filtering
- Enhance assets command with --where, --search, --value filters (fuzzy matching)
- Expand asset searchable columns (status_code, content_type, title, dns_records, tls, tech)
- Add run status and run cancel subcommands with process termination
- Support control-plane and PID-based cancellation with database updates
- Implement fuzzyFilters in GetTableRecords for case-insensitive substring matching
- Add OSM_IGNORE_REGISTRY env var to skip auto binary installation and health checks
- Update health check to display warning when binary folder is empty but OSM_IGNORE_REGISTRY is set
- Update installRequiredBinaries to skip installation when OSM_IGNORE_REGISTRY=true
- Enhance DownloadFile with retry logic and wget/curl fallback for resilient downloads
- Refactor download function to validate content length and handle transient failures
- Refactor test code to use switch statements instead of if-else chains
- Implement agent-acp step type for spawning external ACP agent subprocesses via Agent Communication Protocol
- Add ACPExecutor with validation, field rendering, and subprocess lifecycle management
- Integrate agent-acp field rendering in StepDispatcher (batch and sequential modes)
- Add run_agent() utility function for workflows to execute ACP agents from steps and JS context
- Add osmedeus agent CLI command for interactive agent execution with --agent, --cwd, --timeout, --stdin, and --list flags
- Add /osm/api/agent/chat/completions REST endpoint with OpenAI-compatible chat format and concurrency control
- Support agent selection via: built-in names (claude-code, codex, opencode, gemini) or custom acp_config.command
- Add step-level configuration: cwd, allowed_paths, acp_config (command, args, env, write_enabled)
- Add comprehensive E2E tests for agent-acp workflows (basic, minimal, config, codex variants)
- Add test workflows in test/testdata/workflows/agent-and-llm/
- Update AGENTS.md documentation with agent-acp examples, CLI usage, and API endpoints
- Add stripTemplateVarsForJS() to convert {{var}} outside quotes to bare JS variable names, enabling direct param/export access in condition expressions
- Add normalizeBoolStringsForJS() to convert string 'true'/'false' to actual booleans for correct JS truthiness evaluation
- Auto-classify asset types in db_import_asset_from_file() and default source to 'web' when empty
- Improve table display with terminal auto-width detection via term.GetSize(), custom column weights, and display name aliases (status_code → status)
- Change default --width to 0 (auto-detect) with weighted column distribution and minimum header-based sizing
- Add comprehensive tests for param-based conditions and JSON asset imports
- Update test data to use direct param references in conditions instead of exports
- Refactor table display using tablewriter with box-drawing characters and proper Unicode width handling
- Add --as-cron flag to register cron schedules instead of executing immediately
- Implement new 'osmedeus assets' command for asset querying with filtering and statistics
- Enhance workflow parameter display with categorization and per-module summaries
- Improve column width handling and parameter documentation
- Add jsonl_rename_key() function to rename keys in JSONL files with mapping syntax 'old1:new1,old2:new2'
- Implements fast JSON parsing with fastjson and efficient buffered I/O for large files
- Include comprehensive test coverage with 7 test cases for edge cases and valid mappings
- Remove unused tablewriter dependency from go.mod/go.sum and internal/terminal/table.go
- Refactor printResultSummary() to use markdown table printing instead of tablewriter
- Add test data file ffuf-result.jsonl for integration testing
- Fix function list width default calculation in function.go
- Add sudo_auth() function for TTY prompts and credential management with optional 4-minute keepalive loop
- Add --sudo-aware CLI flag to detect workflows with sudo commands and offer authentication guidance
- Add sudo step scanner to detect sudo usage across all step types (bash, parallel, foreach) and nested structures
- Add parse_url_file() function to batch-process URLs with format directives, supporting bare IPs and CIDR notation
- Add portscan test data with realistic nmap JSONL samples
- Rename {{Workspace}} to {{TargetSpace}} in function examples for clarity
- Add sudo E2E tests covering tip message, flag acceptance, and non-sudo workflows
- Implement cloud provider infrastructure (DigitalOcean, AWS, GCP, Linode, Azure) with Pulumi integration for distributed scanning
- Add nmap and tmux utility functions for port scanning results processing and long-running background session management
- Introduce webhook-triggered run execution with unique UUID and authentication key support for external integrations
- Add QueuePoller for dual-source (DB + Redis) task polling with concurrent execution
- Add worker queue subcommands (list, new, run) for managing queued tasks
- Add uninstall command to safely remove Osmedeus installation
- Add input validation to prevent command injection via dangerous shell characters
- Add queue run support via --queue and --queue-run flags in osmedeus run
- Add queue polling to server with --no-queue-polling flag to disable it
- Support queuing tasks from both CLI and API with deduplication across sources
- Add worker eval command for distributed function execution with Redis hooks registration
- Add worker set command to update worker fields (alias, public-ip, ssh-enabled, ssh-keys-path)
- Enhance worker status with JSON output, search filtering, and column selection (--columns, --exclude-columns, --search)
- Add --keep-setting flag to install base/validate commands to preserve osm-settings.yaml after base installation
- Fix binary installation in Nix: replace CopyInstalledBinaryToFolder with SymlinkInstalledBinaryToFolder
- Add --clean-ws flag to db clean command for removing workspace data
- Add HooksEnabled field to Run records when creating runs from CLI and API
- Add comprehensive test coverage for hook execution (pre/post hooks, execution order, failure handling)
- Add test coverage for worker commands (eval, set, status with JSON) and db clean operations
- Improve usage documentation for worker subcommands and db operations
- Add Docker Compose distributed-e2e.yaml with Redis, master, and worker services
- Add Make targets: distributed-e2e-up, distributed-e2e-run, distributed-e2e-down
- Replace zap logger with terminal printer for consistent CLI output in master/worker
- Add Redis connection info and task details to startup and lifecycle logs
- Fix DNS records field: use 'dns_records' key with backward compat for 'a' key
- Add optional asset_type and source parameters to db_import_custom_asset
- Update Swagger docs for LLM embeddings schema
- Set run_uuid tracking in worker task execution for distributed runs
- Register distributed database hooks and ensure external binaries in PATH
- Improve terminal formatting with colors for worker/master IDs and log messages
- Add skip() function to terminate remaining steps in current module while continuing to next module, with optional message parameter
- Implement isFuzzyModuleExcluded() for substring-based module filtering in ExecuteFlow
- Add fuzzy_exclude_modules CLI flag (-X) to both run and scan commands for flexible module exclusion
- Handle ErrSkipModule sentinel error throughout executor (executeStep, executeStepsDAG, ExecuteModule, ExecuteFlow) with proper status propagation
- Update function registry and Goja runtime to register skip() function
- Add comprehensive unit tests for skip() behavior, SkipModuleError, and fuzzy module matching
- Update snapshot tests to use generic example.com instead of shopee.vn
- Add replayDBOperations to reinitialize databases from imported workspaces, with proper parameter resolution and pre-condition skipping
- Implement ISO 8601 timestamp format for snapshots (2026-02-13T18-20-34Z) instead of Unix epoch
- Add --include-heavy flag to include large fields (raw_response, screenshot, blob_content) in database queries
- Fix asset table default columns (url, status_code, content_length, title) for better UX
- Skip heavy initialization for lightweight commands (help, version, completion) to avoid ~50MB overhead
- Optimize database function execution via lazy config initialization
- Fix binary installation via Nix to copy already-installed binaries to binaries folder
- Add WorkflowHelp struct with Usage and ExampleTargets for CLI documentation
- Add Optional field to Artifact model and database schema with migration support
- Implement workflow search functionality by name, description, and tags in CLI list command
- Add --usage and --search flags to workflow list command with multiple filtering options
- Display workflow usage info in show command when Help is defined
- Support help inheritance in workflow extends/inheritance resolver
- Update vulnerability counters from database after SARIF imports
- Add comprehensive Help unit tests covering parsing, cloning, and mutation isolation
- Improve test helpers with streaming output, diagnostics, and file validation utilities
- Add fourth general canary test for domain-list-recon flow with artifact validation
- Add is_file, is_dir, is_git functions for type validation
- Add is_compress function to detect compressed file extensions
- Implement pure Go wget with parallel segmented downloads (>1MB files)
- Add git_clone_subfolder with ZIP fallback for GitHub repos
- Add helper utilities: ZIP extraction, directory/file copying
- Add SkipWorkspace option to skip output dirs in empty-target mode
- Fix workflow preference loading order for --module-url flag
- Add 20+ comprehensive tests for all new functions
- Add empty_target preference to allow running workflows without explicit target
- Implement FlowName template variable for module context awareness
- Optimize cdnDelete, SyncUpload, SyncDownload with worker pool concurrency
- Load workflow preferences early for --empty-target CLI flag support
- Add unit tests for preferences and E2E tests for FlowName functionality
- Update linter to recognize ModuleName and FlowName as built-in variables
- Add inline module support to ModuleRef allowing modules to be defined directly in flows without external files
- Implement IsInline() and ToWorkflow() methods to convert inline ModuleRef definitions to executable workflows
- Update ModuleRef.Clone() to properly duplicate all fields including Steps, Runner, RunnerConfig, and Description
- Add field alignment formatting and make path optional for inline modules
- Enhance parser validation to allow omitting path for inline modules while requiring at least one step
- Update executor's preloadModules() and ExecuteFlow() to handle inline modules during flow execution
- Improve version command output with JSON format support via --json flag and enhanced colored terminal output
- Add RunClient for submitting runs to server with priority support (low, normal, high, critical)
- Implement --run-priority and --server-url CLI flags for server submission mode
- Add RunPriority and RunMode fields to database Run model for persistence
- Update CreateRunRequest with priority and run_mode validation
- Implement runServerSubmission() to submit workflows with priority to server API
- Comprehensive E2E tests for run endpoints with priority validation and multiple targets
- Add foreach-preprocess test workflow with variable_pre_process support
- Add github-action target to trigger manual and nightly release workflows
- Extract BuildBuiltinVariables() function for reusable context setup
- Simplify RandomString from 8-char alphanumeric to 6-char lowercase
- Update documentation for RandomString variable description
- Enhanced cdnDelete to recursively delete folders with per-file tracking and error counts
- Added optional 'json' mode parameter to cdnSyncUpload/cdnSyncDownload for output format control
- Implemented real-time sync event callbacks for progress visualization with colored terminal output
- Added bash() as primary function with exec_cmd() as alias for backward compatibility
- Introduced SyncEvent type for action tracking (uploading/downloaded/skipped/deleted/error)
- Updated function tests to support new mode parameter
- Add TelegramChannelMap config field to support channel aliases (#channel_name)
- Implement 4 new notification functions for sending to specific channels:
- notify_telegram_channel(channel, message): send markdown message to channel
- send_telegram_file_channel(channel, path, caption?): send file to channel
- notify_message_as_file_telegram(path): read and send file as message
- notify_message_as_file_telegram_channel(channel, path): read and send file to channel
- Enhance telegram.go with channel resolution logic supporting both named channels and numeric IDs
- Update function registry with documentation and examples for new functions
- Add path expansion support (~, $HOME) to file operations for better usability
- Update example config with telegram_channel_map documentation
- Replace 0.0.0.0 with 127.0.0.1 in GetEventReceiverURL for local connectivity
- Fix spinner race condition in parallel binary installation with isFirstRender flag
- Optimize installation by pre-checking and skipping already-installed binaries
- Enhance server startup message to show localhost address alongside 0.0.0.0
- Update Next.js build IDs and chunk hashes in workflow upload static files
- Replace deprecated 'input' field syntax with new exports-style variables in event triggers (e.g., 'Target: event_data.value' instead of type/field/name)
- Simplify parameter override syntax in test workflows (remove nested 'default' structure)
Major features:
- Add run registry for tracking active runs with PID management
- Add API-based run cancellation with process termination
- Add event trigger input vars syntax for multi-variable extraction
- Add filter_functions with utility function support in triggers
- Add event envelope injection for full event context in workflows
- Add write coordinator for batched database operations
API improvements:
- Add logout endpoint and diffs endpoints for assets/vulnerabilities
- Add step-results listing endpoint
- Update schedule model with target, workspace, params fields
- Change run_id to run_uuid across API responses
Performance:
- Add compiled JS program caching for 60-80% faster loop conditions
- Add parallel shard rendering for 20-40% faster workflow startup
- Add memory-mapped I/O for large file line counting
- Add efficient output buffer combining in runners
- Add mtime-based cache invalidation for workflow loader
Other changes:
- Rename trigger field from trigger to triggers in workflow YAML
- Disable pongo2 HTML autoescape for shell command templates
- Update JWT expiration default to 1440 minutes (1 day)
- Change CORS default to reflect-origin for credentials support
- Add source_type field to events (run, eval, api)
- Skip copying core Unix tools to external-binaries
- Add top-level 'eval' command as shorthand for 'func eval' with multiple aliases (e, ev, evl, evla)
- Reorder Makefile snapshot-release to copy registry metadata before building binary
- Update contact URL in Swagger docs from osmedeus/osmedeus to osmedeus
- Update registry metadata with katana v1.4.0 (web crawling and spidering framework)
- Add file target support to heuristics with automatic TargetSpace derivation and comprehensive unit tests for file path parsing
- Display target space folder location during workflow execution for both module and flow runs
- Fix IP address handling in URL parsing to correctly identify and extract root domain for IP targets
- Add goreleaser --mark-latest flag to manual release workflow for consistent release tagging
- Improve install binary help text and add progress feedback for silent mode installations
- Replace deprecated --list-registry-binaries with --list-registry-direct-fetch and --list-registry-nix-build examples
- Add comprehensive API and CLI documentation for event receiver, asset/vulnerability diffing, settings management, and function evaluation
- Improve event receiver startup by separating StartEventReceiver() from StartListener() for better control over trigger registration timing
- Update README with new command examples for db list, func eval, and event-driven triggers
- Refactor PrintStartupInfo to display detailed trigger information with tree formatting and better visual hierarchy
- Change trigger registration logging from Info to Debug level to reduce noise
- Clarify --workspace flag documentation in run/scan commands
- Add Type field to EventReceiverTriggerInfo for better trigger visibility in API responses
j3ssie