Fixes four reported issues.
#321 / #322 — POST /osm/api/runs was unusable when the server was started
with --settings-file. That branch in pkg/cli/root.go loaded the settings via
config.LoadFromFile (a plain YAML unmarshal) and never called ResolvePaths(),
so every derived runtime path stayed empty. An empty WorkflowsPath made
workflow lookups scan "" (reported as "Workflow not found") and made a flow's
relative module refs resolve against the process working directory.
- root.go now resolves paths (and applies env overrides) on that branch
- ResolvePaths backfills any environments.* key a partial settings file omits,
sourced from the new defaultEnvironments() so DefaultConfig and the backfill
can no longer drift
- parser.ErrWorkflowsDirNotConfigured replaces the silent cwd fallback, so all
loader call sites report the misconfiguration instead of a misleading 404
#323 — a run submitted with run_mode:distributed could be lost permanently.
BRPOP is at-most-once: a task popped but not yet recorded in osm:tasks:running
existed nowhere, and the master's recovery sweep only reads that hash.
- workers now claim via BLMOVE onto osm:tasks:processing:{worker_id}
- the claim is acked only once SetTaskRunning succeeds; a failure requeues
instead of executing the task untracked
- recovery on worker startup, on dead workers, and for processing lists left
by workers that are no longer registered
- requires Redis 6.2+
#320 — settings values can now be overridden by OSM_* environment variables so
secrets need not live in osm-settings.yaml. The mapping is derived from the YAML
tags by reflection, so new settings are overridable with no extra code. Applied
in config.Load, hotreload and the --settings-file branch, but deliberately not
in LoadFromFile: `osmedeus config set` round-trips through it and writes back,
which would persist env secrets to disk.
Known gaps: pkg/cli/worker_queue.go still consumes the pending queue with the
at-most-once PopTask, and two distributed e2e tests (TaskSubmission,
FullWorkflow) fail on main independently of these changes.
Org (tenant) layer
- New Org model with org_uuid denormalized onto workspaces, assets,
vulnerabilities and runs so cross-workspace queries need no join
- Automatic attribution via BeforeAppendModel hooks; importers stay org-unaware
- Read semantics: empty org means no filter (backward compatible)
Write semantics: empty org coerced to the default org
- Migration backfills every pre-existing row into the default org
- CLI: osmedeus org create/show/assign/use/rename/delete
- API: /osm/api/orgs CRUD plus ?org= on assets, vulns, runs and workspaces
npm distribution
- npm install -g @j3ssie/osmedeus ships the Go binary through npm
- One npm name with version-suffixed platform builds pulled in as aliased
optionalDependencies, so an install downloads exactly one binary
- Binary ships gzipped and is decompressed on first run into a
version-scoped cache, so an upgrade can never exec a stale binary
- make bump-version is the single source of truth for the version constant
Bundled agent skills
- public/skills/ embedded in the binary, installed via osmedeus skills install
- Filesystem-driven discovery: a new bundle needs no code change
- make sync-skills mirrors bundles out to the standalone skills repo
Platform sub-projects
- Vendor dashboard, registry and workflow under platform/ so they version
with the engine they talk to; make sync-platform publishes them out
- Rebuild the embedded UI in public/ui/
- Reorder Migrate() to add columns to existing tables BEFORE creating
indexes, since CREATE TABLE IF NOT EXISTS is a no-op on existing
databases and some indexes (e.g. finding_hash) reference migrated
columns that would otherwise be missing
- Add explanatory comment in internal/database/database.go documenting
why column migrations must precede index creation
- Surface a helpful CLI hint in pkg/cli/root.go pointing users to
'osmedeus db migrate' when startup fails with 'failed to run
migrations', so stale schemas are easier to recover from
- Add query subcommands (vulns, runs, steps) with JSON output and flexible filtering
- Enhance assets command with --where, --search, --value filters (fuzzy matching)
- Expand asset searchable columns (status_code, content_type, title, dns_records, tls, tech)
- Add run status and run cancel subcommands with process termination
- Support control-plane and PID-based cancellation with database updates
- Implement fuzzyFilters in GetTableRecords for case-insensitive substring matching
- Add OSM_IGNORE_REGISTRY env var to skip auto binary installation and health checks
- Update health check to display warning when binary folder is empty but OSM_IGNORE_REGISTRY is set
- Update installRequiredBinaries to skip installation when OSM_IGNORE_REGISTRY=true
- Enhance DownloadFile with retry logic and wget/curl fallback for resilient downloads
- Refactor download function to validate content length and handle transient failures
- Refactor test code to use switch statements instead of if-else chains
- Implement agent-acp step type for spawning external ACP agent subprocesses via Agent Communication Protocol
- Add ACPExecutor with validation, field rendering, and subprocess lifecycle management
- Integrate agent-acp field rendering in StepDispatcher (batch and sequential modes)
- Add run_agent() utility function for workflows to execute ACP agents from steps and JS context
- Add osmedeus agent CLI command for interactive agent execution with --agent, --cwd, --timeout, --stdin, and --list flags
- Add /osm/api/agent/chat/completions REST endpoint with OpenAI-compatible chat format and concurrency control
- Support agent selection via: built-in names (claude-code, codex, opencode, gemini) or custom acp_config.command
- Add step-level configuration: cwd, allowed_paths, acp_config (command, args, env, write_enabled)
- Add comprehensive E2E tests for agent-acp workflows (basic, minimal, config, codex variants)
- Add test workflows in test/testdata/workflows/agent-and-llm/
- Update AGENTS.md documentation with agent-acp examples, CLI usage, and API endpoints
- Add stripTemplateVarsForJS() to convert {{var}} outside quotes to bare JS variable names, enabling direct param/export access in condition expressions
- Add normalizeBoolStringsForJS() to convert string 'true'/'false' to actual booleans for correct JS truthiness evaluation
- Auto-classify asset types in db_import_asset_from_file() and default source to 'web' when empty
- Improve table display with terminal auto-width detection via term.GetSize(), custom column weights, and display name aliases (status_code → status)
- Change default --width to 0 (auto-detect) with weighted column distribution and minimum header-based sizing
- Add comprehensive tests for param-based conditions and JSON asset imports
- Update test data to use direct param references in conditions instead of exports
- Refactor table display using tablewriter with box-drawing characters and proper Unicode width handling
- Add --as-cron flag to register cron schedules instead of executing immediately
- Implement new 'osmedeus assets' command for asset querying with filtering and statistics
- Enhance workflow parameter display with categorization and per-module summaries
- Improve column width handling and parameter documentation
- Implement cloud provider infrastructure (DigitalOcean, AWS, GCP, Linode, Azure) with Pulumi integration for distributed scanning
- Add nmap and tmux utility functions for port scanning results processing and long-running background session management
- Introduce webhook-triggered run execution with unique UUID and authentication key support for external integrations
- Add QueuePoller for dual-source (DB + Redis) task polling with concurrent execution
- Add worker queue subcommands (list, new, run) for managing queued tasks
- Add uninstall command to safely remove Osmedeus installation
- Add input validation to prevent command injection via dangerous shell characters
- Add queue run support via --queue and --queue-run flags in osmedeus run
- Add queue polling to server with --no-queue-polling flag to disable it
- Support queuing tasks from both CLI and API with deduplication across sources
- Add replayDBOperations to reinitialize databases from imported workspaces, with proper parameter resolution and pre-condition skipping
- Implement ISO 8601 timestamp format for snapshots (2026-02-13T18-20-34Z) instead of Unix epoch
- Add --include-heavy flag to include large fields (raw_response, screenshot, blob_content) in database queries
- Fix asset table default columns (url, status_code, content_length, title) for better UX
- Skip heavy initialization for lightweight commands (help, version, completion) to avoid ~50MB overhead
- Optimize database function execution via lazy config initialization
- Fix binary installation via Nix to copy already-installed binaries to binaries folder
- Add inline module support to ModuleRef allowing modules to be defined directly in flows without external files
- Implement IsInline() and ToWorkflow() methods to convert inline ModuleRef definitions to executable workflows
- Update ModuleRef.Clone() to properly duplicate all fields including Steps, Runner, RunnerConfig, and Description
- Add field alignment formatting and make path optional for inline modules
- Enhance parser validation to allow omitting path for inline modules while requiring at least one step
- Update executor's preloadModules() and ExecuteFlow() to handle inline modules during flow execution
- Improve version command output with JSON format support via --json flag and enhanced colored terminal output
Major features:
- Add run registry for tracking active runs with PID management
- Add API-based run cancellation with process termination
- Add event trigger input vars syntax for multi-variable extraction
- Add filter_functions with utility function support in triggers
- Add event envelope injection for full event context in workflows
- Add write coordinator for batched database operations
API improvements:
- Add logout endpoint and diffs endpoints for assets/vulnerabilities
- Add step-results listing endpoint
- Update schedule model with target, workspace, params fields
- Change run_id to run_uuid across API responses
Performance:
- Add compiled JS program caching for 60-80% faster loop conditions
- Add parallel shard rendering for 20-40% faster workflow startup
- Add memory-mapped I/O for large file line counting
- Add efficient output buffer combining in runners
- Add mtime-based cache invalidation for workflow loader
Other changes:
- Rename trigger field from trigger to triggers in workflow YAML
- Disable pongo2 HTML autoescape for shell command templates
- Update JWT expiration default to 1440 minutes (1 day)
- Change CORS default to reflect-origin for credentials support
- Add source_type field to events (run, eval, api)
- Skip copying core Unix tools to external-binaries
- Add top-level 'eval' command as shorthand for 'func eval' with multiple aliases (e, ev, evl, evla)
- Reorder Makefile snapshot-release to copy registry metadata before building binary
- Update contact URL in Swagger docs from osmedeus/osmedeus to osmedeus
- Update registry metadata with katana v1.4.0 (web crawling and spidering framework)
j3ssie