Commit Graph
124 Commits
Author SHA1 Message Date
j3ssie 5d9ff490f2 chore: update Docker tooling and sponsor links 2026-07-21 23:30:09 +07:00
Jessie Ho (j3ssie)andGitHub 2edef9f4b8 Merge pull request #314 from Friday-s/add-bloome-sponsor
docs: add Bloome sponsor section
2026-07-13 21:52:12 +08:00
Friday-s 6ac9b6f14e docs: add Bloome sponsor section before Installation 2026-07-13 18:28:30 +08:00
j3ssie d5aa39ba30 fix(db_import_vigolium): stream JSONL with bufio.Reader to handle oversized lines
Vigolium http_record and finding entries can embed multi-MB response bodies,
producing JSONL lines that exceed bufio.Scanner's max token size and fail
with 'token too long'. Switch to bufio.Reader.ReadString, which grows as
needed, and extract per-line logic into a processLine closure. Add a
regression test that imports a finding with a 12MB response body.
2026-06-01 13:13:40 +08:00
j3ssie 896527421d fix(executor): preserve unrendered threads template and tidy run output
- Emit the raw `threads` value in step command summaries so template variables
  like `{{deparosParallel}}` survive until templateEngine.Render resolves them,
  instead of silently collapsing to 1 when Int() fails to parse the template
- Only print the Artifacts section in `printResultSummary` when -v/--verbose
  is set, removing state/log file clutter from default run output
- Update embedded UI build artifacts under public/ui/_next to the latest
  dashboard build (chunk renames, manifest updates, page txt/html refreshes)
2026-06-01 12:43:00 +08:00
j3ssie ec19282e64 fix(database): run column migrations before index creation
- Reorder Migrate() to add columns to existing tables BEFORE creating
  indexes, since CREATE TABLE IF NOT EXISTS is a no-op on existing
  databases and some indexes (e.g. finding_hash) reference migrated
  columns that would otherwise be missing
- Add explanatory comment in internal/database/database.go documenting
  why column migrations must precede index creation
- Surface a helpful CLI hint in pkg/cli/root.go pointing users to
  'osmedeus db migrate' when startup fails with 'failed to run
  migrations', so stale schemas are easier to recover from
2026-05-31 01:08:43 +08:00
j3ssie 8ec1de4f84 feat(db): add db_import_vigolium importer and harden CLI run
- Add db_import_vigolium JS function that routes vigolium JSONL records
  by envelope type: http_record -> assets, finding -> vulnerabilities
  (deduped on new finding_hash column), skipping scan/oast_interaction
- Add FindingHash field to Vulnerability model with idempotent ALTER
  TABLE migration and matching index
- Suppress run errors when using --silent --empty-target placeholder mode
- Retry transient fetchURLContent failures (network errors, 408/429/5xx)
  with exponential backoff; leave 4xx untouched so GitHub auth fallback
  can engage
- Add unit tests for the vigolium importer (import + idempotency) and
  the HTTP retry behavior, plus a vigolium juice-shop sample fixture
- Bump katana, naabu, kingfisher, bearer registry entries and add
  vigolium to the direct-fetch registry
v5.0.3
2026-05-29 23:31:56 +08:00
j3ssie 8448791c3c feat: propagate run cancellation to tmux sessions and remote SSH processes
- Track tmux sessions in ActiveRun and kill them on run cancel via new
  TmuxHooks indirection; expose killed_tmux_sessions in CLI and API responses
- Add ExecuteSSHCommand with remote pidfile + process-group kill watcher so
  cancelling a run actually terminates remote scans (not just the local session)
- Route ssh_exec/ssh_rsync/sync_* through the run's cancellable context via
  new RunContextHooks
- Switch docker-publish to sequential per-arch buildx builds + imagetools
  manifest to avoid OOM on multi-arch builds; add docker-buildx-setup target
- Cross-compile Dockerfile via BUILDPLATFORM/TARGETOS/TARGETARCH and retry
  SAST binary installs to survive QEMU-flaky downloads
- Bump version to v5.0.3
2026-05-26 00:06:26 +08:00
Jessie Ho (j3ssie)andGitHub f82afde4f4 Merge pull request #305 from sissl0/main
Webhook Notification on all return paths AND extended on_error actions
2026-05-12 01:26:55 +08:00
Jessie Ho (j3ssie)andGitHub 200f4ee732 Merge pull request #307 from NightRang3r/main
🐛 fix(distributed): link worker step results to master run via UUID
2026-05-12 00:52:15 +08:00
j3ssie 8136c20fa3 chore: fix the Discord link 2026-05-11 21:45:22 +08:00
Shai Rod df8ad1299b 🐛 fix(distributed): link worker step results to master run via UUID
Workers now propagate RunUUID from the submitted task so the master
can resolve the correct DB run record and associate step results.
Also increments completed_steps on successful insert and connects
the worker to the database for db_import_* functions.
2026-05-07 20:29:26 +03:00
j3ssie 9f95043356 chore(registry): bump tool versions and rename httpx to pd-httpx
- Bump amass, subfinder, nuclei, naabu, kingfisher, trivy, gosec, cariddi, cloudfox to latest releases
- Rename httpx registry entry to pd-httpx to disambiguate from other tools
- Refresh kingfisher description and update registry _last_update_at timestamp
2026-05-01 03:17:09 +08:00
Georg HeindlandCopilot 4ea9fda1bb Return artifact paths
Co-authored-by: Copilot <copilot@github.com>
2026-04-23 20:07:00 +02:00
Georg Heindl 771c000e92 Empty error panic on completion fixed 2026-04-23 19:53:24 +02:00
Georg HeindlandCopilot 39827b8b99 Included error, artifacts, message
Co-authored-by: Copilot <copilot@github.com>
2026-04-23 19:27:35 +02:00
Georg Heindl ce7199753f rm error 2026-04-23 17:52:00 +02:00
Georg HeindlandCopilot 67dc6b3de7 Added runUUID, error, steps, exports to webhook notification
Co-authored-by: Copilot <copilot@github.com>
2026-04-23 16:56:28 +02:00
Georg Heindl 29efc1b8e5 Remove duplicate condition check 2026-04-23 14:32:10 +02:00
Georg Heindl a4df58e063 Added on_error actions: Notify, Continue, Abort and run type function 2026-04-22 14:47:45 +02:00
Georg HeindlandGitHub fc827be975 Webhook Notification on every result path of ExecuteFlow/Module 2026-04-22 14:31:23 +02:00
j3ssie f467fa3c08 ci: mirror nightly OOM fixes in manual release (parallelism=1, GOMEMLIMIT, -trimpath) 2026-04-15 22:12:07 +08:00
j3ssie 1c9963f223 ci: drop nightly to parallelism=1, add GOMEMLIMIT and -trimpath, remove tidy hook 2026-04-15 21:46:15 +08:00
j3ssie 418c52f465 ci: fix nightly release OOM by limiting goreleaser parallelism
Build was being killed by the runner ~29min in (OOM on 7GB ubuntu-latest)
because goreleaser builds all targets in parallel by default. Cap to 2,
drop linux/386, and tune GOGC for lower peak memory.
2026-04-15 21:05:58 +08:00
j3ssie cda49b3897 ci: upgrade goreleaser action to v7, improve nightly CI, and tag optional binaries
- Upgrade goreleaser/goreleaser-action from v6 to v7 in manual and nightly release workflows
- Add timeout-minutes and Go cache to nightly release for faster, safer builds
- Normalize YAML string quoting from single to double quotes
- Add 'optional' tag to jq and massdns in direct-fetch registry metadata
2026-04-15 20:33:07 +08:00
j3ssie 52b4f05709 feat: auto-install pulumi cli and adjust ssh retry configuration
- Auto-install Pulumi CLI when not found instead of returning error
- Add installed binary to PATH for immediate availability
- Update SSH pool retry config to 5 attempts with 10s fixed delays
- Improve user feedback with progress indicators
2026-04-06 23:09:01 +08:00
j3ssie 69e3be23eb feat: add passive security scanning modules for frameworks and APIs
- Add 40+ new passive vulnerability detection modules covering:
  - Build/deployment misconfigurations (Next.js, Vite, webpack source maps)
  - Content Security Policy weakness auditing
  - HSTS preload readiness and secure header validation
  - API version detection, gRPC-Web, and WebAssembly discovery
  - Framework fingerprinting (WordPress, Drupal, Joomla, Firebase, Laravel, ASP.NET, Spring, Express, Rails, Django, Flask, FastAPI)
  - Cloud storage detection and signed URL/SAS token leaks
  - REST API analysis with pagination and sensitive field detection
  - GraphQL introspection and error information leakage
  - Debug information exposure (Python, Rails, Django)
  - Session management auditing for Express.js
  - CORS and cache poisoning vulnerabilities

All modules are passive-only with no external HTTP requests sent beyond initial scanning traffic.
2026-04-05 14:32:02 +08:00
j3ssie 9f9f107d95 build: update docker-build target to use explicit Dockerfile path 2026-04-04 15:21:41 +08:00
j3ssie 7804579dbd docs: add cloud setup and query commands documentation
- Add cloud setup section documenting SSH-based machine configuration
- Add query commands for vulnerabilities, runs, and execution steps
- Document Go install fallback for packages without go.mod
- Update README with cloud and query command examples
- Add reference to cloud setup E2E tests
2026-04-04 14:21:14 +08:00
Ai Ho (j3ssie)andGitHub a0a198d31e Merge pull request #302 from tair-m/patch-1
fix: change chat_id type to integer in docker config
2026-04-04 14:10:28 +08:00
j3ssie 0269cf4e26 feat: update Next.js build assets and add cloud setup E2E tests
- Update Next.js generated chunk hashes and build IDs reflecting latest dashboard build
- Update CSS stylesheet references in workflow upload page metadata
- Add comprehensive cloud setup E2E test suite (cloud_setup_test.go) with SSH password/key auth, post-command variable expansion, and Ansible integration
- Fix API priority levels to include 'medium' priority in test coverage
- Add agent-sdk test workflows (minimal, config, codex, multi-agent, session variants)
- Update E2E test utilities with runCLIInBase helper for multi-step cloud config tests
- Fix stderr/stdout capture in dependencies_target_types_test assertions
v5.0.2
2026-04-04 13:57:34 +08:00
tairandGitHub 3e5bad31d0 fix: change chat_id type to integer in docker config
Empty string for chat_id causes unmarshal error in Go. Changing to 0 fixes parsing failure
2026-04-04 02:55:27 +05:00
j3ssie 51bb8bfbac chore: update tool versions in registry metadata
- Update subfinder from 2.12.0 to 2.13.0 with corresponding download URLs
- Update nuclei from 3.7.0 to 3.7.1 with updated release links
- Update httpx from 1.8.1 to 1.9.0 across all platforms
- Update katana from 1.4.0 to 1.5.0 with new download links
- Update naabu from 2.4.0 to 2.5.0 with refreshed URLs
- Update kingfisher from 1.76.0 to 1.88.0 with new release binaries
- Update bearer from 2.0.0 to 2.0.1 with updated links
- Update gosec from 2.22.11 to 2.24.7
- Update interactsh from 1.3.0 to 1.3.1
- Update cloudfox from 1.17.0 to 2.0.1
- Update last_update_at timestamp to 2026-03-13T15:11:36.292828+00:00
2026-03-14 00:45:54 +08:00
Ai Ho (j3ssie)andGitHub b7eeaa524a Merge pull request #300 from YouChenJun/fix/trivy-404
fix: update trivy download link to v0.69.3 to fix 404 error
2026-03-13 22:12:04 +08:00
YouChenJun 16464a0533 fix: update trivy download link from v0.69.1 to v0.69.3 to fix 404 error 2026-03-12 21:23:21 +08:00
j3ssie 2dbec22627 fix: prevent git clone failures by removing existing destinations
- Add extractGitCloneDest() function to parse destination from git clone commands
- Remove existing git clone destination directories before recloning
- Add 'make' build tool to binary registry with platform-specific install commands
2026-03-08 14:00:20 +08:00
j3ssie 90ce5f4a16 refactor: consolidate process killing logic and ACP agent defaults
- Extract KillProcessAndChildren into core.types for reuse across CLI and server handlers
- Replace hardcoded 'claude-code' strings with core.DefaultACPAgent constant
- Precompute absolute allowed paths in ACP client to avoid repeated filepath.Abs calls
- Simplify runQuerySteps to delegate to runQueryTable, reducing code duplication
- Refactor agent_chat concurrency guard using sync.Mutex.TryLock for cleaner code
- Use request context for agent timeout instead of background context
2026-03-07 16:49:48 +08:00
Ai Ho (j3ssie)andGitHub cf98cab616 Merge pull request #299 from anhtn512/main
fix(installer): fix shallow clone and add go install fallback for binary installation
2026-03-07 15:50:40 +08:00
j3ssie ecc77b9a30 refactor: optimize ACP agent initialization and remove dead code
- Remove unreachable custom agent command validation from ACPExecutor.Execute()
- Add IsBuiltinAgent() utility function for cleaner agent name resolution
- Replace inline agent list iteration with IsBuiltinAgent() check in agent_chat.go
- Add explicit stdin pipe cleanup in RunAgentACP() defer block
- Simplify CLI output handling by removing redundant stream output fallback
2026-03-07 14:13:19 +08:00
j3ssie 9bdb3260b6 feat(cli): add query command and enhanced asset/run management
- Add query subcommands (vulns, runs, steps) with JSON output and flexible filtering
- Enhance assets command with --where, --search, --value filters (fuzzy matching)
- Expand asset searchable columns (status_code, content_type, title, dns_records, tls, tech)
- Add run status and run cancel subcommands with process termination
- Support control-plane and PID-based cancellation with database updates
- Implement fuzzyFilters in GetTableRecords for case-insensitive substring matching
2026-03-01 10:54:49 +08:00
j3ssie ca1c0d443a feat: add OSM_IGNORE_REGISTRY environment variable to skip binary installation
- Add OSM_IGNORE_REGISTRY env var to skip auto binary installation and health checks
- Update health check to display warning when binary folder is empty but OSM_IGNORE_REGISTRY is set
- Update installRequiredBinaries to skip installation when OSM_IGNORE_REGISTRY=true
- Enhance DownloadFile with retry logic and wget/curl fallback for resilient downloads
- Refactor download function to validate content length and handle transient failures
- Refactor test code to use switch statements instead of if-else chains
2026-02-28 17:29:42 +08:00
anhtn512 63653076c0 feat: enhance binary installation process with fallback to go install 2026-02-27 15:20:07 +07:00
anhtn512 e753f94877 fix: ensure go-getter uses depth=1 and sets ref=HEAD when version is not specified 2026-02-27 15:19:44 +07:00
j3ssie 520977d1ea feat: Add agent-acp step type with ACP protocol support
- Implement agent-acp step type for spawning external ACP agent subprocesses via Agent Communication Protocol
- Add ACPExecutor with validation, field rendering, and subprocess lifecycle management
- Integrate agent-acp field rendering in StepDispatcher (batch and sequential modes)
- Add run_agent() utility function for workflows to execute ACP agents from steps and JS context
- Add osmedeus agent CLI command for interactive agent execution with --agent, --cwd, --timeout, --stdin, and --list flags
- Add /osm/api/agent/chat/completions REST endpoint with OpenAI-compatible chat format and concurrency control
- Support agent selection via: built-in names (claude-code, codex, opencode, gemini) or custom acp_config.command
- Add step-level configuration: cwd, allowed_paths, acp_config (command, args, env, write_enabled)
- Add comprehensive E2E tests for agent-acp workflows (basic, minimal, config, codex variants)
- Add test workflows in test/testdata/workflows/agent-and-llm/
- Update AGENTS.md documentation with agent-acp examples, CLI usage, and API endpoints
v5.0.1
2026-02-27 16:18:06 +08:00
j3ssie 375c159f48 build: update dashboard UI artifacts from latest Next.js build 2026-02-20 00:34:15 +07:00
j3ssie 45c7ea0bbc feat: downgrade trigger input variables to info-level in linter
- Add tracking of trigger input variables and recognize their availability at runtime
- Downgrade undefined variable warnings to info-level when variables come from event trigger inputs (both new and legacy syntax)
- Add event envelope variables (EventTopic, EventSource, EventTimestamp, etc.) as recognized built-ins when workflow has event triggers
- Include comprehensive test coverage for trigger input scenarios, event triggers, and mixed variable definitions
2026-02-18 11:26:44 +07:00
j3ssie 2bbf7295f6 refactor: enhance linter with comprehensive undefined variable detection
- Add platform variables (PlatformOS, PlatformArch, PlatformInDocker, PlatformInKubernetes, PlatformCloudProvider) to linter's built-in variables list
- Extract checkStepFieldsForUndefinedVars and collectReferencedVarsFromStep helper functions to support comprehensive scanning of all template-renderable fields including parallel_functions, speed_args, config_args, agent fields, HTTP headers, LLM messages, decision cases/conditions, and memory paths
- Add recursion for nested steps (foreach inner steps and parallel_steps) with proper variable scoping and support for loop-specific variables (_id_, foreach variable)
- Enhance message formatting with highlightQuoted() to make quoted terms bold+yellow in error output
- Enable UndefinedVariableRule by default in GetDefaultRules()
- Add comprehensive test coverage for newly supported fields and nested step types (TestUndefinedVariableRule_*, TestUnusedVariableRule_ReferencedInNewFields)
- Add test fixture and E2E test (TestWorkflow_Validate_UndefinedVarsComprehensive) validating detection across functions and foreach
2026-02-18 10:18:44 +07:00
j3ssie 0da63bf9b3 feat: improve decision conditions with param-based JS evaluation
- Add stripTemplateVarsForJS() to convert {{var}} outside quotes to bare JS variable names, enabling direct param/export access in condition expressions
- Add normalizeBoolStringsForJS() to convert string 'true'/'false' to actual booleans for correct JS truthiness evaluation
- Auto-classify asset types in db_import_asset_from_file() and default source to 'web' when empty
- Improve table display with terminal auto-width detection via term.GetSize(), custom column weights, and display name aliases (status_code → status)
- Change default --width to 0 (auto-detect) with weighted column distribution and minimum header-based sizing
- Add comprehensive tests for param-based conditions and JSON asset imports
- Update test data to use direct param references in conditions instead of exports
2026-02-17 22:05:18 +07:00
j3ssie fba43ba3b5 feat: add string utility functions and decision condition tests
- Add cut_to_file() and cut_space() utility functions for file processing and field extraction
- Add comprehensive E2E tests for decision condition routing with function/command execution
- Add test workflows for decision conditions and inline decision execution
- Add short-mode skip guards to all cloud E2E tests to allow quick test runs
- Register new functions in constants and goja runtime
2026-02-17 17:00:43 +07:00
j3ssie 8d413aecb9 feat: table rendering, cron schedules, and assets command
- Refactor table display using tablewriter with box-drawing characters and proper Unicode width handling
- Add --as-cron flag to register cron schedules instead of executing immediately
- Implement new 'osmedeus assets' command for asset querying with filtering and statistics
- Enhance workflow parameter display with categorization and per-module summaries
- Improve column width handling and parameter documentation
2026-02-17 15:19:49 +07:00