Commit Graph
9 Commits
Author SHA1 Message Date
j3ssie 459d5939fb feat: add cloud infrastructure and nmap/tmux function support
- Implement cloud provider infrastructure (DigitalOcean, AWS, GCP, Linode, Azure) with Pulumi integration for distributed scanning
- Add nmap and tmux utility functions for port scanning results processing and long-running background session management
- Introduce webhook-triggered run execution with unique UUID and authentication key support for external integrations
2026-02-16 15:23:24 +07:00
j3ssie 33aa3d82bc feat: add TypeScript execution and CDN/WAF asset classification
- Add exec_ts() and exec_ts_file() utility functions for inline and file-based TypeScript execution via Bun
- Implement CDN/WAF detection system with is_cdn, is_cloud, is_waf boolean fields for assets
- Enhance Python execution to prefer uv package manager with fallback to python3/python
- Update roadmap with cloud integration step, clarify security warning, improve documentation
2026-02-15 11:32:55 +07:00
j3ssie 801583b0c8 feat: distributed E2E testing stack and improve worker/master logging
- Add Docker Compose distributed-e2e.yaml with Redis, master, and worker services
- Add Make targets: distributed-e2e-up, distributed-e2e-run, distributed-e2e-down
- Replace zap logger with terminal printer for consistent CLI output in master/worker
- Add Redis connection info and task details to startup and lifecycle logs
- Fix DNS records field: use 'dns_records' key with backward compat for 'a' key
- Add optional asset_type and source parameters to db_import_custom_asset
- Update Swagger docs for LLM embeddings schema
- Set run_uuid tracking in worker task execution for distributed runs
- Register distributed database hooks and ensure external binaries in PATH
- Improve terminal formatting with colors for worker/master IDs and log messages
2026-02-14 03:00:30 +07:00
j3ssie b6e9d12324 feat: implement path-friendly loop variables and improve asset merge logic
- Add auto-generated _<variable>_ path-friendly variables for foreach loops that sanitize unsafe filesystem characters (/, :, etc.) and deterministically truncate long values
- Refactor asset and vulnerability import/merge logic to preserve existing non-empty fields instead of full overwrite on conflict
- Add mergeAssetFields() and mergeVulnFields() helper functions for consistent field-level merge behavior across all import methods
- Add comprehensive unit tests for merge functions and path-friendly variable behavior in foreach loops
- Add E2E test module (test-foreach-path-friendly) validating sanitization, directory creation, truncation, and variable coexistence
2026-02-13 13:27:04 +07:00
j3ssie 225f2d7d15 feat: add SSH and DNS asset import utilities
- Add ssh_exec() and ssh_rsync() functions for remote command execution and file transfer with connection pooling
- Add db_import_dns_asset() to import DNS zone records and group by domain with A/AAAA records stored separately
- Add db_import_custom_asset() for flexible JSONL asset import with direct field mapping, supporting tags and custom asset types
- Extend Asset model with ExternalURL, Remarks (string array), Language, Size, and LOC fields for better metadata tracking
- Add backward compatibility layer for legacy JSON formats (string remarks, tags array merging)
- Register new functions in function registry with proper documentation and usage examples
2026-02-12 19:44:39 +07:00
j3ssie 438d8ec138 feat: implement agent executor with tool calling, sub-agents, and comprehensive test suite
- Add AgentExecutor implementing LLM-based agentic loop with tool calling, max iterations, and stop conditions
- Introduce agent preset tools (bash, file_exists, http_get, run_module, etc.) with extensible registry pattern
- Add sub-agent spawning capability via spawn_agent tool call with recursive depth limits and validation
- Implement ToolExecutor for custom tool execution with template rendering and error handling
- Add agent session persistence and memory management with sliding window configuration
- Create comprehensive E2E test suite covering 15+ agent workflow scenarios (minimal, custom tools, planning, multi-goal, structured output, tracing hooks, file tools, orchestration, Python tools, sub-agents, nested sub-agents, and validation)
- Add agent-and-llm test data directory with 17 YAML workflow fixtures
- Update integration tests to include agent workflow directories
- Add AgentTool and AgentConfig types with validation for duplicate names and unknown presets
- Implement LLM streaming test utilities
- Update documentation (CLAUDE.md, HACKING.md, README.md) with agent features and CLI examples
2026-02-10 08:44:48 +07:00
j3ssie 7de3e7b2b2 feat: add performance optimizations and robustness improvements
- Add lock-free ResultCollector for parallel execution with atomic operations, eliminating mutex contention for pre-allocated slices
- Implement circuit breaker pattern (internal/retry/circuit_breaker) with configurable thresholds and half-open recovery state for fault tolerance
- Introduce json-iterator replacement (internal/json) for 2-6x faster JSON operations while maintaining stdlib compatibility
- Add lazy template rendering (RenderLazy) with variable reference caching for 50-80% faster rendering on large contexts
- Implement memory-efficient buffer pooling (bufpool) with 10MB pre-allocated reusable buffers for reduced GC pressure
- Add LoadFlowWithModules for parallel module pre-loading using errgroup, improving startup time for complex flows
- Add VarRefCache with LRU eviction for variable extraction caching
- Add streaming output support for foreach loops to process large datasets without memory accumulation
- Fix json import compatibility in llm_executor and db_functions
- Update test fixtures with correct YAML field names (call→function, run→command)
2026-01-31 01:26:28 +07:00
j3ssie 777fb7c2bf chore: update ui build artifacts and refactor event trigger input syntax
- Update Next.js build IDs and chunk hashes in workflow upload static files
- Replace deprecated 'input' field syntax with new exports-style variables in event triggers (e.g., 'Target: event_data.value' instead of type/field/name)
- Simplify parameter override syntax in test workflows (remove nested 'default' structure)
2026-01-24 16:24:50 +08:00
j3ssie 7a2c5a5dc9 Complete rewrite and re-architecture Osmedeus Engine in v5 2026-01-18 19:32:24 +08:00