build: update dashboard artifacts and add repeat-modules test workflow

This commit is contained in:
j3ssie
2026-01-27 01:28:21 +08:00
parent 7f339a69f0
commit 5bb2ff7f0c
343 changed files with 1433 additions and 577 deletions
-65
View File
@@ -1,65 +0,0 @@
# Osmedeus API Documentation
## Overview
The Osmedeus API provides a RESTful interface for managing security automation workflows, runs, and distributed task execution.
**Base URL:** `http://localhost:8002`
**Default Port:** `8002`
## Authentication
Most API endpoints require authentication. Two methods are supported:
1. **JWT Token**: Obtain a token via the login endpoint, then include it in requests using the `Authorization: Bearer <token>` header.
2. **API Key**: Use a static API key via the `x-osm-api-key` header. Configure in `~/osmedeus-base/osm-settings.yaml` under `server.auth_api_key`.
See [Authentication](authentication.md) for details.
## API Reference
| Category | Description |
|----------|-------------|
| [Public Endpoints](public.md) | Server info, health checks, Swagger docs |
| [Authentication](authentication.md) | Login and JWT token management |
| [Workflows](workflows.md) | List, view, and refresh workflows |
| [Runs](runs.md) | Create and manage workflow executions |
| [File Uploads](uploads.md) | Upload target files and workflows |
| [Snapshots](snapshots.md) | Download workspace snapshots |
| [Workspaces](workspaces.md) | List and manage workspaces |
| [Assets](assets.md) | View discovered assets |
| [Vulnerabilities](vulnerabilities.md) | View and manage vulnerabilities |
| [Event Logs](event-logs.md) | View execution event logs |
| [Step Results](steps.md) | Query step execution results |
| [Functions](functions.md) | Execute and list utility functions |
| [System Statistics](system.md) | Get aggregated system stats |
| [Settings](settings.md) | Manage server configuration |
| [Installation](install.md) | Install binaries and workflows |
| [Schedules](schedules.md) | Manage scheduled workflows |
| [Distributed Mode](distributed.md) | Worker and task management |
| [LLM API](llm.md) | Large Language Model API |
| [Reference](reference.md) | Error codes, pagination, cron expressions, step types |
## Quick Start
```bash
# Get server info (no auth required)
curl http://localhost:8002/server-info
# Login and get token
export TOKEN=$(curl -s -X POST http://localhost:8002/osm/api/login \
-H "Content-Type: application/json" \
-d '{"username": "osmedeus", "password": "admin"}' | jq -r '.token')
# List workflows
curl http://localhost:8002/osm/api/workflows \
-H "Authorization: Bearer $TOKEN"
# Start a scan
curl -X POST http://localhost:8002/osm/api/runs \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{"flow": "subdomain-enum", "target": "example.com"}'
```
+73
View File
@@ -0,0 +1,73 @@
---
title: "Osmedeus API Documentation"
description: "RESTful API reference for managing security automation workflows"
---
# Osmedeus API Documentation
## Overview
The Osmedeus API provides a RESTful interface for managing security automation workflows, runs, and distributed task execution.
**Base URL:** `http://localhost:8002`
**Default Port:** `8002`
## Authentication
Most API endpoints require authentication. Two methods are supported:
1. **JWT Token**: Obtain a token via the login endpoint, then include it in requests using the `Authorization: Bearer <token>` header.
2. **API Key**: Use a static API key via the `x-osm-api-key` header. Configure in `~/osmedeus-base/osm-settings.yaml` under `server.auth_api_key`.
See [Authentication](authentication.mdx) for details.
## API Reference
| Category | Description |
|----------|-------------|
| [Public Endpoints](public.mdx) | Server info, health checks, Swagger docs |
| [Authentication](authentication.mdx) | Login, logout, and JWT token management |
| [Workflows](workflows.mdx) | List, view, and refresh workflows |
| [Runs](runs.mdx) | Create and manage workflow executions |
| [File Uploads](uploads.mdx) | Upload target files and workflows |
| [Snapshots](snapshots.mdx) | Export and import workspace snapshots |
| [Workspaces](workspaces.mdx) | List and manage workspaces |
| [Artifacts](artifacts.mdx) | List and download output artifacts |
| [Assets](assets.mdx) | View discovered assets |
| [Vulnerabilities](vulnerabilities.mdx) | View and manage vulnerabilities |
| [Event Logs](event-logs.mdx) | View execution event logs |
| [Step Results](steps.mdx) | Query step execution results |
| [Functions](functions.mdx) | Execute and list utility functions |
| [System Statistics](system.mdx) | Get aggregated system stats |
| [Settings](settings.mdx) | Manage server configuration |
| [Database](database.mdx) | Database management and cleanup |
| [Installation](install.mdx) | Install binaries and workflows |
| [Schedules](schedules.mdx) | Manage scheduled workflows |
| [Event Receiver](event-receiver.mdx) | Event-triggered workflows |
| [Distributed Mode](distributed.mdx) | Worker and task management |
| [LLM API](llm.mdx) | Large Language Model API |
| [Reference](reference.mdx) | Error codes, pagination, cron expressions, step types |
## Quick Start
```bash
# Get server info (no auth required)
curl http://localhost:8002/server-info
# Login and get token
export TOKEN=$(curl -s -X POST http://localhost:8002/osm/api/login \
-H "Content-Type: application/json" \
-d '{"username": "osmedeus", "password": "admin"}' | jq -r '.token')
# List workflows
curl http://localhost:8002/osm/api/workflows \
-H "Authorization: Bearer $TOKEN"
# Start a scan
curl -X POST http://localhost:8002/osm/api/runs \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{"flow": "subdomain-enum", "target": "example.com"}'
```
+116
View File
@@ -0,0 +1,116 @@
---
title: "Artifacts"
description: "List and download output artifacts from workflow runs"
---
# Artifacts
## List Artifacts
Get a paginated list of all artifacts across workspaces.
```bash
curl http://localhost:8002/osm/api/artifacts \
-H "Authorization: Bearer $TOKEN"
```
**With pagination:**
```bash
curl "http://localhost:8002/osm/api/artifacts?offset=0&limit=50" \
-H "Authorization: Bearer $TOKEN"
```
**Filter by workspace:**
```bash
curl "http://localhost:8002/osm/api/artifacts?workspace=example.com" \
-H "Authorization: Bearer $TOKEN"
```
**Query Parameters:**
| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `workspace` | string | - | Filter by workspace name |
| `offset` | int | 0 | Pagination offset |
| `limit` | int | 20 | Maximum records to return |
**Response:**
```json
{
"data": [
{
"id": "c3d4e5f6-a7b8-9012-cdef-123456789012",
"run_id": 1,
"workspace": "example.com",
"name": "subdomains.txt",
"artifact_path": "/workspaces/example.com/subdomains.txt",
"artifact_type": "output",
"content_type": "txt",
"size_bytes": 4523,
"line_count": 150,
"description": "Discovered subdomains",
"created_at": "2025-01-15T10:01:45Z"
},
{
"id": "d4e5f6a7-b8c9-0123-def0-234567890123",
"run_id": 1,
"workspace": "example.com",
"name": "nuclei-results.json",
"artifact_path": "/workspaces/example.com/nuclei-results.json",
"artifact_type": "output",
"content_type": "json",
"size_bytes": 15234,
"line_count": 45,
"description": "Nuclei vulnerability scan results",
"created_at": "2025-01-15T10:15:00Z"
}
],
"pagination": {
"total": 100,
"offset": 0,
"limit": 20
}
}
```
---
## Download Workspace Artifact
Download all artifacts from a workspace as a zip file.
```bash
curl http://localhost:8002/osm/api/artifacts/example.com \
-H "Authorization: Bearer $TOKEN" \
--output example.com_artifacts.zip
```
**Response:**
- On success: Returns a zip file containing all artifacts
- Response headers include:
- `Content-Disposition: attachment; filename=<workspace>_artifacts.zip`
- `Content-Type: application/zip`
**Error Response (404):**
```json
{
"error": true,
"message": "Workspace not found: example.com"
}
```
---
## Artifact Types
| Type | Description |
|------|-------------|
| `report` | Generated reports from the workflow's reports section |
| `state_file` | State files like run-state.json, run-execution.log |
| `output` | General output files from steps |
| `screenshot` | Screenshots captured during the scan |
## Content Types
Detected content types based on file extension:
- `json`, `jsonl`, `yaml`, `html`, `md`, `log`, `pdf`, `png`, `txt`, `zip`, `folder`, `unknown`
@@ -1,3 +1,8 @@
---
title: "Assets"
description: "Query discovered assets and asset diff snapshots"
---
# Assets
## List Assets
@@ -1,3 +1,8 @@
---
title: "Authentication"
description: "JWT and API key authentication for API access"
---
# Authentication
Most API endpoints require JWT authentication. First, obtain a token via the login endpoint, then include it in subsequent requests.
@@ -52,6 +57,28 @@ curl -X POST http://localhost:8002/osm/api/login \
}
```
## Logout
**POST** `/osm/api/logout`
Clear the session cookie to log out.
### Request
```bash
curl -X POST http://localhost:8002/osm/api/logout
```
### Response (200 OK)
```json
{
"message": "Logged out successfully"
}
```
---
## Token Details
- **Algorithm**: HS256 (HMAC-SHA256)
+73
View File
@@ -0,0 +1,73 @@
---
title: "Database Management"
description: "Administrative endpoints for database management"
---
# Database Management
Administrative endpoints for managing the database.
## List Database Tables
Get a list of all database tables with their row counts.
```bash
curl http://localhost:8002/osm/api/database/tables \
-H "Authorization: Bearer $TOKEN"
```
**Response:**
```json
{
"data": [
{"name": "runs", "rows": 150},
{"name": "step_results", "rows": 2500},
{"name": "assets", "rows": 5000},
{"name": "vulnerabilities", "rows": 120},
{"name": "workspaces", "rows": 50},
{"name": "artifacts", "rows": 800},
{"name": "event_logs", "rows": 1200},
{"name": "schedules", "rows": 8}
]
}
```
---
## Clear Database Table
Clear all records from a specific database table. Use with caution.
```bash
curl -X POST http://localhost:8002/osm/api/database/tables/runs/clear \
-H "Authorization: Bearer $TOKEN"
```
**Response:**
```json
{
"message": "Table cleared successfully",
"table": "runs",
"rows_deleted": 150
}
```
**Error Response (Invalid table):**
```json
{
"error": true,
"message": "Invalid table name: unknown_table"
}
```
**Valid Table Names:**
- `runs` - Workflow run records
- `step_results` - Step execution results
- `assets` - Discovered assets
- `vulnerabilities` - Vulnerability records
- `workspaces` - Workspace metadata
- `artifacts` - Output artifacts
- `event_logs` - Event log records
- `schedules` - Scheduled workflows
- `asset_diff_snapshots` - Asset diff snapshots
- `vuln_diff_snapshots` - Vulnerability diff snapshots
@@ -1,3 +1,8 @@
---
title: "Distributed Mode"
description: "Worker and task management for distributed execution"
---
# Distributed Mode
These endpoints are only available when running the server in master mode.
@@ -1,3 +1,8 @@
---
title: "Event Logs"
description: "Query execution event logs with filtering"
---
# Event Logs
## List Event Logs
+147
View File
@@ -0,0 +1,147 @@
---
title: "Event Receiver"
description: "Event-triggered workflow management"
---
# Event Receiver
These endpoints are only available when the event receiver is enabled (default when scheduler is enabled).
## Get Event Receiver Status
Get the current status of the event receiver including registered triggers.
```bash
curl http://localhost:8002/osm/api/event-receiver/status \
-H "Authorization: Bearer $TOKEN"
```
**Response:**
```json
{
"enabled": true,
"running": true,
"triggers": {
"cron": 5,
"event": 3,
"watch": 2
},
"workflows_loaded": 10,
"last_event_at": "2025-01-15T10:30:00Z"
}
```
---
## List Event Receiver Workflows
Get a list of all workflows registered with the event receiver, including their triggers.
```bash
curl http://localhost:8002/osm/api/event-receiver/workflows \
-H "Authorization: Bearer $TOKEN"
```
**Response:**
```json
{
"data": [
{
"name": "subdomain-enum",
"kind": "flow",
"triggers": [
{
"name": "daily-scan",
"type": "cron",
"schedule": "0 2 * * *",
"enabled": true,
"next_run": "2025-01-16T02:00:00Z"
},
{
"name": "on-new-asset",
"type": "event",
"topic": "assets.new",
"enabled": true
}
]
},
{
"name": "vuln-scan",
"kind": "module",
"triggers": [
{
"name": "watch-targets",
"type": "watch",
"path": "/data/targets/*.txt",
"enabled": true
}
]
}
],
"count": 2
}
```
---
## Emit Event
Emit a custom event to trigger event-based workflows.
```bash
curl -X POST http://localhost:8002/osm/api/events/emit \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{
"topic": "assets.new",
"data": {
"url": "https://new-subdomain.example.com",
"source": "external-scanner"
}
}'
```
**Request Body:**
| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `topic` | string | Yes | Event topic to emit (e.g., `assets.new`, `custom.event`) |
| `data` | object | No | Event data payload passed to triggered workflows |
**Response:**
```json
{
"message": "Event emitted",
"topic": "assets.new",
"event_id": "evt_550e8400-e29b-41d4-a716-446655440000",
"triggered_workflows": ["subdomain-enum", "asset-tracker"]
}
```
**Error Response (no matching triggers):**
```json
{
"message": "Event emitted",
"topic": "assets.new",
"event_id": "evt_550e8400-e29b-41d4-a716-446655440000",
"triggered_workflows": [],
"note": "No workflows matched this event topic"
}
```
---
## Event Topics
Common event topics used by workflows:
| Topic | Description |
|-------|-------------|
| `assets.new` | New asset discovered |
| `assets.updated` | Asset information updated |
| `vuln.found` | New vulnerability found |
| `run.completed` | Workflow run completed |
| `run.failed` | Workflow run failed |
| `schedule.triggered` | Scheduled trigger fired |
Custom topics can be used by prefixing with `custom.` (e.g., `custom.my-event`).
@@ -1,3 +1,8 @@
---
title: "Functions"
description: "Execute and list utility functions"
---
# Functions
## Execute Utility Function
@@ -1,3 +1,8 @@
---
title: "Installation"
description: "Binary and workflow installation from registry"
---
# Installation
## Get Registry Info
+5
View File
@@ -1,3 +1,8 @@
---
title: "LLM API"
description: "OpenAI-compatible chat completions and embeddings"
---
# LLM API
Direct API access to Large Language Model capabilities without requiring workflow execution.
@@ -1,3 +1,8 @@
---
title: "Public Endpoints"
description: "Server info, health checks, and Swagger docs"
---
# Public Endpoints
These endpoints do not require authentication.
@@ -1,3 +1,8 @@
---
title: "API Reference"
description: "Error codes, pagination, cron expressions, step types"
---
# API Reference
## Error Responses
+124 -2
View File
@@ -1,3 +1,8 @@
---
title: "Runs"
description: "Create and manage workflow executions"
---
# Runs (Scans)
## Create a New Scan
@@ -80,6 +85,27 @@ curl -X POST http://localhost:8002/osm/api/runs \
}'
```
**Request Body Parameters:**
| Parameter | Type | Required | Default | Description |
|-----------|------|----------|---------|-------------|
| `flow` | string | No* | - | Flow workflow name to execute |
| `module` | string | No* | - | Module workflow name to execute |
| `target` | string | No** | - | Single target to scan |
| `targets` | array | No** | - | Multiple targets to scan |
| `target_file` | string | No** | - | Path to file containing targets |
| `params` | object | No | `{}` | Custom workflow parameters |
| `priority` | string | No | `normal` | Priority level: `low`, `normal`, `high`, `critical` |
| `timeout` | int | No | - | Timeout in minutes for the run |
| `concurrency` | int | No | 1 | Number of concurrent targets |
| `runner_type` | string | No | `host` | Execution environment: `host`, `docker`, `ssh` |
| `docker_image` | string | No | - | Docker image for docker runner |
| `ssh_host` | string | No | - | SSH host for ssh runner |
| `workspace` | string | No | auto | Custom workspace name |
\* One of `flow` or `module` is required.
\** One of `target`, `targets`, or `target_file` is required.
**Response:**
```json
{
@@ -88,7 +114,7 @@ curl -X POST http://localhost:8002/osm/api/runs \
"kind": "flow",
"target": "example.com",
"target_count": 1,
"priority": "high",
"priority": "normal",
"job_id": "a1b2c3d4",
"run_uuid": "550e8400-e29b-41d4-a716-446655440000",
"status": "queued",
@@ -124,7 +150,7 @@ curl -X POST http://localhost:8002/osm/api/runs \
"target_count": 3,
"targets": ["example.com", "test.com", "demo.com"],
"concurrency": 3,
"priority": "medium",
"priority": "normal",
"job_id": "b2c3d4e5",
"status": "queued",
"poll_url": "/osm/api/jobs/b2c3d4e5"
@@ -426,3 +452,99 @@ curl http://localhost:8002/osm/api/runs/1/artifacts \
**Content Types:**
- `json`, `jsonl`, `yaml`, `html`, `md`, `log`, `pdf`, `png`, `txt`, `zip`, `folder`, `unknown`
---
## Duplicate Run
Create a copy of an existing run with the same configuration.
```bash
curl -X POST http://localhost:8002/osm/api/runs/550e8400-e29b-41d4-a716-446655440000/duplicate \
-H "Authorization: Bearer $TOKEN"
```
**Response:**
```json
{
"message": "Run duplicated",
"original_run_id": "550e8400-e29b-41d4-a716-446655440000",
"new_run_uuid": "660e8400-e29b-41d4-a716-446655440001",
"status": "pending"
}
```
---
## Start Run
Start a pending run that was created but not yet started.
```bash
curl -X POST http://localhost:8002/osm/api/runs/550e8400-e29b-41d4-a716-446655440000/start \
-H "Authorization: Bearer $TOKEN"
```
**Response:**
```json
{
"message": "Run started",
"run_uuid": "550e8400-e29b-41d4-a716-446655440000",
"status": "running"
}
```
---
## Get Job Status
Get the status of a job (a group of runs from the same request). This is useful for tracking multi-target scans.
```bash
curl http://localhost:8002/osm/api/jobs/a1b2c3d4 \
-H "Authorization: Bearer $TOKEN"
```
**Response:**
```json
{
"job_id": "a1b2c3d4",
"total_runs": 3,
"completed": 2,
"running": 1,
"failed": 0,
"pending": 0,
"runs": [
{
"run_uuid": "550e8400-e29b-41d4-a716-446655440000",
"target": "example.com",
"status": "completed"
},
{
"run_uuid": "550e8400-e29b-41d4-a716-446655440001",
"target": "test.com",
"status": "completed"
},
{
"run_uuid": "550e8400-e29b-41d4-a716-446655440002",
"target": "demo.com",
"status": "running"
}
]
}
```
---
## Priority Levels
Runs can be assigned a priority level to control execution order when multiple runs are queued.
| Priority | Description |
|----------|-------------|
| `low` | Lowest priority, processed last |
| `normal` | Default priority (used when not specified) |
| `high` | Higher priority, processed before normal/low |
| `critical` | Highest priority, processed first |
**Note:** Priority defaults to `normal` when not specified in the request
@@ -1,3 +1,8 @@
---
title: "Schedules"
description: "Manage scheduled workflow triggers"
---
# Schedules
## List Schedules
@@ -1,3 +1,8 @@
---
title: "Settings"
description: "Server configuration management"
---
# Settings
Manage server configuration settings.
@@ -70,3 +75,51 @@ curl -X PUT http://localhost:8002/osm/api/settings/yaml \
"message": "Invalid YAML configuration: yaml: unmarshal errors: ..."
}
```
---
## Reload Configuration
Trigger a hot-reload of the configuration file without restarting the server.
```bash
curl -X POST http://localhost:8002/osm/api/settings/reload \
-H "Authorization: Bearer $TOKEN"
```
**Response:**
```json
{
"message": "Configuration reloaded successfully",
"path": "/home/user/osmedeus-base/osm-settings.yaml"
}
```
**Error Response:**
```json
{
"error": true,
"message": "Failed to reload configuration: ..."
}
```
---
## Get Configuration Status
Get the current status of the hot-reloadable configuration, including last reload time and any errors.
```bash
curl http://localhost:8002/osm/api/settings/status \
-H "Authorization: Bearer $TOKEN"
```
**Response:**
```json
{
"path": "/home/user/osmedeus-base/osm-settings.yaml",
"last_reload": "2025-01-15T10:30:00Z",
"is_watching": true,
"error": ""
}
```
@@ -1,3 +1,8 @@
---
title: "Snapshots"
description: "Export and import workspace snapshots"
---
# Snapshots
## List Snapshots
+5
View File
@@ -1,3 +1,8 @@
---
title: "Step Results"
description: "Query step execution results"
---
# Step Results
## List Step Results
@@ -1,3 +1,8 @@
---
title: "System Statistics"
description: "Aggregated system statistics"
---
# System Statistics
## Get System Stats
@@ -1,3 +1,8 @@
---
title: "File Uploads"
description: "Upload target files and workflows"
---
# File Uploads
## Upload Input File
@@ -1,3 +1,8 @@
---
title: "Vulnerabilities"
description: "View and manage vulnerability records"
---
# Vulnerabilities
## List Vulnerabilities
@@ -1,3 +1,8 @@
---
title: "Workflows"
description: "List, view, and refresh workflows"
---
# Workflows
## List All Workflows
@@ -280,4 +285,3 @@ Steps can have dependencies on other steps using the `depends_on` field:
]
}
```
@@ -1,8 +1,13 @@
---
title: "Workspaces"
description: "List and manage scan workspaces"
---
# Workspaces
## List Workspaces
Get a list of all run workspaces.
Get a list of all run workspaces with full metadata.
**List workspaces from database (default):**
```bash
@@ -110,6 +115,29 @@ curl "http://localhost:8002/osm/api/workspaces?filesystem=true&offset=20&limit=1
---
## List Workspace Names
Get a lightweight list of workspace names only (without full metadata).
```bash
curl http://localhost:8002/osm/api/workspace-names \
-H "Authorization: Bearer $TOKEN"
```
**Response:**
```json
{
"data": [
"example.com",
"test.com",
"demo.org"
],
"count": 3
}
```
---
## Get Workspace State File
Retrieve the content of a workspace state file. This endpoint provides access to execution logs, completion markers, and workflow files associated with a workspace.