mirror of
https://github.com/langchain-ai/langgraph.git
synced 2026-11-05 13:51:17 +01:00
Compare commits
8
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
f9e3a639c4 | ||
|
|
157ba9288d | ||
|
|
59c762e0cd | ||
|
|
6d1fddc3d4 | ||
|
|
9da6381025 | ||
|
|
95cdb86d28 | ||
|
|
0a05f15e21 | ||
|
|
2a30615bd7 |
No files matched your search
@@ -86,6 +86,93 @@ langgraph dockerfile SAVE_PATH [OPTIONS]
|
||||
-c, --config FILE Config file path
|
||||
```
|
||||
|
||||
### `langgraph deploy metrics`
|
||||
|
||||
Fetch deployment monitoring time series as JSON through the LangSmith control plane.
|
||||
Use `langgraph deploy list` to find the deployment ID. Authentication uses the same
|
||||
API key and workspace configuration as the other deployment commands.
|
||||
|
||||
```bash
|
||||
langgraph deploy metrics DEPLOYMENT_ID \
|
||||
--start-time 2026-10-28T00:00:00Z \
|
||||
--end-time 2026-10-28T01:00:00Z \
|
||||
--metrics cpu_usage,memory_usage,worker_count
|
||||
```
|
||||
|
||||
`--metrics` accepts a comma-separated list of these names (also listed in
|
||||
`langgraph deploy metrics --help`):
|
||||
|
||||
- Resource usage: `memory_usage`, `cpu_usage`, `disk_usage`
|
||||
- Counts: `restart_count`, `replica_count`, `worker_count`, `lg_run_count`
|
||||
- HTTP traffic: `responses_per_second`, `error_responses_per_second`, `p95_latency`
|
||||
- Queue: `run_queue_wait_time`
|
||||
|
||||
These are exact metric names, not free-text search terms. Availability depends on
|
||||
the deployment and platform; a supported metric may have no data. Use
|
||||
`lg_run_count` for deployment runs, not the UI schema's placeholder `run_count`.
|
||||
|
||||
Omit `--metrics` to fetch all available metrics. Time bounds default to the last
|
||||
15 minutes, and the backend picks the resolution from the time
|
||||
range. This command never prompts; for an organization-scoped key, set
|
||||
`LANGSMITH_TENANT_ID` to the workspace ID.
|
||||
|
||||
Default output is compact JSON grouped by section title, chart title, and series
|
||||
name. Each series contains its real `project_metric` as `metric`, inferred `unit`
|
||||
(`percent`, `count`, or `null`), `latest`, `min`, `max`, `avg`, and
|
||||
`points: [[iso_timestamp, value], ...]`. Values are rounded to three decimals;
|
||||
statistics use unrounded finite values and ignore missing samples. `latest` is
|
||||
the latest non-missing value. Missing samples remain `null` gaps, and series with
|
||||
no values have null statistics. Empty sections are marked `{"_empty":true}`.
|
||||
|
||||
The top level includes `deployment_id`, `start_time`, `end_time`, and
|
||||
`step_seconds`. Bounds always describe the requested window, not the observed
|
||||
point bounds. The CLI resolves omitted bounds once: end defaults to now (UTC),
|
||||
and start defaults to 15 minutes before the resolved end. With only `--end-time`,
|
||||
start is 15 minutes before that value; with only `--start-time`, end is now. Both
|
||||
values are sent explicitly to the backend and output, even when there are no points. `step_seconds` is the most common gap between consecutive
|
||||
timestamps within series, with ties broken by the smallest gap. Whole seconds
|
||||
are integers; fractional seconds are preserved. It is `null` only when every
|
||||
series has fewer than two timestamps. UI metadata and repeated series IDs are
|
||||
omitted. The host-backend response itself is unchanged.
|
||||
|
||||
```bash
|
||||
langgraph deploy metrics DEPLOYMENT_ID --summary
|
||||
langgraph deploy metrics DEPLOYMENT_ID --raw
|
||||
```
|
||||
|
||||
`--summary` returns the same shape without `points`. `--raw` returns the original
|
||||
UI chart schema without reshaping or rounding. These flags are mutually exclusive.
|
||||
Neither mode returns traces.
|
||||
|
||||
Size comparison using the realistic synthetic unit-test fixture (compact JSON
|
||||
serialization for all modes; not a live deployment benchmark):
|
||||
|
||||
| Before: raw UI schema | After: default | After: summary |
|
||||
| --- | --- | --- |
|
||||
| 4,302 bytes | 1,226 bytes (72% smaller) | 806 bytes (81% smaller) |
|
||||
|
||||
### `langgraph deploy scale`
|
||||
|
||||
Change a deployment's compute tier, dedicated Postgres tier, or both. Omitted
|
||||
settings remain unchanged. The backend enforces deployment permissions, supported
|
||||
tiers, and platform restrictions.
|
||||
|
||||
```bash
|
||||
langgraph deploy scale DEPLOYMENT_ID \
|
||||
--compute-tier DEDICATED_M --database-tier DEDICATED_L
|
||||
```
|
||||
|
||||
Compute tiers are `SERVERLESS_S`, `SERVERLESS_M`, `SERVERLESS_L`, `DEDICATED_S`,
|
||||
`DEDICATED_M`, and `DEDICATED_L`. Database tiers are available only for dedicated
|
||||
deployments: `DEDICATED_S`, `DEDICATED_M`, and `DEDICATED_L`. Availability depends
|
||||
on the deployment's platform and enabled features.
|
||||
|
||||
Scaling prompts for confirmation because it creates a revision, may interrupt
|
||||
service, and may change billing. Pass `--yes` to confirm without prompting, with
|
||||
credentials and workspace configured for non-interactive use. The command reports
|
||||
that scaling was requested; it does not wait for the revision to become healthy.
|
||||
Arbitrary worker replica overrides are not supported by this command.
|
||||
|
||||
## Configuration
|
||||
|
||||
The CLI uses a `langgraph.json` configuration file with these key settings:
|
||||
|
||||
@@ -11,8 +11,9 @@ import time
|
||||
from collections.abc import Callable, Mapping, Sequence
|
||||
from contextlib import contextmanager
|
||||
from dataclasses import asdict, dataclass, field
|
||||
from datetime import datetime, timezone
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from typing import Generic, Protocol, TypeVar
|
||||
from uuid import UUID
|
||||
|
||||
import click
|
||||
import click.exceptions
|
||||
@@ -23,6 +24,7 @@ from langgraph_cli.analytics import log_command
|
||||
from langgraph_cli.config import Config
|
||||
from langgraph_cli.constants import DEFAULT_CONFIG
|
||||
from langgraph_cli.dependency_tracking import find_tracked_packages
|
||||
from langgraph_cli.deployment_metrics import compact_deployment_metrics
|
||||
from langgraph_cli.docker import build_docker_image, can_build_locally
|
||||
from langgraph_cli.exec import CommandRunner, Runner, subp_exec
|
||||
from langgraph_cli.host_backend import (
|
||||
@@ -2635,6 +2637,144 @@ def deploy_list(
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@OPT_HOST_API_KEY
|
||||
@OPT_HOST_URL
|
||||
@click.argument("deployment_id", type=click.UUID)
|
||||
@click.option(
|
||||
"--start-time",
|
||||
help="ISO8601 start time. Defaults to 15 minutes before --end-time.",
|
||||
)
|
||||
@click.option("--end-time", help="ISO8601 end time. Defaults to now.")
|
||||
@click.option(
|
||||
"--metrics",
|
||||
help=(
|
||||
"Comma-separated metric names. Defaults to all metrics. Supported names: "
|
||||
"memory_usage, cpu_usage, disk_usage, restart_count, replica_count, "
|
||||
"worker_count, lg_run_count, responses_per_second, "
|
||||
"error_responses_per_second, p95_latency, run_queue_wait_time. "
|
||||
"Availability depends on the deployment and platform."
|
||||
),
|
||||
)
|
||||
@click.option(
|
||||
"--summary", is_flag=True, help="Omit points and print per-series statistics only."
|
||||
)
|
||||
@click.option(
|
||||
"--raw", is_flag=True, help="Print the original host-backend chart schema."
|
||||
)
|
||||
@deploy.command(
|
||||
"metrics", help="[Beta] Fetch deployment monitoring time series as compact JSON."
|
||||
)
|
||||
def deploy_metrics(
|
||||
api_key: str | None,
|
||||
host_url: str | None,
|
||||
deployment_id: UUID,
|
||||
start_time: str | None,
|
||||
end_time: str | None,
|
||||
metrics: str | None,
|
||||
summary: bool,
|
||||
raw: bool,
|
||||
) -> None:
|
||||
if summary and raw:
|
||||
raise click.UsageError("--summary and --raw are mutually exclusive.")
|
||||
end_time = end_time or datetime.now(timezone.utc).isoformat()
|
||||
if start_time is None:
|
||||
try:
|
||||
end = datetime.fromisoformat(end_time.replace("Z", "+00:00"))
|
||||
except ValueError as exc:
|
||||
raise click.BadParameter(
|
||||
"Expected an ISO8601 timestamp.", param_hint="--end-time"
|
||||
) from exc
|
||||
end = end.replace(tzinfo=end.tzinfo or timezone.utc).astimezone(timezone.utc)
|
||||
start_time = (end - timedelta(minutes=15)).isoformat()
|
||||
params = {
|
||||
key: value
|
||||
for key, value in {
|
||||
"start_time": start_time,
|
||||
"end_time": end_time,
|
||||
"metrics": metrics,
|
||||
}.items()
|
||||
if value is not None
|
||||
}
|
||||
client = _create_host_backend_client(host_url, api_key, no_input=True)
|
||||
result = _call_host_backend_with_optional_tenant(
|
||||
client,
|
||||
lambda c: c.get_deployment_metrics(str(deployment_id), params),
|
||||
no_input=True,
|
||||
)
|
||||
if not raw:
|
||||
result = compact_deployment_metrics(
|
||||
result,
|
||||
str(deployment_id),
|
||||
start_time=start_time,
|
||||
end_time=end_time,
|
||||
summary=summary,
|
||||
)
|
||||
click.echo(json_mod.dumps(result, separators=(",", ":")))
|
||||
|
||||
|
||||
@OPT_HOST_API_KEY
|
||||
@OPT_HOST_URL
|
||||
@click.argument("deployment_id", type=click.UUID)
|
||||
@click.option(
|
||||
"--compute-tier",
|
||||
type=click.Choice(
|
||||
[
|
||||
"SERVERLESS_S",
|
||||
"SERVERLESS_M",
|
||||
"SERVERLESS_L",
|
||||
"DEDICATED_S",
|
||||
"DEDICATED_M",
|
||||
"DEDICATED_L",
|
||||
]
|
||||
),
|
||||
help="Compute resource tier. Omit to leave compute unchanged.",
|
||||
)
|
||||
@click.option(
|
||||
"--database-tier",
|
||||
type=click.Choice(["DEDICATED_S", "DEDICATED_M", "DEDICATED_L"]),
|
||||
help="Postgres resource tier for dedicated deployments. Omit to leave unchanged.",
|
||||
)
|
||||
@click.option(
|
||||
"--yes", is_flag=True, help="Confirm the scaling change without prompting."
|
||||
)
|
||||
@deploy.command(
|
||||
"scale",
|
||||
help="[Beta] Change compute or Postgres tiers. Creates a revision and may change billing.",
|
||||
)
|
||||
def deploy_scale(
|
||||
api_key: str | None,
|
||||
host_url: str | None,
|
||||
deployment_id: UUID,
|
||||
compute_tier: str | None,
|
||||
database_tier: str | None,
|
||||
yes: bool,
|
||||
) -> None:
|
||||
payload = {
|
||||
key: value
|
||||
for key, value in {
|
||||
"compute_tier": compute_tier,
|
||||
"database_tier": database_tier,
|
||||
}.items()
|
||||
if value is not None
|
||||
}
|
||||
if not payload:
|
||||
raise click.UsageError("Specify --compute-tier or --database-tier.")
|
||||
if not yes:
|
||||
changes = ", ".join(f"{key}={value}" for key, value in payload.items())
|
||||
click.confirm(
|
||||
f"Scale deployment {deployment_id} ({changes})? "
|
||||
"This creates a revision and may change billing or interrupt service.",
|
||||
abort=True,
|
||||
)
|
||||
client = _create_host_backend_client(host_url, api_key, no_input=yes)
|
||||
_call_host_backend_with_optional_tenant(
|
||||
client,
|
||||
lambda c: c.scale_deployment(str(deployment_id), payload),
|
||||
no_input=yes,
|
||||
)
|
||||
click.echo(f"Scaling requested for deployment {deployment_id}.")
|
||||
|
||||
|
||||
@deploy.group(
|
||||
"listeners",
|
||||
cls=NestedHelpGroup,
|
||||
|
||||
@@ -0,0 +1,95 @@
|
||||
"""Compact deployment chart responses for command-line consumers."""
|
||||
|
||||
import math
|
||||
from collections import Counter, defaultdict
|
||||
from datetime import datetime, timezone
|
||||
from statistics import fmean
|
||||
from typing import Any
|
||||
|
||||
|
||||
def _timestamp(value: str) -> datetime:
|
||||
timestamp = datetime.fromisoformat(value.replace("Z", "+00:00"))
|
||||
return timestamp.replace(tzinfo=timestamp.tzinfo or timezone.utc).astimezone(
|
||||
timezone.utc
|
||||
)
|
||||
|
||||
|
||||
def _iso(value: datetime) -> str:
|
||||
return value.isoformat().replace("+00:00", "Z")
|
||||
|
||||
|
||||
def _unit(title: str, metric: str | None) -> str | None:
|
||||
if "%" in title:
|
||||
return "percent"
|
||||
if metric in {
|
||||
"restart_count",
|
||||
"replica_count",
|
||||
"worker_count",
|
||||
"lg_run_count",
|
||||
"run_count",
|
||||
}:
|
||||
return "count"
|
||||
return None
|
||||
|
||||
|
||||
def compact_deployment_metrics(
|
||||
response: dict[str, Any],
|
||||
deployment_id: str,
|
||||
*,
|
||||
start_time: str,
|
||||
end_time: str,
|
||||
summary: bool = False,
|
||||
) -> dict[str, Any]:
|
||||
"""Reshape UI charts without mutating the response; step is in seconds."""
|
||||
sections = {}
|
||||
spacings: Counter[float] = Counter()
|
||||
source_sections = list(response.get("sub_sections") or [])
|
||||
if response.get("charts"):
|
||||
source_sections.insert(0, response)
|
||||
for section in source_sections:
|
||||
charts = {}
|
||||
for chart in section.get("charts") or []:
|
||||
by_series = defaultdict(list)
|
||||
for point in chart.get("data") or []:
|
||||
timestamp = _timestamp(point["timestamp"])
|
||||
value = point.get("value")
|
||||
if value is not None and not math.isfinite(value):
|
||||
value = None
|
||||
by_series[point["series_id"]].append((timestamp, value))
|
||||
series_output = {}
|
||||
for series in chart.get("series") or []:
|
||||
points = sorted(by_series[series["id"]], key=lambda point: point[0])
|
||||
times = sorted({timestamp for timestamp, _ in points})
|
||||
spacings.update(
|
||||
(b - a).total_seconds()
|
||||
for a, b in zip(times, times[1:], strict=False)
|
||||
)
|
||||
values = [value for _, value in points if value is not None]
|
||||
metric = series.get("project_metric")
|
||||
stats = {
|
||||
"metric": metric,
|
||||
"unit": _unit(chart["title"], metric),
|
||||
"latest": round(values[-1], 3) if values else None,
|
||||
"min": round(min(values), 3) if values else None,
|
||||
"max": round(max(values), 3) if values else None,
|
||||
"avg": round(fmean(values), 3) if values else None,
|
||||
}
|
||||
if not summary:
|
||||
stats["points"] = [
|
||||
[
|
||||
_iso(timestamp),
|
||||
round(value, 3) if value is not None else None,
|
||||
]
|
||||
for timestamp, value in points
|
||||
]
|
||||
series_output[series["name"]] = stats
|
||||
charts[chart["title"]] = series_output or {"_empty": True}
|
||||
sections[section["title"]] = charts or {"_empty": True}
|
||||
step = min(spacings, key=lambda gap: (-spacings[gap], gap)) if spacings else None
|
||||
return {
|
||||
"deployment_id": deployment_id,
|
||||
"start_time": start_time,
|
||||
"end_time": end_time,
|
||||
"step_seconds": int(step) if step is not None and step.is_integer() else step,
|
||||
"sections": sections or {"_empty": True},
|
||||
}
|
||||
@@ -362,6 +362,18 @@ class HostBackendClient:
|
||||
f"/v2/deployments/{deployment_id}/revisions/{revision_id}",
|
||||
)
|
||||
|
||||
def get_deployment_metrics(
|
||||
self, deployment_id: str, params: dict[str, Any]
|
||||
) -> dict[str, Any]:
|
||||
return self._request(
|
||||
"GET", f"/v2/deployments/{deployment_id}/metrics", params=params
|
||||
)
|
||||
|
||||
def scale_deployment(self, deployment_id: str, payload: dict[str, str]) -> None:
|
||||
self._request(
|
||||
"PATCH", f"/v2/deployments/{deployment_id}/resource-tiers", payload
|
||||
)
|
||||
|
||||
def get_build_logs(
|
||||
self, project_id: str, revision_id: str, payload: dict[str, Any]
|
||||
) -> Any:
|
||||
|
||||
@@ -0,0 +1,215 @@
|
||||
import json
|
||||
from datetime import datetime, timezone
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
from click.testing import CliRunner
|
||||
|
||||
import langgraph_cli.deploy as deploy_module
|
||||
from langgraph_cli.cli import cli
|
||||
from langgraph_cli.deployment_metrics import compact_deployment_metrics
|
||||
from langgraph_cli.host_backend import HostBackendClient
|
||||
|
||||
DEPLOYMENT_ID = "019a0000-0000-7000-8000-000000000001"
|
||||
|
||||
|
||||
def invoke(monkeypatch, args, handler, input=None):
|
||||
client = HostBackendClient(
|
||||
"https://api.example.com",
|
||||
"key",
|
||||
tenant_id="workspace-id",
|
||||
transport=httpx.MockTransport(handler),
|
||||
)
|
||||
monkeypatch.setattr(deploy_module, "HostBackendClient", lambda *a, **kw: client)
|
||||
return CliRunner().invoke(
|
||||
cli,
|
||||
["deploy", *args, "--api-key", "key"],
|
||||
input=input,
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"bounds", [[], ["start_time"], ["end_time"], ["start_time", "end_time"]]
|
||||
)
|
||||
@pytest.mark.parametrize("mode", [None, "--summary", "--raw"])
|
||||
def test_metrics(monkeypatch, bounds, mode, deployment_metrics_response):
|
||||
charts = deployment_metrics_response
|
||||
|
||||
class FrozenDatetime(datetime):
|
||||
@classmethod
|
||||
def now(cls, tz=None):
|
||||
return cls(2026, 10, 28, 23, 2, 33, tzinfo=timezone.utc)
|
||||
|
||||
monkeypatch.setattr(deploy_module, "datetime", FrozenDatetime)
|
||||
explicit = {
|
||||
"start_time": "2026-10-27T22:32:33Z",
|
||||
"end_time": "2026-10-28T21:32:33Z",
|
||||
}
|
||||
options = {key: explicit[key] for key in bounds}
|
||||
if bounds:
|
||||
options["metrics"] = "cpu_usage,memory_usage"
|
||||
params = {
|
||||
"start_time": (
|
||||
"2026-10-28T21:17:33+00:00"
|
||||
if "end_time" in bounds
|
||||
else "2026-10-28T22:47:33+00:00"
|
||||
),
|
||||
"end_time": "2026-10-28T23:02:33+00:00",
|
||||
**options,
|
||||
}
|
||||
|
||||
def handler(request):
|
||||
assert request.method == "GET"
|
||||
assert request.url.path == f"/v2/deployments/{DEPLOYMENT_ID}/metrics"
|
||||
assert dict(request.url.params) == params
|
||||
assert request.headers["x-tenant-id"] == "workspace-id"
|
||||
assert request.headers["x-api-key"] == "key"
|
||||
return httpx.Response(200, json=charts)
|
||||
|
||||
args = ["metrics", DEPLOYMENT_ID]
|
||||
for key, value in options.items():
|
||||
args.extend([f"--{key.replace('_', '-')}", value])
|
||||
if mode:
|
||||
args.append(mode)
|
||||
result = invoke(monkeypatch, args, handler)
|
||||
assert result.exit_code == 0, result.output
|
||||
expected = (
|
||||
charts
|
||||
if mode == "--raw"
|
||||
else compact_deployment_metrics(
|
||||
charts,
|
||||
DEPLOYMENT_ID,
|
||||
start_time=params.get("start_time"),
|
||||
end_time=params.get("end_time"),
|
||||
summary=mode == "--summary",
|
||||
)
|
||||
)
|
||||
assert json.loads(result.output) == expected
|
||||
if mode != "--raw":
|
||||
assert expected["start_time"] == params["start_time"]
|
||||
assert expected["end_time"] == params["end_time"]
|
||||
assert result.output.strip() == json.dumps(expected, separators=(",", ":"))
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"tiers",
|
||||
[
|
||||
{"compute_tier": "SERVERLESS_M"},
|
||||
{"database_tier": "DEDICATED_L"},
|
||||
{"compute_tier": "DEDICATED_M", "database_tier": "DEDICATED_L"},
|
||||
],
|
||||
)
|
||||
def test_scale(monkeypatch, tiers):
|
||||
def handler(request):
|
||||
assert request.method == "PATCH"
|
||||
assert request.url.path == f"/v2/deployments/{DEPLOYMENT_ID}/resource-tiers"
|
||||
assert request.headers["x-tenant-id"] == "workspace-id"
|
||||
assert json.loads(request.content) == tiers
|
||||
return httpx.Response(200, json={"secrets": [{"value": "secret-value"}]})
|
||||
|
||||
args = ["scale", DEPLOYMENT_ID, "--yes"]
|
||||
for key, value in tiers.items():
|
||||
args.extend([f"--{key.replace('_', '-')}", value])
|
||||
result = invoke(monkeypatch, args, handler)
|
||||
assert result.exit_code == 0, result.output
|
||||
assert "Scaling requested" in result.output
|
||||
assert "secret-value" not in result.output
|
||||
|
||||
|
||||
@pytest.mark.parametrize("answer,expected_code", [("y\n", 0), ("n\n", 1)])
|
||||
def test_scale_confirmation(monkeypatch, answer, expected_code):
|
||||
requests = []
|
||||
|
||||
def handler(request):
|
||||
requests.append(request)
|
||||
return httpx.Response(200, json={})
|
||||
|
||||
result = invoke(
|
||||
monkeypatch,
|
||||
["scale", DEPLOYMENT_ID, "--compute-tier", "DEDICATED_S"],
|
||||
handler,
|
||||
input=answer,
|
||||
)
|
||||
assert result.exit_code == expected_code, result.output
|
||||
assert len(requests) == (1 if expected_code == 0 else 0)
|
||||
assert "may change billing or interrupt service" in result.output
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"args",
|
||||
[
|
||||
["scale", DEPLOYMENT_ID, "--yes"],
|
||||
["scale", DEPLOYMENT_ID, "--database-tier", "SERVERLESS_M", "--yes"],
|
||||
["metrics", DEPLOYMENT_ID, "--step-seconds", "60"],
|
||||
["metrics", "invalid-id"],
|
||||
["metrics", DEPLOYMENT_ID, "--summary", "--raw"],
|
||||
],
|
||||
)
|
||||
def test_invalid_options_do_not_send_requests(monkeypatch, args):
|
||||
def handler(request):
|
||||
pytest.fail("Invalid options must not send a request")
|
||||
|
||||
result = invoke(monkeypatch, args, handler)
|
||||
assert result.exit_code == 2, result.output
|
||||
|
||||
|
||||
def test_end_only_with_timezone_offset(monkeypatch):
|
||||
def handler(request):
|
||||
assert request.url.params["end_time"] == "2026-10-28T01:05:00+02:00"
|
||||
assert request.url.params["start_time"] == "2026-10-27T22:50:00+00:00"
|
||||
return httpx.Response(200, json={"charts": [], "sub_sections": []})
|
||||
|
||||
result = invoke(
|
||||
monkeypatch,
|
||||
["metrics", DEPLOYMENT_ID, "--end-time", "2026-10-28T01:05:00+02:00"],
|
||||
handler,
|
||||
)
|
||||
assert result.exit_code == 0, result.output
|
||||
output = json.loads(result.output)
|
||||
assert output["start_time"] == "2026-10-27T22:50:00+00:00"
|
||||
assert output["end_time"] == "2026-10-28T01:05:00+02:00"
|
||||
|
||||
|
||||
def test_invalid_end_only_does_not_send_request(monkeypatch):
|
||||
def handler(request):
|
||||
pytest.fail("Invalid end time must not send a request")
|
||||
|
||||
result = invoke(
|
||||
monkeypatch, ["metrics", DEPLOYMENT_ID, "--end-time", "not-a-date"], handler
|
||||
)
|
||||
assert result.exit_code == 2, result.output
|
||||
assert "--end-time" in result.output
|
||||
assert "ISO8601" in result.output
|
||||
|
||||
|
||||
def test_metrics_help_lists_supported_filters():
|
||||
result = CliRunner().invoke(cli, ["deploy", "metrics", "--help"])
|
||||
assert result.exit_code == 0, result.output
|
||||
for metric in (
|
||||
"memory_usage",
|
||||
"cpu_usage",
|
||||
"disk_usage",
|
||||
"restart_count",
|
||||
"replica_count",
|
||||
"worker_count",
|
||||
"lg_run_count",
|
||||
"responses_per_second",
|
||||
"error_responses_per_second",
|
||||
"p95_latency",
|
||||
"run_queue_wait_time",
|
||||
):
|
||||
assert metric in result.output
|
||||
|
||||
|
||||
@pytest.mark.parametrize("command", ["metrics", "scale"])
|
||||
@pytest.mark.parametrize("status", [403, 422, 429])
|
||||
def test_backend_errors(monkeypatch, command, status):
|
||||
def handler(request):
|
||||
return httpx.Response(status, json={"detail": "Operation rejected"})
|
||||
|
||||
args = [command, DEPLOYMENT_ID]
|
||||
if command == "scale":
|
||||
args.extend(["--compute-tier", "DEDICATED_M", "--yes"])
|
||||
result = invoke(monkeypatch, args, handler)
|
||||
assert result.exit_code == 1
|
||||
assert "Operation rejected" in result.output
|
||||
@@ -1,9 +1,16 @@
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def deployment_metrics_response():
|
||||
return json.loads(Path(__file__).with_name("deployment_metrics.json").read_text())
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def disable_analytics_env() -> None:
|
||||
"""Disable analytics for unit tests LANGGRAPH_CLI_NO_ANALYTICS."""
|
||||
|
||||
@@ -0,0 +1,215 @@
|
||||
{
|
||||
"id": "019a0000-0000-7000-8000-000000000001-metrics",
|
||||
"title": "Metrics for deployment",
|
||||
"charts": [],
|
||||
"sub_sections": [
|
||||
{
|
||||
"id": "019a0000-0000-7000-8000-000000000001-queue",
|
||||
"title": "Queue",
|
||||
"description": "Queue metrics",
|
||||
"index": 0,
|
||||
"charts": [
|
||||
{
|
||||
"id": "019a0000-0000-7000-8000-000000000001-Memory Usage %",
|
||||
"title": "Memory Usage %",
|
||||
"description": "Resource usage over time",
|
||||
"index": 0,
|
||||
"chart_type": "line",
|
||||
"series": [
|
||||
{
|
||||
"id": "019a0000-0000-7000-8000-000000000001-Avg Memory Usage %",
|
||||
"name": "Avg Memory Usage %",
|
||||
"metric": "run_count",
|
||||
"project_metric": "memory_usage",
|
||||
"metadata": null,
|
||||
"filters": null,
|
||||
"feedback_key": null,
|
||||
"workspace_id": null,
|
||||
"metric_definition": null,
|
||||
"group_by_definitions": null,
|
||||
"filter_definition": null,
|
||||
"group_by": null
|
||||
},
|
||||
{
|
||||
"id": "019a0000-0000-7000-8000-000000000001-Max Memory Usage %",
|
||||
"name": "Max Memory Usage %",
|
||||
"metric": "run_count",
|
||||
"project_metric": "memory_usage",
|
||||
"metadata": null,
|
||||
"filters": null,
|
||||
"feedback_key": null,
|
||||
"workspace_id": null,
|
||||
"metric_definition": null,
|
||||
"group_by_definitions": null,
|
||||
"filter_definition": null,
|
||||
"group_by": null
|
||||
}
|
||||
],
|
||||
"data": [
|
||||
{
|
||||
"series_id": "019a0000-0000-7000-8000-000000000001-Max Memory Usage %",
|
||||
"timestamp": "2026-10-28T00:02:00+00:00",
|
||||
"value": 50.33333,
|
||||
"group": null
|
||||
},
|
||||
{
|
||||
"series_id": "019a0000-0000-7000-8000-000000000001-Max Memory Usage %",
|
||||
"timestamp": "2026-10-28T00:01:00+00:00",
|
||||
"value": 40.22222,
|
||||
"group": null
|
||||
},
|
||||
{
|
||||
"series_id": "019a0000-0000-7000-8000-000000000001-Max Memory Usage %",
|
||||
"timestamp": "2026-10-28T00:00:00+00:00",
|
||||
"value": 30.11111,
|
||||
"group": null
|
||||
},
|
||||
{
|
||||
"series_id": "019a0000-0000-7000-8000-000000000001-Avg Memory Usage %",
|
||||
"timestamp": "2026-10-28T00:02:00+00:00",
|
||||
"value": null,
|
||||
"group": null
|
||||
},
|
||||
{
|
||||
"series_id": "019a0000-0000-7000-8000-000000000001-Avg Memory Usage %",
|
||||
"timestamp": "2026-10-28T00:01:00+00:00",
|
||||
"value": 20.23456,
|
||||
"group": null
|
||||
},
|
||||
{
|
||||
"series_id": "019a0000-0000-7000-8000-000000000001-Avg Memory Usage %",
|
||||
"timestamp": "2026-10-28T00:00:00+00:00",
|
||||
"value": 10.12345,
|
||||
"group": null
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "019a0000-0000-7000-8000-000000000001-Restarts",
|
||||
"title": "Restarts",
|
||||
"description": "Resource usage over time",
|
||||
"index": 0,
|
||||
"chart_type": "line",
|
||||
"series": [
|
||||
{
|
||||
"id": "019a0000-0000-7000-8000-000000000001-Restarts",
|
||||
"name": "Restarts",
|
||||
"metric": "run_count",
|
||||
"project_metric": "restart_count",
|
||||
"metadata": null,
|
||||
"filters": null,
|
||||
"feedback_key": null,
|
||||
"workspace_id": null,
|
||||
"metric_definition": null,
|
||||
"group_by_definitions": null,
|
||||
"filter_definition": null,
|
||||
"group_by": null
|
||||
}
|
||||
],
|
||||
"data": [
|
||||
{
|
||||
"series_id": "019a0000-0000-7000-8000-000000000001-Restarts",
|
||||
"timestamp": "2026-10-28T00:02:00+00:00",
|
||||
"value": 2,
|
||||
"group": null
|
||||
},
|
||||
{
|
||||
"series_id": "019a0000-0000-7000-8000-000000000001-Restarts",
|
||||
"timestamp": "2026-10-28T00:01:00+00:00",
|
||||
"value": 1,
|
||||
"group": null
|
||||
},
|
||||
{
|
||||
"series_id": "019a0000-0000-7000-8000-000000000001-Restarts",
|
||||
"timestamp": "2026-10-28T00:00:00+00:00",
|
||||
"value": 0,
|
||||
"group": null
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "019a0000-0000-7000-8000-000000000001-Queue wait time",
|
||||
"title": "Queue wait time",
|
||||
"description": "Resource usage over time",
|
||||
"index": 0,
|
||||
"chart_type": "line",
|
||||
"series": [
|
||||
{
|
||||
"id": "019a0000-0000-7000-8000-000000000001-Queue wait time",
|
||||
"name": "Queue wait time",
|
||||
"metric": "run_count",
|
||||
"project_metric": "run_queue_wait_time",
|
||||
"metadata": null,
|
||||
"filters": null,
|
||||
"feedback_key": null,
|
||||
"workspace_id": null,
|
||||
"metric_definition": null,
|
||||
"group_by_definitions": null,
|
||||
"filter_definition": null,
|
||||
"group_by": null
|
||||
}
|
||||
],
|
||||
"data": []
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "019a0000-0000-7000-8000-000000000001-api",
|
||||
"title": "API",
|
||||
"description": "API metrics",
|
||||
"index": 1,
|
||||
"charts": [
|
||||
{
|
||||
"id": "019a0000-0000-7000-8000-000000000001-Latency",
|
||||
"title": "Latency",
|
||||
"description": "Resource usage over time",
|
||||
"index": 0,
|
||||
"chart_type": "line",
|
||||
"series": [
|
||||
{
|
||||
"id": "019a0000-0000-7000-8000-000000000001-P95 Latency",
|
||||
"name": "P95 Latency",
|
||||
"metric": "run_count",
|
||||
"project_metric": "p95_latency",
|
||||
"metadata": null,
|
||||
"filters": null,
|
||||
"feedback_key": null,
|
||||
"workspace_id": null,
|
||||
"metric_definition": null,
|
||||
"group_by_definitions": null,
|
||||
"filter_definition": null,
|
||||
"group_by": null
|
||||
}
|
||||
],
|
||||
"data": [
|
||||
{
|
||||
"series_id": "019a0000-0000-7000-8000-000000000001-P95 Latency",
|
||||
"timestamp": "2026-10-28T00:02:00+00:00",
|
||||
"value": 0.34567,
|
||||
"group": null
|
||||
},
|
||||
{
|
||||
"series_id": "019a0000-0000-7000-8000-000000000001-P95 Latency",
|
||||
"timestamp": "2026-10-28T00:01:00+00:00",
|
||||
"value": 0.23456,
|
||||
"group": null
|
||||
},
|
||||
{
|
||||
"series_id": "019a0000-0000-7000-8000-000000000001-P95 Latency",
|
||||
"timestamp": "2026-10-28T00:00:00+00:00",
|
||||
"value": 0.12345,
|
||||
"group": null
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "019a0000-0000-7000-8000-000000000001-pg",
|
||||
"title": "Postgres",
|
||||
"description": "Postgres metrics",
|
||||
"index": 2,
|
||||
"charts": []
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,139 @@
|
||||
from copy import deepcopy
|
||||
|
||||
import pytest
|
||||
|
||||
from langgraph_cli.deployment_metrics import compact_deployment_metrics
|
||||
|
||||
|
||||
def reshape(response, deployment_id, **kwargs):
|
||||
return compact_deployment_metrics(
|
||||
response,
|
||||
deployment_id,
|
||||
start_time=kwargs.pop("start_time", "2026-10-28T00:00:00Z"),
|
||||
end_time=kwargs.pop("end_time", "2026-10-28T01:00:00Z"),
|
||||
**kwargs,
|
||||
)
|
||||
|
||||
|
||||
def test_compact_metrics(deployment_metrics_response):
|
||||
original = deepcopy(deployment_metrics_response)
|
||||
result = reshape(deployment_metrics_response, "deployment-id")
|
||||
assert deployment_metrics_response == original
|
||||
assert result["deployment_id"] == "deployment-id"
|
||||
assert result["start_time"] == "2026-10-28T00:00:00Z"
|
||||
assert result["end_time"] == "2026-10-28T01:00:00Z"
|
||||
assert result["step_seconds"] == 60
|
||||
assert result["sections"]["Postgres"] == {"_empty": True}
|
||||
queue = result["sections"]["Queue"]
|
||||
memory = queue["Memory Usage %"]
|
||||
assert memory["Avg Memory Usage %"] == {
|
||||
"metric": "memory_usage",
|
||||
"unit": "percent",
|
||||
"latest": 20.235,
|
||||
"min": 10.123,
|
||||
"max": 20.235,
|
||||
"avg": 15.179,
|
||||
"points": [
|
||||
["2026-10-28T00:00:00Z", 10.123],
|
||||
["2026-10-28T00:01:00Z", 20.235],
|
||||
["2026-10-28T00:02:00Z", None],
|
||||
],
|
||||
}
|
||||
assert memory["Max Memory Usage %"]["max"] == 50.333
|
||||
assert queue["Restarts"]["Restarts"]["unit"] == "count"
|
||||
assert queue["Queue wait time"]["Queue wait time"] == {
|
||||
"metric": "run_queue_wait_time",
|
||||
"unit": None,
|
||||
"latest": None,
|
||||
"min": None,
|
||||
"max": None,
|
||||
"avg": None,
|
||||
"points": [],
|
||||
}
|
||||
assert result["sections"]["API"]["Latency"]["P95 Latency"]["unit"] is None
|
||||
|
||||
|
||||
def test_summary_and_requested_bounds(deployment_metrics_response):
|
||||
result = reshape(
|
||||
deployment_metrics_response,
|
||||
"id",
|
||||
start_time="2026-10-27T23:00:00Z",
|
||||
end_time="2026-10-28T01:00:00Z",
|
||||
summary=True,
|
||||
)
|
||||
assert result["start_time"] == "2026-10-27T23:00:00Z"
|
||||
assert result["end_time"] == "2026-10-28T01:00:00Z"
|
||||
for section in result["sections"].values():
|
||||
if section == {"_empty": True}:
|
||||
continue
|
||||
for chart in section.values():
|
||||
for series in chart.values():
|
||||
assert set(series) == {"metric", "unit", "latest", "min", "max", "avg"}
|
||||
|
||||
|
||||
@pytest.mark.parametrize("values", [[], [None], [float("nan"), float("inf")]])
|
||||
def test_empty_and_nonfinite_values(deployment_metrics_response, values):
|
||||
chart = deployment_metrics_response["sub_sections"][0]["charts"][0]
|
||||
chart["series"] = chart["series"][:1]
|
||||
chart["data"] = [
|
||||
{
|
||||
"series_id": chart["series"][0]["id"],
|
||||
"timestamp": f"2026-10-28T00:0{i}:00Z",
|
||||
"value": value,
|
||||
}
|
||||
for i, value in enumerate(values)
|
||||
]
|
||||
result = reshape(deployment_metrics_response, "id")
|
||||
series = result["sections"]["Queue"]["Memory Usage %"]["Avg Memory Usage %"]
|
||||
assert all(series[key] is None for key in ["latest", "min", "max", "avg"])
|
||||
assert all(point[1] is None for point in series["points"])
|
||||
|
||||
|
||||
def test_tied_spacing_and_top_level_charts(deployment_metrics_response):
|
||||
response = deployment_metrics_response
|
||||
chart = response["sub_sections"][1]["charts"][0]
|
||||
chart["data"][0]["timestamp"] = "2026-10-28T00:02:30Z"
|
||||
response["charts"] = [chart]
|
||||
response["sub_sections"] = []
|
||||
result = reshape(response, "id")
|
||||
assert result["step_seconds"] == 60
|
||||
assert "Latency" in result["sections"][response["title"]]
|
||||
|
||||
|
||||
def test_missing_point_preserves_common_spacing(deployment_metrics_response):
|
||||
chart = deployment_metrics_response["sub_sections"][0]["charts"][0]
|
||||
series_id = chart["series"][0]["id"]
|
||||
chart["data"] = [
|
||||
point
|
||||
for point in chart["data"]
|
||||
if not (point["series_id"] == series_id and "00:01:00" in point["timestamp"])
|
||||
]
|
||||
result = reshape(deployment_metrics_response, "id")
|
||||
assert result["step_seconds"] == 60
|
||||
assert type(result["step_seconds"]) is int
|
||||
|
||||
|
||||
def test_only_single_point_and_empty_series(deployment_metrics_response):
|
||||
for section in deployment_metrics_response["sub_sections"]:
|
||||
for chart in section["charts"]:
|
||||
chart["data"] = chart["data"][:1]
|
||||
assert reshape(deployment_metrics_response, "id")["step_seconds"] is None
|
||||
|
||||
|
||||
def test_fractional_spacing(deployment_metrics_response):
|
||||
chart = deployment_metrics_response["sub_sections"][1]["charts"][0]
|
||||
chart["data"][0]["timestamp"] = "2026-10-28T00:00:01Z"
|
||||
chart["data"][1]["timestamp"] = "2026-10-28T00:00:00.500Z"
|
||||
deployment_metrics_response["sub_sections"] = []
|
||||
deployment_metrics_response["charts"] = [chart]
|
||||
assert reshape(deployment_metrics_response, "id")["step_seconds"] == 0.5
|
||||
|
||||
|
||||
def test_no_data():
|
||||
assert reshape({"charts": [], "sub_sections": []}, "id") == {
|
||||
"deployment_id": "id",
|
||||
"start_time": "2026-10-28T00:00:00Z",
|
||||
"end_time": "2026-10-28T01:00:00Z",
|
||||
"step_seconds": None,
|
||||
"sections": {"_empty": True},
|
||||
}
|
||||
Reference in new issue
Block a user