mirror of
https://github.com/langchain-ai/langgraph.git
synced 2026-08-23 08:02:23 +02:00
William FHGitHubWill Fu-HinthornClaude Opus 4.7copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>## Summary
Adds per-node `timeout` support to async StateGraph/Pregel nodes and to
the functional API (`@task` / `@entrypoint`). A timeout caps how long a
single node attempt may run, either as a hard wall-clock budget
(`run_timeout`), or as an idle window that resets on observable progress
(`idle_timeout`), or both. When exceeded, LangGraph raises
`NodeTimeoutError`, clears writes from the failed attempt, and lets the
existing retry policy decide whether to retry.
## Public API
A single `timeout=` kwarg on `add_node`, `@task`, `@entrypoint`, and
`NodeBuilder.set_timeout`. Pass a number/`timedelta` for the simple case
(treated as a hard wall-clock cap), or a `TimeoutPolicy` (in
`langgraph.types`) for finer control:
```python
from datetime import timedelta
from langgraph.types import TimeoutPolicy
# simple: hard wall-clock cap on each attempt
builder.add_node("call_model", call_model, timeout=60)
builder.add_node("call_model", call_model, timeout=timedelta(minutes=2))
# full control
builder.add_node(
"call_model",
call_model,
timeout=TimeoutPolicy(
run_timeout=120, # hard wall-clock cap in seconds, never refreshed
idle_timeout=30, # cap on time without observable progress, units in seconds
refresh_on="auto", # "auto" | "heartbeat"
),
)
```
- `run_timeout`: hard wall-clock cap on a single attempt; never
refreshed.
- `idle_timeout`: progress-resetting cap. Refreshed by writes, stream
output, yielded async stream chunks, child-task scheduling, runtime
stream-writer calls, and any LangChain callback event from descendants
of the node's run. `runtime.heartbeat()` is a manual signal for work
that doesn't naturally emit any of these.
- `refresh_on="heartbeat"` narrows the refresh source to explicit
`runtime.heartbeat()` only — useful when you want a strict idle
definition that isn't reset by chatty subordinates.
For long-running async work that doesn't naturally emit progress:
```python
async def call_model(state: State, runtime: Runtime) -> State:
while still_working:
...
runtime.heartbeat()
return {"messages": [response]}
```
`NodeTimeoutError` subclasses `TimeoutError` and carries `node`,
`timeout`, `run_timeout`, `idle_timeout`, `elapsed`, and `kind` (`"run"`
or `"idle"`). If the node's `retry_policy` permits `TimeoutError` it'll
retry; the timer resets per attempt.
## Why async-only
Sync Python code cannot be safely cancelled in-process, so timeouts only
apply to async nodes/tasks. Sync nodes with a `timeout` are rejected at
compile time (covers direct nodes, wrapped runnables, sequences, and
`RunnableParallel` branches); `run_with_retry` rejects them at runtime
as a safety net.
## What gets cancelled and what doesn't
When a watchdog fires:
1. The attempt scope is closed under a lock so any in-flight
`CONFIG_KEY_SEND` / stream / child-task scheduling that races with the
timeout is dropped atomically.
2. Buffered `task.writes` are cleared so pre-timeout writes from the
failed attempt don't leak into the checkpoint after a retry succeeds.
3. The background `asyncio.Task` is cancelled; its eventual exception is
drained via a done-callback so asyncio doesn't log it.
Only the watchdog's own `TimeoutError` converts to `NodeTimeoutError`,
so user-raised `asyncio.TimeoutError`, built-in `TimeoutError`, and
`NodeTimeoutError` from a child node continue to propagate unchanged.
Child tasks already scheduled before the timeout fired still complete —
they aren't part of the cancelled task's structured cancellation
surface. This is intentional and tested.
## External-watchdog hook
WARNING: THIS API IS IN ALPHA AND SUBJECT TO CHANGE.
`CONFIG_KEY_TIMED_ATTEMPT_OBSERVER` is a per-config callback that
receives lifecycle events for each timed attempt:
- `start` — fired before the proc runs, with `task_id`, `task_name`,
`attempt`, `run_id`, `thread_id`, `checkpoint_ns`, `started_at`, and the
configured `run_timeout_secs` / `idle_timeout_secs` / `refresh_on`.
- `progress` — fired on each progress signal that resets the idle clock,
rate-limited to ~4 events per `idle_timeout` window so token-rate
callbacks don't flood the observer. Carries the same context plus
`progress_at`.
- `finish` — fired with `finished_at`, `status` (`"success"`/`"error"`),
`error_type`, `error_message`. `ParentCommand` and `GraphBubbleUp` are
treated as control flow, not errors.
This lets an orchestrating process listen to `start` + rolling
`progress` to compute its own kill deadline (`progress_at +
idle_timeout_secs`) and hard-kill a worker process if the in-process
cancellation deadlocks. Observer callbacks run in whatever thread fires
them, and any exception they raise is logged and swallowed.
## Implementation Notes
`runtime.heartbeat()` updates the progress timestamp without taking the
guarded write lock. This avoids lock overhead on high-frequency
callback/token paths and accepts a small timestamp race window, which is
negligible for expected coarse idle-timeout configurations.
## Testing
Covers timeout retry behavior, user-raised timeout propagation,
stale-write suppression, stream / callback / heartbeat progress resets,
sync-node rejection, `RunnableParallel` branch validation,
`StateGraph.add_node` behavior, lower-level Pregel behavior, observer
start/progress/finish events, and functional API compatibility.
---------
Co-authored-by: Will Fu-Hinthorn <will@langchain.dev>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
124 lines
5.1 KiB
Python
124 lines
5.1 KiB
Python
"""Constants used for Pregel operations."""
|
|
|
|
import sys
|
|
from typing import Literal, cast
|
|
|
|
# --- Reserved write keys ---
|
|
INPUT = sys.intern("__input__")
|
|
# for values passed as input to the graph
|
|
INTERRUPT = sys.intern("__interrupt__")
|
|
# for dynamic interrupts raised by nodes
|
|
RESUME = sys.intern("__resume__")
|
|
# for values passed to resume a node after an interrupt
|
|
ERROR = sys.intern("__error__")
|
|
# for errors raised by nodes
|
|
NO_WRITES = sys.intern("__no_writes__")
|
|
# marker to signal node didn't write anything
|
|
TASKS = sys.intern("__pregel_tasks")
|
|
# for Send objects returned by nodes/edges, corresponds to PUSH below
|
|
RETURN = sys.intern("__return__")
|
|
# for writes of a task where we simply record the return value
|
|
PREVIOUS = sys.intern("__previous__")
|
|
# the implicit branch that handles each node's Control values
|
|
|
|
|
|
# --- Reserved cache namespaces ---
|
|
CACHE_NS_WRITES = sys.intern("__pregel_ns_writes")
|
|
# cache namespace for node writes
|
|
|
|
# --- Reserved config.configurable keys ---
|
|
CONFIG_KEY_SEND = sys.intern("__pregel_send")
|
|
# holds the `write` function that accepts writes to state/edges/reserved keys
|
|
CONFIG_KEY_READ = sys.intern("__pregel_read")
|
|
# holds the `read` function that returns a copy of the current state
|
|
CONFIG_KEY_CALL = sys.intern("__pregel_call")
|
|
# holds the `call` function that accepts a node/func, args and returns a future
|
|
CONFIG_KEY_CHECKPOINTER = sys.intern("__pregel_checkpointer")
|
|
# holds a `BaseCheckpointSaver` passed from parent graph to child graphs
|
|
CONFIG_KEY_STREAM = sys.intern("__pregel_stream")
|
|
# holds a `StreamProtocol` passed from parent graph to child graphs
|
|
CONFIG_KEY_CACHE = sys.intern("__pregel_cache")
|
|
# holds a `BaseCache` made available to subgraphs
|
|
CONFIG_KEY_RESUMING = sys.intern("__pregel_resuming")
|
|
# holds a boolean indicating if subgraphs should resume from a previous checkpoint
|
|
CONFIG_KEY_REPLAY_STATE = sys.intern("__pregel_replay_state")
|
|
# holds a ReplayState tracking the parent checkpoint_id upper bound and which
|
|
# subgraph namespaces have already loaded their pre-replay checkpoint
|
|
CONFIG_KEY_TASK_ID = sys.intern("__pregel_task_id")
|
|
# holds the task ID for the current task
|
|
CONFIG_KEY_THREAD_ID = sys.intern("thread_id")
|
|
# holds the thread ID for the current invocation
|
|
CONFIG_KEY_CHECKPOINT_MAP = sys.intern("checkpoint_map")
|
|
# holds a mapping of checkpoint_ns -> checkpoint_id for parent graphs
|
|
CONFIG_KEY_CHECKPOINT_ID = sys.intern("checkpoint_id")
|
|
# holds the current checkpoint_id, if any
|
|
CONFIG_KEY_CHECKPOINT_NS = sys.intern("checkpoint_ns")
|
|
# holds the current checkpoint_ns, "" for root graph
|
|
CONFIG_KEY_NODE_FINISHED = sys.intern("__pregel_node_finished")
|
|
# holds a callback to be called when a node is finished
|
|
CONFIG_KEY_TIMED_ATTEMPT_OBSERVER = sys.intern("__pregel_timed_attempt_observer")
|
|
# holds a callback to be called when an idle-timed node attempt starts or finishes
|
|
CONFIG_KEY_SCRATCHPAD = sys.intern("__pregel_scratchpad")
|
|
# holds a mutable dict for temporary storage scoped to the current task
|
|
CONFIG_KEY_RUNNER_SUBMIT = sys.intern("__pregel_runner_submit")
|
|
# holds a function that receives tasks from runner, executes them and returns results
|
|
CONFIG_KEY_DURABILITY = sys.intern("__pregel_durability")
|
|
# holds the durability mode, one of "sync", "async", or "exit"
|
|
CONFIG_KEY_RUNTIME = sys.intern("__pregel_runtime")
|
|
# holds a `Runtime` instance with context, store, stream writer, etc.
|
|
CONFIG_KEY_RESUME_MAP = sys.intern("__pregel_resume_map")
|
|
# holds a mapping of task ns -> resume value for resuming tasks
|
|
CONFIG_KEY_STREAM_MESSAGES_V2 = sys.intern("__pregel_stream_messages_v2")
|
|
# when True, attach StreamMessagesHandlerV2 so content-block (v2) events
|
|
# flow through stream_mode="messages"; set by StreamingHandler only.
|
|
|
|
# --- Other constants ---
|
|
PUSH = sys.intern("__pregel_push")
|
|
# denotes push-style tasks, ie. those created by Send objects
|
|
PULL = sys.intern("__pregel_pull")
|
|
# denotes pull-style tasks, ie. those triggered by edges
|
|
NS_SEP = sys.intern("|")
|
|
# for checkpoint_ns, separates each level (ie. graph|subgraph|subsubgraph)
|
|
NS_END = sys.intern(":")
|
|
# for checkpoint_ns, for each level, separates the namespace from the task_id
|
|
CONF = cast(Literal["configurable"], sys.intern("configurable"))
|
|
# key for the configurable dict in RunnableConfig
|
|
NULL_TASK_ID = sys.intern("00000000-0000-0000-0000-000000000000")
|
|
# the task_id to use for writes that are not associated with a task
|
|
OVERWRITE = sys.intern("__overwrite__")
|
|
# dict key for the overwrite value, used as `{'__overwrite__': value}`
|
|
|
|
# redefined to avoid circular import with langgraph.constants
|
|
_TAG_HIDDEN = sys.intern("langsmith:hidden")
|
|
|
|
RESERVED = {
|
|
_TAG_HIDDEN,
|
|
# reserved write keys
|
|
INPUT,
|
|
INTERRUPT,
|
|
RESUME,
|
|
ERROR,
|
|
NO_WRITES,
|
|
# reserved config.configurable keys
|
|
CONFIG_KEY_SEND,
|
|
CONFIG_KEY_READ,
|
|
CONFIG_KEY_CHECKPOINTER,
|
|
CONFIG_KEY_STREAM,
|
|
CONFIG_KEY_CHECKPOINT_MAP,
|
|
CONFIG_KEY_RESUMING,
|
|
CONFIG_KEY_REPLAY_STATE,
|
|
CONFIG_KEY_TASK_ID,
|
|
CONFIG_KEY_CHECKPOINT_MAP,
|
|
CONFIG_KEY_CHECKPOINT_ID,
|
|
CONFIG_KEY_CHECKPOINT_NS,
|
|
CONFIG_KEY_TIMED_ATTEMPT_OBSERVER,
|
|
CONFIG_KEY_RESUME_MAP,
|
|
CONFIG_KEY_STREAM_MESSAGES_V2,
|
|
# other constants
|
|
PUSH,
|
|
PULL,
|
|
NS_SEP,
|
|
NS_END,
|
|
CONF,
|
|
}
|