## Summary
Adds cooperative drain support for Pregel runs so a graph can be asked
to stop at the next superstep boundary, persist its checkpoint, and
surface a resumable terminal exception.
- New `RunControl` (in `langgraph.runtime`) — a thread-safe handle whose
`request_drain(reason="shutdown")` sets a single flag.
- New `GraphDrained(GraphBubbleUp)` exception (in `langgraph.errors`)
raised when a run exits early due to drain. Carries the `reason` string.
- New `control: RunControl | None` kwarg on `invoke` / `ainvoke` /
`stream` / `astream` / `stream_v2` / `astream_v2`. Wired through to
`Runtime.control`, so nodes can read `runtime.control.drain_requested` /
`drain_reason` and even call `request_drain()` from inside a node.
- Stream transformers learn `"drained"` as a terminal `SubgraphStatus`.
The intended use is hooking SIGTERM (or any external supervisor signal)
to `control.request_drain("sigterm")` so an in-flight graph run can stop
cleanly and be resumed later from the saved checkpoint.
## Semantics: cooperative, between-superstep
`request_drain()` flips a flag. The Pregel loop checks it at the top of
each `tick()`, **after** the previous superstep's writes have been
applied and checkpointed. It never preempts work that is already
running.
| Scenario | Behavior |
|---|---|
| Node mid-execution (blocking I/O, sleeps, etc.) | Runs to completion.
Drain takes effect on the next superstep. |
| Node with a retry policy currently retrying | Retry loop runs to
exhaustion or success (drain is not checked between retries). Drain
takes effect on the next superstep. |
| Functional API: `@entrypoint` with pending `@task` futures |
Entrypoint and all dispatched tasks complete; drain takes effect after
the entrypoint returns. |
| Graph naturally finishes on the same tick where drain was requested
(no more tasks) | Treated as `done`; returns normally. **No
`GraphDrained` is raised.** The caller can inspect
`control.drain_requested` afterwards to distinguish a
drained-but-completed run from a normal one. |
| More tasks remain | Raises `GraphDrained(reason)`. The checkpoint of
the last completed superstep is saved (also under `durability="exit"`).
Resume with `invoke(None, config)` / `ainvoke(None, config)`. |
| Subgraph requests drain | `GraphDrained` bubbles up through the parent
loop and stops it at its own next superstep boundary; the parent's
checkpoint is saved and resumable. |
Drain does **not** cancel asyncio tasks or kill threads. Pair it with a
graceful timeout + `task.cancel()` (or process exit) if you need a hard
upper bound — see `test_drain_then_cancel_after_graceful_timeout` for
the recommended pattern.
## Usage
```python
from langgraph.runtime import RunControl
from langgraph.errors import GraphDrained
control = RunControl()
# In a signal handler, supervisor, etc.:
# control.request_drain("sigterm")
try:
result = graph.invoke(input, config, control=control)
if control.drain_requested:
# finished naturally on the same tick where drain was requested
...
except GraphDrained as e:
# checkpoint saved; resume later with the same config
log.info("graph drained: %s", e.reason)
```
## Test plan
- [x] Sync + async drain stops the next superstep
(`test_run_control_request_drain_stops_future_steps[_async]`)
- [x] Drain on the terminal step finishes normally
(`test_drain_requested_in_terminal_step_finishes_normally[_async]`)
- [x] `durability=\"exit\"` persists a resumable checkpoint on drain
(`test_drain_with_exit_durability_persists_resume_checkpoint`)
- [x] Subgraph drain bubbles up and parent resumes correctly
(`test_drain_from_subgraph_can_resume_parent`)
- [x] External thread / task triggering drain mid-run
(`test_external_drain_concurrent_sync` / `_async`)
- [x] Drain + hard cancel after graceful timeout
(`test_drain_then_cancel_after_graceful_timeout`)
- [x] Functional API: in-flight `@task` futures still resolve after
`request_drain()`
(`test_request_drain_allows_inflight_[a]call_scheduling`)
- [x] `control` kwarg wired through `stream_v2`
(`test_stream_v2_accepts_control_for_drain`)
- [x] `Runtime.merge` preserves `control`
(`test_merge_runtime_preserves_run_control`)
---------
Co-authored-by: Quanzheng Long <long@langchain.dev>
Co-authored-by: Will Fu-Hinthorn <will@langchain.dev>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
You can Send(..., timeout=...) now. Much fun.
This would allow us to do something like adding support for an
annotation to let the LLM to pick a timeout for a given tool call, etc.
---------
Co-authored-by: Will Fu-Hinthorn <will@langchain.dev>
## Summary
Adds per-node `timeout` support to async StateGraph/Pregel nodes and to
the functional API (`@task` / `@entrypoint`). A timeout caps how long a
single node attempt may run, either as a hard wall-clock budget
(`run_timeout`), or as an idle window that resets on observable progress
(`idle_timeout`), or both. When exceeded, LangGraph raises
`NodeTimeoutError`, clears writes from the failed attempt, and lets the
existing retry policy decide whether to retry.
## Public API
A single `timeout=` kwarg on `add_node`, `@task`, `@entrypoint`, and
`NodeBuilder.set_timeout`. Pass a number/`timedelta` for the simple case
(treated as a hard wall-clock cap), or a `TimeoutPolicy` (in
`langgraph.types`) for finer control:
```python
from datetime import timedelta
from langgraph.types import TimeoutPolicy
# simple: hard wall-clock cap on each attempt
builder.add_node("call_model", call_model, timeout=60)
builder.add_node("call_model", call_model, timeout=timedelta(minutes=2))
# full control
builder.add_node(
"call_model",
call_model,
timeout=TimeoutPolicy(
run_timeout=120, # hard wall-clock cap in seconds, never refreshed
idle_timeout=30, # cap on time without observable progress, units in seconds
refresh_on="auto", # "auto" | "heartbeat"
),
)
```
- `run_timeout`: hard wall-clock cap on a single attempt; never
refreshed.
- `idle_timeout`: progress-resetting cap. Refreshed by writes, stream
output, yielded async stream chunks, child-task scheduling, runtime
stream-writer calls, and any LangChain callback event from descendants
of the node's run. `runtime.heartbeat()` is a manual signal for work
that doesn't naturally emit any of these.
- `refresh_on="heartbeat"` narrows the refresh source to explicit
`runtime.heartbeat()` only — useful when you want a strict idle
definition that isn't reset by chatty subordinates.
For long-running async work that doesn't naturally emit progress:
```python
async def call_model(state: State, runtime: Runtime) -> State:
while still_working:
...
runtime.heartbeat()
return {"messages": [response]}
```
`NodeTimeoutError` subclasses `TimeoutError` and carries `node`,
`timeout`, `run_timeout`, `idle_timeout`, `elapsed`, and `kind` (`"run"`
or `"idle"`). If the node's `retry_policy` permits `TimeoutError` it'll
retry; the timer resets per attempt.
## Why async-only
Sync Python code cannot be safely cancelled in-process, so timeouts only
apply to async nodes/tasks. Sync nodes with a `timeout` are rejected at
compile time (covers direct nodes, wrapped runnables, sequences, and
`RunnableParallel` branches); `run_with_retry` rejects them at runtime
as a safety net.
## What gets cancelled and what doesn't
When a watchdog fires:
1. The attempt scope is closed under a lock so any in-flight
`CONFIG_KEY_SEND` / stream / child-task scheduling that races with the
timeout is dropped atomically.
2. Buffered `task.writes` are cleared so pre-timeout writes from the
failed attempt don't leak into the checkpoint after a retry succeeds.
3. The background `asyncio.Task` is cancelled; its eventual exception is
drained via a done-callback so asyncio doesn't log it.
Only the watchdog's own `TimeoutError` converts to `NodeTimeoutError`,
so user-raised `asyncio.TimeoutError`, built-in `TimeoutError`, and
`NodeTimeoutError` from a child node continue to propagate unchanged.
Child tasks already scheduled before the timeout fired still complete —
they aren't part of the cancelled task's structured cancellation
surface. This is intentional and tested.
## External-watchdog hook
WARNING: THIS API IS IN ALPHA AND SUBJECT TO CHANGE.
`CONFIG_KEY_TIMED_ATTEMPT_OBSERVER` is a per-config callback that
receives lifecycle events for each timed attempt:
- `start` — fired before the proc runs, with `task_id`, `task_name`,
`attempt`, `run_id`, `thread_id`, `checkpoint_ns`, `started_at`, and the
configured `run_timeout_secs` / `idle_timeout_secs` / `refresh_on`.
- `progress` — fired on each progress signal that resets the idle clock,
rate-limited to ~4 events per `idle_timeout` window so token-rate
callbacks don't flood the observer. Carries the same context plus
`progress_at`.
- `finish` — fired with `finished_at`, `status` (`"success"`/`"error"`),
`error_type`, `error_message`. `ParentCommand` and `GraphBubbleUp` are
treated as control flow, not errors.
This lets an orchestrating process listen to `start` + rolling
`progress` to compute its own kill deadline (`progress_at +
idle_timeout_secs`) and hard-kill a worker process if the in-process
cancellation deadlocks. Observer callbacks run in whatever thread fires
them, and any exception they raise is logged and swallowed.
## Implementation Notes
`runtime.heartbeat()` updates the progress timestamp without taking the
guarded write lock. This avoids lock overhead on high-frequency
callback/token paths and accepts a small timestamp race window, which is
negligible for expected coarse idle-timeout configurations.
## Testing
Covers timeout retry behavior, user-raised timeout propagation,
stale-write suppression, stream / callback / heartbeat progress resets,
sync-node rejection, `RunnableParallel` branch validation,
`StateGraph.add_node` behavior, lower-level Pregel behavior, observer
start/progress/finish events, and functional API compatibility.
---------
Co-authored-by: Will Fu-Hinthorn <will@langchain.dev>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
This PR implements task/node-level timeouts.
Since python has a terrible multi-processing model, we make two
concessions:
- we only support for async functions/nodes. Sync nodes with a timeout
raise an error at compile time
- we implement this with asyncio wait_for in the async path.
Each timed attempt is wrapped in _retry.py, and the timer is reset on
each node-level retry. When the deadline is exceeded LangGraph raises
NodeTimeoutError, clears buffered writes, and prevents any late writes
or child-task scheduling from leaking past the timeout via
_TimedAttemptScope.
The design also adds a timed-attempt observer hook
(CONFIG_KEY_TIMED_ATTEMPT_OBSERVER) that emits start/finish events with
identifiers and deadlines. This means that if you have an orchestrating
process starting a worker process, it can listen to start/end events and
hard-kill the process to enforce a timeout if there is a deadlock.
---------
Co-authored-by: Will Fu-Hinthorn <will@langchain.dev>
Allows us to put some more information for tracing purposes (e.g.,
ls_integration) without dumping it into the streaming APIs (good for
performance)
Move some other metadata into tracing only since it's not needed in
streaming APIs
---------
Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>
## Summary
- Remove `test_list_global_search` from the conformance test suite. This
test required cross-thread `alist(None, filter=...)` support that not
all checkpointer implementations provide.
- Remove the corresponding entry from `ALL_LIST_TESTS`.
- Bump `langgraph-checkpoint-conformance` version from 0.0.1 to 0.0.2.
## Test plan
- [x] Verify `test_list_global_search` function definition is fully
removed
- [x] Verify `test_list_global_search` is removed from `ALL_LIST_TESTS`
- [x] Verify version bumped to 0.0.2 in pyproject.toml
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Will Fu-Hinthorn <will@langchain.dev>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
## Summary
Adds native uv workspace/lockfile support to the LangGraph CLI's Docker
build pipeline. Instead of listing dependencies manually, users can
point at their existing `uv.lock` and the CLI will:
1. Discover workspace packages and their dependency graph
2. Export locked requirements via `uv export --package <name> --frozen`
3. Copy only the necessary workspace closure into the container
4. Install packages in dependency order with `--no-deps` for
reproducibility
5. Rewrite all import paths (graphs, auth, encryption, etc.) to
container paths
### New config field: `source`
Rather than using `pip` or `uv pip`, we add a new `uv_lock` installer.
The previous installers should still remain unchanged.
To avoid ambiguity, we discriminate by "source" field and **do not
permit** other arbitrary "dependencies". In this mode, we will treat the
provided root (defaults to the current directory) as the source of
truth.
This also would natively support uv workspaces, so you can specify the
target package within a larger workspace.
**Simple single-package project:**
```json
{
"python_version": "3.11",
"graphs": {
"agent": "./agent.py:graph"
},
"source": {
"kind": "uv"
}
}
```
**Multi-package workspace with explicit package:**
```json
{
"python_version": "3.11",
"graphs": {
"agent": "../../apps/agent/src/agent/graph.py:graph"
},
"source": {
"kind": "uv",
"root": "../..",
"package": "agent"
}
}
```
**Traditional pip deployment (unchanged):**
```json
{
"python_version": "3.11",
"dependencies": ["langgraph", "my-package"],
"graphs": {
"agent": "./agent.py:graph"
}
}
```
Config validation enforces mutual exclusivity. you must use either
`dependencies` or `source`, not both.
---------
Co-authored-by: Will Fu-Hinthorn <will@langchain.dev>
## Summary
- Bump `langgraph` version from `1.1.2` to `1.1.5` in the CI integration
test workflow version check
- Bump `langgraph` version from `1.1.2` to `1.1.5` in the prerelease
example `pyproject.toml` files
## Test plan
- [ ] CI integration tests pass with the updated version expectation
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Will Fu-Hinthorn <will@langchain.dev>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
## Summary
- Adds a `langsmith_tracing` parameter to `runs.create()`,
`runs.stream()`, and `runs.wait()` on both async and sync SDK clients
- Accepts a `LangSmithTracing` TypedDict with optional `project_name`
and `example_id` fields
- Maps to the server's existing `langsmith_tracer` payload key, enabling
users to route traces to specific LangSmith projects or associate with
dataset examples from the SDK
## Test plan
- [x] Unit tests verify payload serialization (langsmith_tracing →
langsmith_tracer mapping)
- [x] Unit tests verify key is excluded when param not provided
- [x] Unit tests verify partial configs (project_name only) work
- [x] API parity tests pass (async/sync client signatures match)
- [x] Full test suite passes (86 tests)
- [x] Lint + type check pass
Release Notes: Added `langsmith_tracing` parameter to `runs.create()`,
`runs.stream()`, and `runs.wait()` in the Python SDK, allowing users to
route traces to a specific LangSmith project or associate with a dataset
example.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Will Fu-Hinthorn <will@langchain.dev>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
## Summary
- When `langgraph deploy` prompts for the deployment name interactively,
persist it as `LANGSMITH_DEPLOYMENT_NAME` in the `.env` file so
subsequent deploys pick it up automatically
- Extracted `_resolve_env_path()` helper from `_parse_env_from_config()`
to share env file path resolution logic
- Skips writing when env vars are configured as an inline dict in
`langgraph.json` (no file to write to)
- **CI fix**: CLI integration tests were using `pip install` but
`setup-uv` with caching enabled, so the post-step cache save failed on a
non-existent directory — switched to `uv pip install`
## Test plan
- [x] All 207 existing unit tests pass
- [ ] Manual: run `langgraph deploy` without `--name`, enter a name at
prompt, verify `.env` now contains `LANGSMITH_DEPLOYMENT_NAME=<name>`
- [ ] Manual: run `langgraph deploy` again, verify the name is picked up
from `.env` without re-prompting
- [x] CI integration test cache fix (broken on main since Feb 2026)
Cleaning up CI for #6701
```md
I'm trying to help a customer with some issues related to their checkpointing in postgres. They have some timeouts and retries around the langgraph checkpoint queries and I suspect the queue may be filling up with cancelled tasks.
This PR adds some eager checks to not execute an operation if the future was already cancelled.
This is to prevent the queue executing tasks that might have already timed out, which would otherwise cause more tasks to timeout due to the longer execution delay.
Thank you for contributing to LangGraph! Follow these steps to mark your pull request as ready for review. **If any of these steps are not completed, your PR will not be considered for review.**
```
Co-authored-by: Conrad Ludgate <conradludgate@gmail.com>
## Summary
- Updated all auth docstring examples in `libs/sdk-py` to follow a
**default-closed** pattern
- Every example now first registers a global `@auth.on` handler that
**denies** all requests, then adds resource/action-specific handlers to
selectively **allow** access
- Fixed inconsistencies (e.g., `@auth.on` vs `@my_auth.on`, sync vs
async handlers) and made examples more realistic
## Test plan
- [x] `make format` passes
- [x] `make lint` passes
- [x] Changes are docstring-only — no runtime behavior affected
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
## Summary
- Adds `extract` parameter to `threads.search()` in both async and sync
clients
- Adds `extracted` field to the `Thread` TypedDict response type
- The `extract` parameter accepts a `dict[str, str]` mapping aliases to
JSONB paths (e.g., `{"last_msg": "values.messages[-1]"}`)
- Depends on server-side support in
https://github.com/langchain-ai/langgraph-api/pull/2609
## Test plan
- [ ] Verify types are correct via lint/format (already passing)
- [ ] Integration test against server with extract feature enabled
Release Notes: Add `extract` parameter to `threads.search()` for
extracting nested values from thread data during search.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
## Summary
- The config processing code in `config.py` already handles graphs
defined as `{"path": "...", "description": "..."}` dicts, but the
`Config` TypedDict and JSON schema only declared `dict[str, str]`
- Added a `GraphDef` TypedDict with `path` and optional `description`
fields
- Updated `Config.graphs` to `dict[str, str | GraphDef]` and regenerated
the JSON schemas
- This makes the schema match the actual runtime behavior and fixes
IDE/schema validation for users who use the dict format
## Test plan
- [x] `make format` passes
- [x] `make lint` passes
- [x] `make test` passes (all 85 tests)
- [x] Schema regeneration produces consistent output
Release Notes: Update `langgraph.json` schema to support `{"path":
"...", "description": "..."}` format for graph definitions.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
## Summary
- Adds `stream_mode`, `stream_subgraphs`, `stream_resumable`, and
`durability` parameters to cron `create`, `create_for_thread`, and
`update` methods in both async and sync clients
- Adds corresponding fields to the `CronUpdate` TypedDict in `schema.py`
- Adds `checkpoint_during` deprecation warnings to cron create methods
(consistent with the runs client pattern)
These fields were added to the OpenAPI spec in langgraph-api but were
not yet reflected in the Python SDK.
## Test plan
- [x] `make format` passes
- [x] `make lint` passes
- [x] `make test` passes (69/69, including sync/async API parity test)
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
## Summary
- Add `_StoreActionOn` class with action-specific decorator properties
(`.put`, `.get`, `.search`, `.delete`, `.list_namespaces`) to
`_StoreOn`, enabling `@auth.on.store.put` etc. which was documented but
not implemented
- Update docstring examples to show namespace-rewriting pattern as the
canonical store auth approach
- Fix `AuthContext.action` docstring: add missing `search` and `delete`
store actions
- Fix typo in `StoreSearch.query` docstring
- Add auth-handler-aware docstrings to all store TypedDicts
## Test plan
- [x] `make format && make lint` passes in `libs/sdk-py`
- [] Verify `@auth.on.store.put` decorator works at runtime
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Add some conformance tests for checkpointer implementations. Includes
additona methods that will be useful if you want to integarte in the
agent server.
Additional guidelines:
- Make sure optional dependencies are imported within a function.
- Please do not add dependencies to `pyproject.toml` files (even
optional ones) unless they are **required** for unit tests.
- Most PRs should not touch more than one package.
- Changes should be backwards compatible.
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
## Summary
- Add `"keep_latest"` to `ThreadTTLConfig.strategy` to match
langgraph-api support for pruning old checkpoints while retaining the
thread and its latest state
- Add `sweep_limit` to `ThreadTTLConfig` where the API actually reads it
(was previously a no-op on `CheckpointerConfig`)
- Regenerate `schema.json` / `schema.v0.json`
## Test plan
- [x] `make format && make lint` passes
- [x] `make test` passes (85/85)
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Main jtbd here:
a) clarify who/how a graph is being accessed and make the factory aware
of the `context` where relevant (and make it obvious when it is
available)
b) make it more clear when you can bypass / defer resources with
expensive lifespans (like MCp connections)
c) make auth access more type-safe.
Gives us room to add other information, like:
- langsmith distributed tracing information
---- old
Can start doing things like this:
```
def my_graph(runtime: ServerRuntime):
if runtime.ensure_user().permissions not in ("foo"):
raise ValueError("bar")
```
etc.
Points of expected confusion:
- You won't have a stream_writer in this context.
- This won't be an accessible object within the graph, only the graph
factory.
For maintainers, related draft PR int he server
https://github.com/langchain-ai/langgraph/pull/6774
`aiosqlite` changed it's Connection type to no longer subclass
`threading.Thread`. This removed the is_alive method, which is called
proactively in setup().
This PR handles this in a backwards compat way.
Otherwise, you cannot use `context` with stateful runs, because the
server throws if you provide both configurable and context in a single
call (due to ambiguous parameters)
Extract two common cases from the big switch statement of
`prepare_single_task` since it's a tad more composable.
All this does is shift/extract code to separate functions
In this PR:
- Bump `langgraph-checkpoint` to 3.0
- Bump `langgraph-checkpoint-sqlite` to 3.0; Update
`langgraph-checkpoint` deps to >=3,<4
- Bump `langgraph-checkpoint-postgres` to 3.0; Update
`langgraph-checkpoint` max to <4 (keep prior min since the deprecated
functionality wasn't explicitly used)
- Bump `langgraph` to 1.0.1; update `langgraph-checkpoint` max bound to
4
- Bump `prebuilt` to 1.0.1; update `langgraph-checkpoint` max bound to 4
This PR updates the OpenAPI specification with changes detected from the
LangGraph API server.
**Changes detected as of LangGraph API version 0.4.42**
This update was automatically generated by the sync workflow in the
langgraph-api repository.
Co-authored-by: hinthornw <hinthornw@users.noreply.github.com>
This PR updates the OpenAPI specification with changes detected from the
LangGraph API server.
**Changes detected as of LangGraph API version 0.4.11**
This update was automatically generated by the sync workflow in the
langgraph-api repository.
Co-authored-by: hinthornw <hinthornw@users.noreply.github.com>
