Compare commits

...
Author SHA1 Message Date
Nick Hollon 83b0d1215b overview: restore thread.output (final-state Awaitable, shares values SSE) 2026-05-14 09:45:36 -07:00
Nick Hollon 08a43ad674 diagram: rename .input.respond to .run.respond to match §3 table 2026-05-13 15:20:25 -07:00
Nick Hollon 07095fdf8b roll in Notion edits: move nested handles to §3.1; rename respond; pin server version 2026-05-13 15:16:33 -07:00
Nick Hollon e450e49afc address Notion review: §2.1 nested handles + §4.1 available projections 2026-05-13 15:13:28 -07:00
Nick Hollon 766103173a high-level design overview for v3 streaming (RFC) 2026-05-13 12:47:51 -07:00
Mason DaughertyandGitHub 4a86705bd7 ci: disable uv cache in publish pipeline (#7777)
Disable `uv` dependency caching throughout the release workflow so
build, validation, PyPI publish, and GitHub release jobs do not restore
shared cache state.

Release jobs already pass the built distributions through explicit
artifacts; keeping dependency caches out of this path avoids
cache-poisoning risk and makes the existing pre-release validation
comment match the actual workflow behavior.
2026-05-12 10:10:50 -07:00
Sydney RunkleandGitHub 3614e88c58 release: bump alpha packages to official versions (#7775)
## Summary

Bumps all alpha-versioned packages to their official releases:

- `langgraph`: `1.2.0a7` → `1.2.0`
- `langgraph-checkpoint`: `4.1.0a4` → `4.1.0`
- `langgraph-checkpoint-postgres`: `3.1.0a4` → `3.1.0`
- `langgraph-checkpoint-sqlite`: `3.1.0a1` → `3.1.0`
- `langgraph-prebuilt`: `1.1.0a2` → `1.1.0`

Also removes the `a*` alpha specifiers from cross-dependency pins in
`langgraph`, `checkpoint-postgres`, and `checkpoint-sqlite`, and
regenerates all `uv.lock` files.
2026-05-11 23:27:38 -04:00
Quanzheng LongandGitHub 6dff3b3bce feat(langgraph): durable error-handler resume across host crashes (#7773)
## Summary

- **Consolidate error writes:** When a node fails and has an error
handler, `commit()` now appends both `ERROR` and `ERROR_SOURCE_NODE` in
a single `put_writes` call, eliminating the redundant overwrite that
`schedule_error_handler` used to do.
- **Ensure durability before handler execution:** Reuses the
`_delta_write_futs` pattern — a new `_error_handler_write_futs` list
collects the persistence future from `put_writes` when
`ERROR_SOURCE_NODE` is written, and `schedule_error_handler` /
`aschedule_error_handler` drain it (sync: `concurrent.futures.wait`,
async: `asyncio.gather`) before preparing the handler task.
- **Resume directly to error handler:** Adds
`_resume_error_handlers_if_applicable()` to `PregelLoop`, called from
`tick()` after `_reapply_writes_to_succeeded_nodes()`. On resume, it
detects `ERROR_SOURCE_NODE` markers in `checkpoint_pending_writes`,
marks the original task as done (so the runner skips it), and schedules
a fresh handler task.
- **Rename internal methods for clarity:** `_match_writes` →
`_reapply_writes_to_succeeded_nodes` (makes it clear that
failed/interrupted tasks are skipped); `_resume_error_handlers` →
`_resume_error_handlers_if_applicable`.

## Test plan

- [x] `test_error_handler_resumes_after_crash`: single node fails,
handler crashes, resume re-runs the handler (not the original node).
Verifies `NodeError.node` and error content survive checkpoint
round-trip.
- [x] `test_error_handler_resumes_after_crash_multiple_nodes`: two nodes
fail concurrently in the same superstep, each with its own handler.
Verifies error handler starts while other nodes are still in-flight (via
`threading.Event`), and on resume both handlers re-run with correct
`NodeError.node` and error content.
- [x] All 101 tests in `test_retry.py` pass (including 19 error-handler
tests).
- [x] `make format` + `make lint` clean.
2026-05-12 01:39:03 +00:00
17 changed files with 428 additions and 46 deletions
+3
View File
@@ -32,6 +32,7 @@ jobs:
with:
python-version: ${{ env.PYTHON_VERSION }}
cache-suffix: "release"
enable-cache: false
working-directory: ${{ inputs.working-directory }}
# We want to keep this build stage *separate* from the release stage,
@@ -268,6 +269,7 @@ jobs:
with:
python-version: ${{ env.PYTHON_VERSION }}
cache-suffix: "release"
enable-cache: false
working-directory: ${{ inputs.working-directory }}
- uses: actions/download-artifact@v8
@@ -309,6 +311,7 @@ jobs:
with:
python-version: ${{ env.PYTHON_VERSION }}
cache-suffix: "release"
enable-cache: false
working-directory: ${{ inputs.working-directory }}
- uses: actions/download-artifact@v8
+1 -1
View File
@@ -279,7 +279,7 @@ wheels = [
[[package]]
name = "langgraph-checkpoint"
version = "4.1.0a4"
version = "4.1.0"
source = { editable = "../checkpoint" }
dependencies = [
{ name = "langchain-core" },
+2 -2
View File
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project]
name = "langgraph-checkpoint-postgres"
version = "3.1.0a4"
version = "3.1.0"
description = "Library with a Postgres implementation of LangGraph checkpoint saver."
authors = []
requires-python = ">=3.10"
@@ -12,7 +12,7 @@ readme = "README.md"
license = "MIT"
license-files = ['LICENSE']
dependencies = [
"langgraph-checkpoint>=4.1.0a4,<5.0.0",
"langgraph-checkpoint>=4.1.0,<5.0.0",
"orjson>=3.11.5",
"psycopg>=3.2.0",
"psycopg-pool>=3.2.0",
+2 -2
View File
@@ -276,7 +276,7 @@ wheels = [
[[package]]
name = "langgraph-checkpoint"
version = "4.1.0a4"
version = "4.1.0"
source = { editable = "../checkpoint" }
dependencies = [
{ name = "langchain-core" },
@@ -324,7 +324,7 @@ test = [
[[package]]
name = "langgraph-checkpoint-postgres"
version = "3.1.0a4"
version = "3.1.0"
source = { editable = "." }
dependencies = [
{ name = "langgraph-checkpoint" },
+2 -2
View File
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project]
name = "langgraph-checkpoint-sqlite"
version = "3.1.0a1"
version = "3.1.0"
description = "Library with a SQLite implementation of LangGraph checkpoint saver."
authors = []
requires-python = ">=3.10"
@@ -12,7 +12,7 @@ readme = "README.md"
license = "MIT"
license-files = ['LICENSE']
dependencies = [
"langgraph-checkpoint>=4.1.0a4,<5.0.0",
"langgraph-checkpoint>=4.1.0,<5.0.0",
"aiosqlite>=0.20",
"sqlite-vec>=0.1.6",
]
+2 -2
View File
@@ -285,7 +285,7 @@ wheels = [
[[package]]
name = "langgraph-checkpoint"
version = "4.1.0a4"
version = "4.1.0"
source = { editable = "../checkpoint" }
dependencies = [
{ name = "langchain-core" },
@@ -333,7 +333,7 @@ test = [
[[package]]
name = "langgraph-checkpoint-sqlite"
version = "3.1.0a1"
version = "3.1.0"
source = { editable = "." }
dependencies = [
{ name = "aiosqlite" },
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project]
name = "langgraph-checkpoint"
version = "4.1.0a4"
version = "4.1.0"
description = "Library with base interfaces for LangGraph checkpoint savers."
authors = []
requires-python = ">=3.10"
+1 -1
View File
@@ -300,7 +300,7 @@ wheels = [
[[package]]
name = "langgraph-checkpoint"
version = "4.1.0a4"
version = "4.1.0"
source = { editable = "." }
dependencies = [
{ name = "langchain-core" },
+104 -17
View File
@@ -203,6 +203,12 @@ class PregelLoop:
# `__enter__`; stays `None` only when no checkpointer.
_delta_write_futs: list[Any] | None = None
# Same pattern as `_delta_write_futs` but for error-handler writes.
# When `put_writes` persists an ERROR_SOURCE_NODE marker, the future is
# appended here. `schedule_error_handler` / `aschedule_error_handler`
# drain this list so the write is durable before the handler starts.
_error_handler_write_futs: list[Any] | None = None
# Exit-mode accumulator: every delta-channel write produced during this
# run (input writes from `_first` + per-superstep writes captured in
# `after_tick`). At exit, `_put_exit_delta_writes` filters out channels
@@ -474,6 +480,13 @@ class PregelLoop:
isinstance(self.specs.get(c), DeltaChannel) for c, _ in writes_to_save
):
self._delta_write_futs.append(fut)
# ERROR_SOURCE_NODE is only appended by commit() when the task
# has an error handler (_should_route_to_error_handler), so this
# check naturally limits future collection to those tasks.
if self._error_handler_write_futs is not None and any(
c == ERROR_SOURCE_NODE for c, _ in writes
):
self._error_handler_write_futs.append(fut)
# output writes
if hasattr(self, "tasks"):
self.output_writes(task_id, writes)
@@ -553,7 +566,7 @@ class PregelLoop:
self.tasks[pushed.id] = pushed
# match any pending writes to the new task
if not self.is_replaying:
self._match_writes({pushed.id: pushed})
self._reapply_writes_to_succeeded_nodes({pushed.id: pushed})
# return the new task, to be started if not run before
return pushed
@@ -631,7 +644,8 @@ class PregelLoop:
# if there are pending writes from a previous loop, apply them
if not self.is_replaying and self.checkpoint_pending_writes:
self._match_writes(self.tasks)
self._reapply_writes_to_succeeded_nodes(self.tasks)
self._resume_error_handlers_if_applicable()
# before execution, check if we should interrupt
if self.interrupt_before and should_interrupt(
@@ -698,13 +712,88 @@ class PregelLoop:
# private
def _match_writes(self, tasks: Mapping[str, PregelExecutableTask]) -> None:
def _reapply_writes_to_succeeded_nodes(
self, tasks: Mapping[str, PregelExecutableTask]
) -> None:
"""Restore successful channel writes from checkpoint to in-memory tasks.
Skips control signals (ERROR, ERROR_SOURCE_NODE, INTERRUPT, RESUME)
so that failed/interrupted tasks remain with empty writes and will be
re-executed (or routed to error handlers) by the runner.
"""
for tid, k, v in self.checkpoint_pending_writes:
if k in (ERROR, ERROR_SOURCE_NODE, INTERRUPT, RESUME):
continue
if task := tasks.get(tid):
task.writes.append((k, v))
def _resume_error_handlers_if_applicable(self) -> None:
"""On resume, schedule error handlers for tasks that failed in a prior run.
Called right after ``_reapply_writes_to_succeeded_nodes`` during ``tick()``.
At that point, ``_reapply_writes_to_succeeded_nodes`` has already skipped
ERROR / ERROR_SOURCE_NODE writes, so a previously-failed task still has
empty ``writes``. Without intervention the runner (which executes only
tasks where ``not t.writes``) would re-run the original node.
This method prevents that re-execution for nodes that have an error
handler:
1. Scan ``checkpoint_pending_writes`` for ERROR_SOURCE_NODE markers
persisted by a prior ``commit()``. Each marker means "this task
already failed and was routed to an error handler".
2. For each such task, write ``(ERROR, error)`` into ``task.writes``
so the task is no longer empty — the runner will skip it.
3. Prepare a fresh error-handler task and add it to ``self.tasks``.
Because the handler task starts with empty ``writes``, the runner
will pick it up and execute it.
"""
# Phase 1: collect task-ids that have ERROR_SOURCE_NODE + ERROR pairs.
failed: dict[str, BaseException] = {}
for tid, chan, val in self.checkpoint_pending_writes:
if chan == ERROR_SOURCE_NODE:
error = next(
(
v
for t, c, v in self.checkpoint_pending_writes
if t == tid and c == ERROR
),
None,
)
if error is not None:
failed[tid] = error
# Phase 2: mark originals as done, schedule handler tasks.
for task_id, error in failed.items():
task = self.tasks.get(task_id)
if task is None:
continue
handler_node = self.nodes[task.name].error_handler_node
if not handler_node:
continue
# Non-empty writes → runner's `not t.writes` filter skips this task.
task.writes.append((ERROR, error))
# The handler task starts with empty writes → runner will execute it.
handler_task = prepare_node_error_handler_task(
task,
handler_node_name=handler_node,
failed_error=error,
checkpoint=self.checkpoint,
pending_writes=self.checkpoint_pending_writes,
processes=self.nodes,
channels=self.channels,
managed=self.managed,
config=task.config,
step=self.step,
stop=self.stop,
store=self.store,
checkpointer=self.checkpointer,
manager=self.manager,
retry_policy=self.retry_policy,
cache_policy=self.cache_policy,
)
if handler_task is not None:
self.tasks[handler_task.id] = handler_task
def _pending_interrupts(self) -> set[str]:
"""Return the set of interrupt ids that are pending without corresponding resume values."""
# mapping of task ids to interrupt ids
@@ -1454,12 +1543,10 @@ class SyncPregelLoop(PregelLoop, AbstractContextManager):
handler_node = self.nodes[failed_task.name].error_handler_node
if not handler_node:
return None
writes = list(failed_task.writes)
writes.append((ERROR_SOURCE_NODE, failed_task.name))
self.put_writes(
failed_task.id,
writes,
)
# ensure error + ERROR_SOURCE_NODE writes are durable before handler runs
if self._error_handler_write_futs:
futs, self._error_handler_write_futs = self._error_handler_write_futs, []
concurrent.futures.wait(futs)
handler_task = prepare_node_error_handler_task(
failed_task,
handler_node_name=handler_node,
@@ -1482,7 +1569,7 @@ class SyncPregelLoop(PregelLoop, AbstractContextManager):
return None
self.tasks[handler_task.id] = handler_task
if not self.is_replaying:
self._match_writes({handler_task.id: handler_task})
self._reapply_writes_to_succeeded_nodes({handler_task.id: handler_task})
for task in self.match_cached_writes():
self.output_writes(task.id, task.writes, cached=True)
return handler_task
@@ -1564,6 +1651,7 @@ class SyncPregelLoop(PregelLoop, AbstractContextManager):
else []
)
self._delta_write_futs = []
self._error_handler_write_futs = []
self._exit_delta_writes = (
[] if self.durability == "exit" and self.checkpointer is not None else None
)
@@ -1709,12 +1797,10 @@ class AsyncPregelLoop(PregelLoop, AbstractAsyncContextManager):
handler_node = self.nodes[failed_task.name].error_handler_node
if not handler_node:
return None
writes = list(failed_task.writes)
writes.append((ERROR_SOURCE_NODE, failed_task.name))
self.put_writes(
failed_task.id,
writes,
)
# ensure error + ERROR_SOURCE_NODE writes are durable before handler runs
if self._error_handler_write_futs:
futs, self._error_handler_write_futs = self._error_handler_write_futs, []
await asyncio.gather(*futs)
handler_task = prepare_node_error_handler_task(
failed_task,
handler_node_name=handler_node,
@@ -1737,7 +1823,7 @@ class AsyncPregelLoop(PregelLoop, AbstractAsyncContextManager):
return None
self.tasks[handler_task.id] = handler_task
if not self.is_replaying:
self._match_writes({handler_task.id: handler_task})
self._reapply_writes_to_succeeded_nodes({handler_task.id: handler_task})
for task in await self.amatch_cached_writes():
self.output_writes(task.id, task.writes, cached=True)
return handler_task
@@ -1822,6 +1908,7 @@ class AsyncPregelLoop(PregelLoop, AbstractAsyncContextManager):
else []
)
self._delta_write_futs = []
self._error_handler_write_futs = []
self._exit_delta_writes = (
[] if self.durability == "exit" and self.checkpointer is not None else None
)
+2 -1
View File
@@ -31,6 +31,7 @@ from langgraph._internal._constants import (
CONFIG_KEY_CALL,
CONFIG_KEY_SCRATCHPAD,
ERROR,
ERROR_SOURCE_NODE,
INTERRUPT,
NO_WRITES,
RESUME,
@@ -597,7 +598,7 @@ class PregelRunner:
if self._should_route_to_error_handler(task) and not isinstance(
exception, GraphBubbleUp
):
# Mark early in commit path; loop-side routing may happen later.
task.writes.append((ERROR_SOURCE_NODE, task.name))
self._handled_exception_ids.add(id(exception))
self.put_writes()(task.id, task.writes) # type: ignore[misc]
else:
+3 -3
View File
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project]
name = "langgraph"
version = "1.2.0a7"
version = "1.2.0"
description = "Building stateful, multi-actor applications with LLMs"
authors = []
requires-python = ">=3.10"
@@ -25,9 +25,9 @@ classifiers = [
]
dependencies = [
"langchain-core>=1.4.0,<2",
"langgraph-checkpoint>=4.1.0a4,<5.0.0",
"langgraph-checkpoint>=4.1.0,<5.0.0",
"langgraph-sdk>=0.3.0,<0.4.0",
"langgraph-prebuilt>=1.1.0a2,<1.2.0",
"langgraph-prebuilt>=1.1.0,<1.2.0",
"xxhash>=3.5.0",
"pydantic>=2.7.4",
]
+135
View File
@@ -2667,3 +2667,138 @@ def test_set_node_defaults_combined_retry_and_error_handler():
assert result["foo"] == "handled"
assert attempts == 2
assert captured["error"] == "Always fails"
def test_error_handler_resumes_after_crash():
"""If the error handler crashes, resuming should re-schedule the handler
(not re-execute the original failed node)."""
class State(TypedDict):
foo: str
call_count = {"node": 0, "handler": 0}
captured_errors: list[NodeError] = []
def failing_node(state: State) -> State:
call_count["node"] += 1
raise RuntimeError("boom")
handler_should_fail = [True]
def handler(state: State, error: NodeError) -> State:
call_count["handler"] += 1
captured_errors.append(error)
if handler_should_fail[0]:
raise RuntimeError("handler crash")
return {"foo": "recovered"}
checkpointer = MemorySaver()
graph = (
StateGraph(State)
.set_node_defaults(error_handler=handler)
.add_node("fail", failing_node)
.add_edge(START, "fail")
.compile(checkpointer=checkpointer)
)
config = {"configurable": {"thread_id": "t1"}}
# First invoke: node fails -> handler runs -> handler crashes -> run fails
with pytest.raises(RuntimeError, match="handler crash"):
graph.invoke({"foo": ""}, config)
assert call_count["node"] == 1
assert call_count["handler"] == 1
assert captured_errors[0].node == "fail"
assert isinstance(captured_errors[0].error, RuntimeError)
assert str(captured_errors[0].error) == "boom"
# Resume: handler should run again, NOT the original node
handler_should_fail[0] = False
result = graph.invoke(None, config)
assert result["foo"] == "recovered"
assert call_count["node"] == 1 # NOT re-executed
assert call_count["handler"] == 2 # ran again on resume
# on resume the error was round-tripped through the checkpointer, so it
# may be deserialized as a string representation rather than the original
# exception type — verify the node name and that the error content matches.
assert captured_errors[1].node == "fail"
assert "boom" in str(captured_errors[1].error)
def test_error_handler_resumes_after_crash_multiple_nodes():
"""When multiple nodes fail in the same superstep and all have error handlers:
- error handlers start running while other nodes may still be in-flight
- resuming re-schedules each handler (not re-executes the original nodes)
"""
class State(TypedDict):
results: Annotated[list[str], operator.add]
call_count = {"a": 0, "b": 0, "handler_a": 0, "handler_b": 0}
handler_a_started = threading.Event()
def node_a(state: State) -> State:
call_count["a"] += 1
raise RuntimeError("a failed")
def node_b(state: State) -> State:
call_count["b"] += 1
# Block until handler_a has started — proves the error handler runs
# concurrently with in-flight nodes in the same superstep.
assert handler_a_started.wait(timeout=5), "handler_a never started"
raise RuntimeError("b failed")
handler_should_fail = [True]
def handler_a(state: State, error: NodeError) -> State:
call_count["handler_a"] += 1
assert error.node == "a"
assert "a failed" in str(error.error)
handler_a_started.set()
if handler_should_fail[0]:
raise RuntimeError("handler_a crash")
return {"results": [f"recovered_a:{error.node}"]}
def handler_b(state: State, error: NodeError) -> State:
call_count["handler_b"] += 1
assert error.node == "b"
assert "b failed" in str(error.error)
if handler_should_fail[0]:
raise RuntimeError("handler_b crash")
return {"results": [f"recovered_b:{error.node}"]}
checkpointer = MemorySaver()
graph = (
StateGraph(State)
.add_node("a", node_a, error_handler=handler_a)
.add_node("b", node_b, error_handler=handler_b)
.add_edge(START, "a")
.add_edge(START, "b")
.compile(checkpointer=checkpointer)
)
config = {"configurable": {"thread_id": "t1"}}
# First invoke: node_a fails immediately -> handler_a starts (sets event) ->
# node_b unblocks and fails -> handler_b starts -> both handlers crash
with pytest.raises(RuntimeError):
graph.invoke({"results": []}, config)
assert call_count["a"] == 1
assert call_count["b"] == 1
assert call_count["handler_a"] == 1
assert call_count["handler_b"] == 1
# Resume: both handlers should run again, NOT the original nodes
handler_should_fail[0] = False
handler_a_started.clear()
result = graph.invoke(None, config)
assert call_count["a"] == 1 # NOT re-executed
assert call_count["b"] == 1 # NOT re-executed
assert call_count["handler_a"] == 2 # ran again on resume
assert call_count["handler_b"] == 2 # ran again on resume
assert "recovered_a:a" in result["results"]
assert "recovered_b:b" in result["results"]
+5 -5
View File
@@ -1382,7 +1382,7 @@ wheels = [
[[package]]
name = "langgraph"
version = "1.2.0a7"
version = "1.2.0"
source = { editable = "." }
dependencies = [
{ name = "langchain-core" },
@@ -1563,7 +1563,7 @@ wheels = [
[[package]]
name = "langgraph-checkpoint"
version = "4.1.0a4"
version = "4.1.0"
source = { editable = "../checkpoint" }
dependencies = [
{ name = "langchain-core" },
@@ -1611,7 +1611,7 @@ test = [
[[package]]
name = "langgraph-checkpoint-postgres"
version = "3.1.0a4"
version = "3.1.0"
source = { editable = "../checkpoint-postgres" }
dependencies = [
{ name = "langgraph-checkpoint" },
@@ -1658,7 +1658,7 @@ test = [
[[package]]
name = "langgraph-checkpoint-sqlite"
version = "3.1.0a1"
version = "3.1.0"
source = { editable = "../checkpoint-sqlite" }
dependencies = [
{ name = "aiosqlite" },
@@ -1757,7 +1757,7 @@ test = [
[[package]]
name = "langgraph-prebuilt"
version = "1.1.0a2"
version = "1.1.0"
source = { editable = "../prebuilt" }
dependencies = [
{ name = "langchain-core" },
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project]
name = "langgraph-prebuilt"
version = "1.1.0a2"
version = "1.1.0"
description = "Library with high-level APIs for creating and executing LangGraph agents and tools."
authors = []
requires-python = ">=3.10"
+5 -5
View File
@@ -285,7 +285,7 @@ wheels = [
[[package]]
name = "langgraph"
version = "1.2.0a7"
version = "1.2.0"
source = { editable = "../langgraph" }
dependencies = [
{ name = "langchain-core" },
@@ -369,7 +369,7 @@ test = [
[[package]]
name = "langgraph-checkpoint"
version = "4.1.0a4"
version = "4.1.0"
source = { editable = "../checkpoint" }
dependencies = [
{ name = "langchain-core" },
@@ -417,7 +417,7 @@ test = [
[[package]]
name = "langgraph-checkpoint-postgres"
version = "3.1.0a4"
version = "3.1.0"
source = { editable = "../checkpoint-postgres" }
dependencies = [
{ name = "langgraph-checkpoint" },
@@ -464,7 +464,7 @@ test = [
[[package]]
name = "langgraph-checkpoint-sqlite"
version = "3.1.0a1"
version = "3.1.0"
source = { editable = "../checkpoint-sqlite" }
dependencies = [
{ name = "aiosqlite" },
@@ -507,7 +507,7 @@ test = [
[[package]]
name = "langgraph-prebuilt"
version = "1.1.0a2"
version = "1.1.0"
source = { editable = "." }
dependencies = [
{ name = "langchain-core" },
@@ -0,0 +1,156 @@
# Python SDK v3 streaming — high-level design
---
## 1. What it is
A thread-centric streaming surface for `langgraph-sdk` (Python). One context manager per thread, typed command dispatch, typed projections. Mirrors `@langchain/langgraph-sdk` (JS).
```python
async with client.threads.stream(assistant_id="agent") as thread:
await thread.run.start(input={"x": 1})
async for message in thread.messages:
...
final_state = await thread.values
```
Additive — `client.runs.stream(...)` and `client.threads.join_stream(...)` stay.
---
## 2. Architecture
```
User code
│
│ async with client.threads.stream(...) as thread:
│ await thread.run.start(...)
│ async for msg in thread.messages: ...
▼
┌─────────────────────────────────────────────────────────────────┐
│ Public surface — langgraph_sdk._async.stream │
│ │
│ AsyncThreadStream │
│ .run.start .run.respond .agent.get_tree │
│ .events │
│ .values .output .messages .tool_calls │
│ .subgraphs .subagents .extensions["name"] │
│ .interrupted .interrupts │
└─────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────┐
│ Internals — langgraph_sdk.stream │
│ │
│ StreamController (event router, per-run dispatch) │
│ ├─ projections (values / messages / tool_calls / …) │
│ └─ MultiCursorBuffer (per-projection replay log) │
│ │
│ Subscription matcher (channel + namespace filters) │
│ Union-filter SSE (one shared connection per thread) │
│ Lifecycle watcher SSE (always-on, surfaces interrupts) │
│ │
│ ProtocolSseTransport ──── WebSocketTransport │
└─────────────────────────────────────────────────────────────────┘
│
v3 protocol over HTTP/SSE or WebSocket
▼
┌─────────────────────────────────────────────────────────────────┐
│ langgraph-api server │
│ POST /threads/{thread_id}/commands │
│ POST /threads/{thread_id}/stream/events │
│ WS /threads/{thread_id}/stream/events │
└─────────────────────────────────────────────────────────────────┘
```
Each `AsyncThreadStream` holds one union-filter SSE (subscription set rotates as projections come and go) plus one always-on lifecycle SSE — two HTTP connections per active thread. `AsyncThreadStream` is transport-agnostic; SSE and WebSocket implement the same internal `TransportAdapter` contract.
---
## 3. User-facing surface
| API | Notes |
|---|---|
| `client.threads.stream(thread_id=None, *, assistant_id, headers=None)` | Entry point. Returns an `AsyncThreadStream` async context manager. Mints `uuid.uuid4()` when `thread_id` is None. |
| `thread.run.start(input=, config=, metadata=)` | Dispatches `run.start`. Returns `{"run_id": "..."}`. |
| `thread.run.respond(...)` | Resume after interrupt. |
| `thread.agent.get_tree(...)` | Agent introspection. |
| `thread.events` | Raw `AsyncIterator[Event]` over every channel. |
| `thread.values` | `AsyncIterator[snapshot]` plus `Awaitable[final_state]`. |
| `thread.output` | `Awaitable[final_state]`. Convenience — shares the `values` SSE; skip when you don't need intermediate snapshots. |
| `thread.messages` | `AsyncIterator[StreamingMessageHandle]` — typed over `langchain-core` `BaseMessage`. |
| `thread.tool_calls` | `AsyncIterator[ToolCallHandle]`. |
| `thread.subgraphs` / `thread.subagents` | Nested handles for graph composition. |
| `thread.extensions["name"]` | Per-extension dispatch on `custom:<name>`. |
| `thread.interrupted` / `thread.interrupts` | Lifecycle state, always current. |
### 3.1 Nested handles — subgraphs and subagents
`thread.subgraphs` and `thread.subagents` are streams of **invocations**, not static lists of registered components. Each iteration yields one handle scoped to one execution:
```python
async with client.threads.stream(assistant_id="agent") as thread:
await thread.run.start(input={...})
async for subgraph in thread.subgraphs:
async for message in subgraph.messages:
...
async for call in subgraph.tool_calls:
...
```
A `SubgraphHandle` (or `SubagentHandle`) exposes the same projection surface as the top-level thread — `messages`, `tool_calls`, `subgraphs`, `subagents`, media — filtered to events whose namespace matches that invocation's path. Nesting composes: a subgraph that itself invokes another subgraph yields a fresh handle from `subgraph.subgraphs`. There is no static registration step on the SDK side; the set of subgraphs/subagents that appear is discovered at runtime from event namespaces emitted by the graph.
---
## 4. Migration
Pre-v3 (untyped, one run per call):
```python
async for chunk in client.runs.stream(
thread_id, assistant_id, input={...}, stream_mode="messages",
):
# chunk.event and chunk.data are untyped
...
```
v3 (typed, thread-centric, multiple runs per thread):
```python
async with client.threads.stream(thread_id, assistant_id="agent") as thread:
await thread.run.start(input={...})
async for message in thread.messages:
...
```
Differences:
- The thread is the context manager — multiple `run.start` calls compose inside one session.
- Projections replace `stream_mode` — pick the typed iterable you need.
- Reattach is automatic — `client.threads.stream(thread_id="existing-id", ...)` replays buffered events and goes live. Replaces `client.threads.join_stream(...)`.
- The protocol is parsed once at the SDK boundary; projections expose Python objects, not raw frames.
### 4.1 Available projections
The complete set, all defined in `langgraph_sdk._async.stream`:
- `thread.events` — raw `Event` dicts over every channel; untyped, useful for debug or to drop below the typed surface.
- `thread.values` — state snapshots plus final state. Replaces `stream_mode="values"`.
- `thread.output` — `Awaitable[final_state]` only. Shares the `values` SSE; convenience for callers who don't need intermediate snapshots.
- `thread.messages` — `StreamingMessageHandle` typed over `langchain-core` `BaseMessage`. Replaces `stream_mode="messages"`.
- `thread.tool_calls` — `ToolCallHandle` per tool invocation.
- `thread.subgraphs` / `thread.subagents` — nested handles per invocation (see §3.1).
- `thread.extensions["name"]` — per-extension events on `custom:<name>` channels. Replaces `stream_mode="custom"`.
The set is closed at the SDK boundary; `extensions["name"]` is the open-ended escape hatch for server-side transformers that emit on custom channels.
---
## 5. Dependencies
- `langchain-protocol>=0.0.15` — CDDL-derived `TypedDict`s for the protocol wire shape.
- `langchain-core>=1.4.0,<2` — required by the messages projection to construct `BaseMessage` subclasses. Matches the JS SDK's hard-dep stance.
- `httpx`, `orjson` — already required.
- Python 3.10+.
- Server: `langgraph-api` with `FF_V2_EVENT_STREAMING` enabled and ≥ 0.9.0rc1.
+3 -3
View File
@@ -298,7 +298,7 @@ wheels = [
[[package]]
name = "langgraph"
version = "1.2.0a7"
version = "1.2.0"
source = { editable = "../langgraph" }
dependencies = [
{ name = "langchain-core" },
@@ -382,7 +382,7 @@ test = [
[[package]]
name = "langgraph-checkpoint"
version = "4.1.0a4"
version = "4.1.0"
source = { editable = "../checkpoint" }
dependencies = [
{ name = "langchain-core" },
@@ -430,7 +430,7 @@ test = [
[[package]]
name = "langgraph-prebuilt"
version = "1.1.0a2"
version = "1.1.0"
source = { editable = "../prebuilt" }
dependencies = [
{ name = "langchain-core" },