This PR adds error handling and retry mechanisms for create_agent
structured output via a `handle_errors` parameter in `ToolOutput`.
Changes:
* Adds `MultipleStructuredOutputsError` exception for when models
incorrectly call multiple structured output tools simultaneously
* Adds `StructuredOutputParsingError` exception for when tool arguments
fail to parse according to the schema
* Implements automatic error handling logic that re-prompts the model
with a configurable error message when structured output failures occur
via `handle_errors` policy in `ToolOutput`:
```python
class ToolOutput:
...
handle_errors: Union[
bool, # True: retry all, False: no retry
str, # Custom static error message for all errors
type[Exception], # Retry only this exception type
tuple[type[Exception], ...], # Retry only these exception types
Callable[[Exception], str], # Custom callable returning error message
]
"""Error handling strategy. Default: True (retry on all error types with default error message)"""
```
Examples:
```python
# Retry all errors
ToolOutput(WeatherReport)
# No retry
ToolOutput(WeatherReport, handle_errors=False)
# Custom message for all errors
ToolOutput(WeatherReport, handle_errors="Please provide valid data")
# Only retry specific error type
ToolOutput(WeatherReport, handle_errors=StructuredOutputParsingError)
# Multiple error types
ToolOutput(WeatherReport, handle_errors=(MultipleStructuredOutputsError, StructuredOutputParsingError))
# Custom logic
ToolOutput(
Union[WeatherReport, LocationInfo],
handle_errors=lambda e: "Only one response please" if isinstance(e, MultipleStructuredOutputsError) else "Invalid format"
)
```
---------
Co-authored-by: Sydney Runkle <sydneymarierunkle@gmail.com>
Change how we handle tool errors by default in tool node.
1. If a tool invocation failed due to arg validation, we return an
artificial `ToolMessage` back to the model w/ a request for arg
correction (the same, but we've improved the message)
2. If a tool execution failed for an unknown reason, we raise
(different, we used to automatically return to model)
This PR improves standard integration tests in prebuilt to ensure
logical equivalence between the Python and JavaScript implementations of
`create_agent`.
* Cleans up `test_responses_int` test harness
* Adds new test utils to dynamically load JSON test specs
* Adds `test_return_direct_int` test harness to validate model behavior
when `return_direct` tool property is set
* Adds support for when the user instantiates `ToolOutput` with multiple
JSON schemas unified by the `oneOf` keyword
Failing tests:
* `test_inference_to_native_output`: there is some odd behavior where
the model makes a second call to `get_weather` despite having just
received the tool message, so there are 6 messages instead of the 4
expected.
* `test_responses_integration_matrix[asking for information that does
not fit into the response format]`: `XFAIL`, currently failing due to
undefined behavior when the model cannot conform to any of the
structured response formats.
TODO in future PRs:
* Add exception handling to pass `test_responses_integration_matrix`.
* Adds support for `NativeOutput` via a new `NativeOutput` dataclass
* Adds support for structured output specification via the following
(pydantic models already supported)
* dataclasses
* typed dicts
* json schemas
* Adds mocking support to support native strategies with
`FakeToolCallingModel`
* Add new default tool message when `tool_message_content` not provided
* Smart "selection" of native vs tool output based on provider support,
necessitates profiles down the line
Considered questions
* do we want to enforce docstrings? -- decided on no for now
* do we want to enforce names (titles) on json schemas? -- decided no
for now, defaulting to `structured_output`
* do we want to validate that json schemas coming in are valid? --
decided no for now
* do we want to validate model results against a given json schema? we
validate against all other types (typed dict, dataclass, etc) w/
pydantic -- decided no for now
TODO in future PRs:
* Figure out retry policy
* Add standard testing (handed off to @casparb)
* Further privatize certain structures (like the bindings) -- this is
low prio
---------
Co-authored-by: Sydney Runkle <sydneymarierunkle@gmail.com>
Co-authored-by: Sydney Runkle <54324534+sydney-runkle@users.noreply.github.com>
precursor to `prepare_call` PR, cleaning up existing logic w/ pre model
hook
* use one combined `AgentState` instead of the one w/ and w/o structured
response
* remove exposed pydantic agent state + loosen bounds on state type
* remove llm input messages pattern, should be made possible with
prepare_call
also
* some remaining test fixes in `langgraph` to adapt to new `model` node
name (used to be `agent`)
* Add support for ToolOutput response format.
* I don't love the name -- it's confusing unless you know that it's parameterizing a strategy.
We should determine if we want to support our old strategy for doing
things -- it has a higher latency (one extra LLM call), but it's a
reasonable built-in strategy as it doesn't do anything awkward with
conversation history. (Wouldn't surprising if it has overall better
performance than tool choice for longer conversations)
Do not support prebound tools on the model. There's reason users should be prebinding tools to the model!
This is a breaking change that might affect some users, but the work-around is simple -- provide tools into the create_react_agent api.
* Add structured output tools to ToolNode
* Fix default tool node name to match the actual default ('tools')
* Update doc-strings to explain what inputs/outputs are for the ToolNode.
* Mark internal attributes as private (potentially breaking -- although hopefully users aren't accessing these)
## Decisions points
* OK with two properties? Done since users may be relying on
`tools_by_name` and expanding the return type will break user code.
## Changes in public/private interface
### Marked as public
* Make `tools_by_name` an official public property
* Make `structured_output_tools` a public property
### Marked as private
There should be no reason why users are accessing these attributes
```python
_tool_to_state_args
_tool_to_store_arg
_handle_tool_errors
_messages_key
```
### Usage
```python
class OutputSchema(BaseModel):
name: str
age: int
location: str
tool_node = ToolNode([OutputSchema])
# Test that the structured output tool is registered correctly
assert "OutputSchema" in tool_node.structured_output_tools
# Create a tool call that matches the schema
tool_call = {
"name": "OutputSchema",
"args": {"name": "Alice", "age": 30, "location": "NYC"},
"id": "call_123",
"type": "tool_call",
}
# Test sync execution
result = tool_node.invoke(
{"messages": [AIMessage(content="", tool_calls=[tool_call])]}
)
# Should return a Command with structured response
assert isinstance(result, list)
assert len(result) == 1
command = result[0]
assert isinstance(command, Command)
# Check the update structure
assert "messages" in command.update
assert "structured_response" in command.update
# Check the tool message
tool_message = command.update["messages"][0]
assert isinstance(tool_message, ToolMessage)
assert tool_message.name == "OutputSchema"
assert tool_message.tool_call_id == "call_123"
# Check the structured response
structured_response = command.update["structured_response"]
assert isinstance(structured_response, OutputSchema)
assert structured_response.name == "Alice"
assert structured_response.age == 30
assert structured_response.location == "NYC"
```
Add option to split tool node to individual nodes.
Summary:
* User code (specifically streaming) may break if it's relying on the
name of the `tools` node
* The boolean flag in the interface is likely **temporary** (especially
if there are no major breaking changes)
* We'll need to decide if we can get rid of the version in create react
agent. "v1" is not consistent conceptually with a node per tool.
### Description
Adds Redis as a supported cache backend for LangGraph node-level
caching, enabling distributed caching across multiple processes/servers.
This implementation follows the same patterns as existing InMemoryCache
and SqliteCache.
### Key changes
- New RedisCache class implementing the BaseCache interface
- Support for TTL-based expiration and batch operations
- Worker-specific cache prefixes for parallel test isolation
### Dependencies
- redis package (already included in dev dependencies)
### Test Plan
- Unit tests: Added Redis cache tests covering basic operations, TTL,
batch operations, and error handling
- Integration tests: Redis cache integrated into existing LangGraph test
suite, tested with all checkpointer combinations
Reproduces:
https://github.com/langchain-ai/langgraph/issues/5249#issuecomment-3156519635
Caused after this change:
https://github.com/langchain-ai/langgraph/pull/4843
Fix to allow emitting messages from subgraphs if the subgraphs
explicitly used a stream mode "messages".
```python
def node_in_parent(...):
# subgraph was called as a function.
# messages are explicitly requested.
for event in subgraph.stream(..., stream_mode="messages"):
# something is done with `event`
return ...
# subgraphs = False!
parent_graph.invoke(..., subgraphs=False)
```
The code above should continue to work correctly regardless of the value
of subgraphs as streaming messages was requested explicitly in the
parent node!
---------
Co-authored-by: Sydney Runkle <54324534+sydney-runkle@users.noreply.github.com>
- This makes prepare_next_tasks constant on number of nodes in all
cases, whereas before we were falling back to node iteration when
resuming from an existing checkpoint
`MessageGraph` is deprecated, to be removed in v2.
A `StateGraph` with a `messages` key should be used instead.
Alternatively, folks can use `Annotated[list[AnyMessage], add_messages]` as their state schema.
Reverts https://github.com/langchain-ai/langgraph/pull/5562
I anticipate that we want to do another pass at a refactor here in the
short term, but this makes it easier to adapt to new langchain core
message types for v0.4 support in the short term.
Fixes https://github.com/langchain-ai/langgraph/issues/5784
* Removes usage of `is_last_step`, no longer needed with
`remaining_steps`
* Make `remaining_steps` `NotRequired` so that json schema doesn't
suggest need for user input
* Move `PregelScratchpad` to shared utils file to prevent circular
import issue (it's used from `channels/managed` and other pregel files).
* Ensures that managed values wrapped in `NotRequired` or `Required` are
still recognized!
Fixes: #5787
Ensures that if `config` is not typed as one of `RunanbleConfig` or
`Optional[RunnableConfig]` a warning is raised to help developers avoid
unexpected results at invocation time.
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
Co-authored-by: Sydney Runkle <sydneymarierunkle@gmail.com>
Fixes https://github.com/langchain-ai/langgraph/issues/5795
* Must use `category=None` on decorator so that we get type checking
support but no dupe warning
* Fixed tuple on `confix_type` warning causing false warning
I suppose that the code snippets are intended to run each on its own. To
guarantee this the snippet for the example:
"Write long-temr memory from tools"
needs to include `RunnableConfig`
Same also for the second commit of this pull request.
The other commits are about similar issues, where imports are missing to
make a snippet executable on its own.
---------
Signed-off-by: Kai Wendel <kai.wendel@iws.uni-stuttgart.de>
Co-authored-by: Eugene Yurtsev <eugene@langchain.dev>
Co-authored-by: Lauren Hirata Singh <lauren@langchain.dev>
- **Description:** Update the type annotations in create_react_agent to
allow one to provide a callable for the model that uses bind_tools and
returns a Runnable[LanguageModelInput, BaseMessage]
- **Issue:** #5739
---------
Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>
Fixes#5735
Implement context coercion functionality for LangGraph runtime to
improve API usability.
Key changes:
- Added `_coerce_context` function in `pregel/main.py`
- Supports coercion for:
- Pydantic BaseModel
- Dataclasses
- TypedDict
- Comprehensive test coverage added in `tests/test_runtime.py`
- Handles edge cases like None context and missing fields
The implementation allows users to pass dictionaries as context, which
will be automatically converted to the expected schema type, making the
API more flexible and user-friendly.
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
Co-authored-by: Sydney Runkle <sydneymarierunkle@gmail.com>
Co-authored-by: Sydney Runkle <54324534+sydney-runkle@users.noreply.github.com>
Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>
## Description
Previously, when a tool returned `Command` to update the graph's state,
the `_validate_tool_command` method in `ToolNode` would raise a
`ValueError` if the `messages_update` list contained only a
`RemoveMessage(id=REMOVE_ALL_MESSAGES)` object. This was because the
validation logic expected a matching `ToolMessage` for the tool call and
did not account for this specific state-clearing scenario.
This commit modifies the validation logic to check if the
`messages_update` list contains a single
`RemoveMessage(id=REMOVE_ALL_MESSAGES)` element. If this condition is
met, the `ToolMessage` validation is bypassed, allowing a tool to clear
the entire message history without causing a validation error.
A new test case, `test_tool_node_command_remove_all_messages`, has been
added to `tests/test_tool_node.py` to verify this change and prevent
future regressions.
## Example
Here is a self-contained example that illustrates the problem and the
fix. Without this change, the code block for `Example 2` would raise a
`ValueError`.
```python
from typing import Annotated, List
from langchain_core.messages import (
AIMessage,
AnyMessage,
HumanMessage,
RemoveMessage,
ToolMessage,
)
from langchain_core.tools import InjectedToolCallId, tool
from langchain_openai import ChatOpenAI
from langgraph.graph import END, StateGraph, add_messages
from langgraph.graph.message import REMOVE_ALL_MESSAGES
from langgraph.prebuilt import InjectedState, ToolNode
from langgraph.types import Command
from pydantic import BaseModel, Field
# Agent state tracks current and all messages
class AgentState(BaseModel):
messages: Annotated[List[AnyMessage], add_messages] = Field(
default_factory=list, description="Current conversation messages."
)
all_messages: Annotated[List[AnyMessage], add_messages] = Field(
default_factory=list, description="All messages, including removed ones."
)
# Tool to clear history if long enough, otherwise returns a warning
@tool
def clear_history_tool(
state: Annotated[AgentState, InjectedState],
tool_call_id: Annotated[str, InjectedToolCallId],
):
"""Clears message history if it's long enough."""
if len(state.messages) < 3:
return Command(
update={
"messages": [
ToolMessage(
"History is not long enough to be cleared. Please try again.",
tool_call_id=tool_call_id,
)
]
}
)
else:
return Command(
update={
"messages": [RemoveMessage(id=REMOVE_ALL_MESSAGES)],
"all_messages": state.messages
+ [
ToolMessage(
"History has been successfully cleared.",
tool_call_id=tool_call_id,
)
],
}
)
# Bind the tool to the model
model = ChatOpenAI(model="gpt-4o-mini").bind_tools([clear_history_tool])
def model_node(state: AgentState):
return {"messages": [model.invoke(state.messages)]}
# Build the agent graph
graph_builder = StateGraph(AgentState)
graph_builder.add_node("model", model_node)
graph_builder.add_node("tools", ToolNode([clear_history_tool]))
graph_builder.set_entry_point("model")
graph_builder.add_edge("model", "tools")
graph_builder.add_edge("tools", END)
graph = graph_builder.compile()
def print_messages(header, messages):
print(f"\n{header}")
for message in messages:
message.pretty_print()
### Example 1: Not enough history to clear
state_1 = AgentState(
messages=[HumanMessage(content="Please clear my message history.")]
)
output_1 = graph.invoke(state_1)
print_messages("First call: State 'messages'", output_1["messages"])
print_messages("First call: State 'all_messages'", output_1["all_messages"])
### Example 2: History is cleared
state_2 = AgentState(
messages=[
HumanMessage(content="Will this PR get merged?"),
AIMessage(content="Maybe, if it's good enough."),
HumanMessage(content="Please clear my message history."),
]
)
# Without the changes in this PR, the following line will raise a ValueError
output_2 = graph.invoke(state_2)
print_messages("Second call: State 'messages'", output_2["messages"])
print_messages("Second call: State 'all_messages'", output_2["all_messages"])
```
### Outputs
*Without the changes in this PR:*
```
First call: State 'messages'
================================ Human Message =================================
Please clear my message history.
================================== Ai Message ==================================
Tool Calls:
clear_history_tool (ba421ac3-1e1a-4208-a8f6-c5500ee0abcc)
Call ID: ba421ac3-1e1a-4208-a8f6-c5500ee0abcc
Args:
================================= Tool Message =================================
Name: clear_history_tool
History is not long enough to be cleared. Please try again.
First call: State 'all_messages'
Traceback (most recent call last):
File "main.py", line 114, in <module>
output_2 = graph.invoke(state_2)
^^^^^^^^^^^^^^^^^^^^^
File ".venv/lib/python3.11/site-packages/langgraph/pregel/__init__.py", line 2844, in invoke
for chunk in self.stream(
File ".venv/lib/python3.11/site-packages/langgraph/pregel/__init__.py", line 2534, in stream
for _ in runner.tick(
File ".venv/lib/python3.11/site-packages/langgraph/prebuilt/tool_node.py", line 241, in _func
outputs = [
^
File ".venv/lib/python3.11/concurrent/futures/_base.py", line 619, in result_iterator
yield _result_or_cancel(fs.pop())
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ".venv/lib/python3.11/concurrent/futures/_base.py", line 317, in _result_or_cancel
return fut.result(timeout)
^^^^^^^^^^^^^^^^^^^
File ".venv/lib/python3.11/concurrent/futures/_base.py", line 449, in result
return self.__get_result()
^^^^^^^^^^^^^^^^^^^
File ".venv/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
raise self._exception
File ".venv/lib/python3.11/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ".venv/lib/python3.11/site-packages/langchain_core/runnables/config.py", line 555, in _wrapped_fn
return contexts.pop().run(fn, *args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ".venv/lib/python3.11/site-packages/langgraph/prebuilt/tool_node.py", line 353, in _run_one
return self._validate_tool_command(response, call, input_type)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ".venv/lib/python3.11/site-packages/langgraph/prebuilt/tool_node.py", line 616, in _validate_tool_command
raise ValueError(
ValueError: Expected to have a matching ToolMessage in Command.update for tool 'clear_history_tool', got: [RemoveMessage(content='', additional_kwargs={}, response_metadata={}, id='__remove_all__')]. Every tool call (LLM requesting to call a tool) in the message history MUST have a corresponding ToolMessage. You can fix it by modifying the tool to return `Command(update={"messages": [ToolMessage("Success", tool_call_id=tool_call_id), ...]}, ...)`.
```
*With the changes in this PR:*
```
First call: State 'messages'
================================ Human Message =================================
Please clear my message history.
================================== Ai Message ==================================
Tool Calls:
clear_history_tool (ba421ac3-1e1a-4208-a8f6-c5500ee0abcc)
Call ID: ba421ac3-1e1a-4208-a8f6-c5500ee0abcc
Args:
================================= Tool Message =================================
Name: clear_history_tool
History is not long enough to be cleared. Please try again.
First call: State 'all_messages'
Second call: State 'messages'
Second call: State 'all_messages'
================================ Human Message =================================
Will this PR get merged?
================================== Ai Message ==================================
Maybe, if it's good enough.
================================ Human Message =================================
Please clear my message history.
================================== Ai Message ==================================
Tool Calls:
clear_history_tool (499b1be3-6df1-493f-85e5-8d7e429dead8)
Call ID: 499b1be3-6df1-493f-85e5-8d7e429dead8
Args:
================================= Tool Message =================================
Name: clear_history_tool
History has been successfully cleared.
```
## Twitter handle
[@samuelpullely](https://x.com/samuelpullely)
---------
Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>
