mirror of
https://github.com/langchain-ai/langgraph.git
synced 2026-09-09 11:17:53 +02:00
x
This commit is contained in:
Eugene Yurtsev
@@ -43,7 +43,7 @@ def human_approval(state: State):
|
||||
...
|
||||
else:
|
||||
# Do something else
|
||||
...
|
||||
...
|
||||
|
||||
|
||||
# Add the node to the graph
|
||||
@@ -276,5 +276,6 @@ Keep the following considerations in mind when using the `interrupt` function:
|
||||
|
||||
## Additional Resources 📚
|
||||
|
||||
- [**Conceptual Guide: Persistence**](https://langchain-ai.github.io/langgraph/concepts/persistence/#replay): Read the persistence guide for more context on replaying.
|
||||
- [**Conceptual Guide: Persistence**](persistence.md): Read the persistence guide for more context about persistence.
|
||||
- [**Conceptual Guide: Human-in-the-loop**](human_in_the_loop.md): Read the human-in-the-loop guide for more context on integrating human feedback into LangGraph applications using breakpoints.
|
||||
- [**How to View and Update Past Graph State**](../how-tos/human_in_the_loop/time-travel.ipynb): Step-by-step instructions for working with graph state that demonstrate the **replay** and **fork** actions.
|
||||
@@ -8,209 +8,157 @@
|
||||
|
||||
A **human-in-the-loop** (or "on-the-loop") workflow integrates human input into automated processes, allowing for decisions, validation, or corrections at key stages. This is especially useful in **LLM-based applications**, where the underlying model may generate occasional inaccuracies. In low-error-tolerance scenarios like compliance, decision-making, or content generation, human involvement ensures reliability by enabling review, correction, or override of model outputs.
|
||||
|
||||
## Interaction Patterns
|
||||
|
||||
1. **Approval**/**Rejection**: Pause the graph before a critical step, such as an API call, to review and approve the action. If the action is rejected, you can prevent the graph from executing the step, and potentially take an alternative action.
|
||||
2. **Editing**: Pause the graph to review and edit the agent's state. This is useful for correcting mistakes or updating the agent's state.
|
||||
3. **Input**: Explicitly request human input at a particular step in the graph. This is useful for collecting additional information or context to inform the agent's decision-making process.
|
||||
|
||||
## Use cases
|
||||
|
||||
Key use cases for **human-in-the-loop** workflows in LLM-based applications include:
|
||||
|
||||
1. **🛠️ [Reviewing tool calls](#reviewing-tool-calls)**: Humans can review, edit, or approve tool calls requested by the LLM before tool execution.
|
||||
1. **🛠️ [Reviewing tool calls](#review-and-edit)**: Humans can review, edit, or approve tool calls requested by the LLM before tool execution.
|
||||
2. **✅ Validating LLM outputs**: Humans can review, edit, or approve content generated by the LLM.
|
||||
3. **💡 Providing context**: Enable the LLM to explicitly request human input for clarification or additional details.
|
||||
4. **🔍 Debugging**: Investigate and correct errors in the LLM's decision-making process.
|
||||
|
||||
## Interrupt & Resume
|
||||
3. **💡 Providing context**: Enable the LLM to explicitly request human input for clarification or additional details or to support multi-turn conversations.
|
||||
|
||||
**Human-in-the-loop** workflow consists of four key steps:
|
||||
## Design Patterns
|
||||
|
||||
1. [**Persistence**](./persistence.md): the graph state is saved after each graph step, enabling **pausing** and **resuming** execution.
|
||||
2. [**Interrupting execution**](#interrupting-execution): the [`interrupt`](../reference/types.md#langgraph.types.interrupt) function is used to **pause** the graph at specific points for **human input**.
|
||||
3. [**Running the graph**](#run): the graph is executed until it reaches the **breakpoint**.
|
||||
4. [**Resuming execution**](#resuming-execution): the [`Command`](../reference/types.md#langgraph.types.Command) primitive allows **resuming** execution based on **human input**.
|
||||
1. **Approval**: Pause the graph before a critical step, such as an API call, to review and approve the action. If the action is rejected, you can prevent the graph from executing the step, and potentially take an alternative action.
|
||||
2. **Editing**: Pause the graph to review and edit the agent's state. This is useful for correcting mistakes or updating the agent's state.
|
||||
3. **Input**: Explicitly request human input at a particular step in the graph. This is useful for collecting additional information or context to inform the agent's decision-making process or for supporting **multi-turn conversations**.
|
||||
|
||||
> **Note:** While there are other ways to set breakpoints (e.g., static breakpoints or dynamic exceptions) and resume execution (e.g., by relying on state updates `graph.update_state`), this guide focuses on the `interrupt` function and `Command` primitive as the recommended methods.
|
||||
|
||||
### Interrupt
|
||||
### Approval
|
||||
|
||||
Use the [interrupt](../reference/types.md/#langgraph.types.interrupt) function to **pause** the graph at specific points to collect user input. The `interrupt` function surfaces interrupt information to the client, allowing you to collect user input, validate the graph state, or make decisions before resuming execution. The graph must be compiled with a [checkpointer](./persistence.md) so graph execution can be paused and resumed.
|
||||
<figure markdown="1">
|
||||
{: style="max-height:400px"}
|
||||
<figcaption>Depending on the human's approval or rejection, the graph can proceed with the action or take an alternative path.</figcaption>
|
||||
</figure>
|
||||
|
||||
Pause the graph before a critical step, such as an API call, to review and approve the action. If the action is rejected, you can prevent the graph from executing the step, and potentially take an alternative action.
|
||||
|
||||
```python
|
||||
from langgraph.types import interrupt
|
||||
|
||||
def node(state: State):
|
||||
def human_approval(state: State):
|
||||
...
|
||||
# Pause the graph and wait for user input.
|
||||
answer = interrupt(
|
||||
is_approved = interrupt(
|
||||
{
|
||||
"question": "What is your age?",
|
||||
"question": "Is this correct?",
|
||||
# Surface the output that should be
|
||||
# reviewed and approved by the human.
|
||||
"llm_output": state["llm_output"]
|
||||
}
|
||||
)
|
||||
# Answer will be assigned a value when the graph resumes (see below).
|
||||
print(f"Value received from interrupt: {answer}")
|
||||
# Do something with the answer.
|
||||
...
|
||||
|
||||
graph_builder.add_node("node", node)
|
||||
|
||||
# The checkpointer is required for the `interrupt` function to work.
|
||||
graph = graph_builder.compile(checkpointer=checkpointer)
|
||||
```
|
||||
|
||||
??? warning "Graph execution resumes from the beginning of the node not the exact point of the `interrupt`"
|
||||
|
||||
### Run
|
||||
|
||||
**Run the graph** and observe the `interrupt` function in action:
|
||||
|
||||
```python
|
||||
# Run the graph up to the breakpoint
|
||||
thread_config = {"configurable": {"thread_id": "some_id"}}
|
||||
for event in graph.stream(inputs, thread_config, stream_mode="values"):
|
||||
print(event)
|
||||
```
|
||||
|
||||
```pycon
|
||||
{'__interrupt__': (
|
||||
Interrupt(
|
||||
value={'question': 'what is your age?'},
|
||||
resumable=True,
|
||||
ns=['node:5df255f7-d683-1a99-b7c8-00dd534aed8e'],
|
||||
when='during'
|
||||
),
|
||||
)
|
||||
}
|
||||
```
|
||||
|
||||
??? note "Using with `invoke` and `ainvoke`"
|
||||
|
||||
`invoke` and `ainvoke` do not return the interrupt information. To access this information, you must use the [get_state](../reference/graphs.md#langgraph.graph.graph.CompiledGraph.get_state) method to retrieve the graph state after calling `invoke` or `ainvoke`.
|
||||
|
||||
```python
|
||||
# Run the graph up to the breakpoint
|
||||
result = graph.invoke(inputs, thread_config)
|
||||
# Get the graph state to get interrupt information.
|
||||
state = graph.get_state(thread_config)
|
||||
# Resume the graph with the user's input.
|
||||
graph.invoke(Command(resume={"age": "25"}), thread_config)
|
||||
```
|
||||
|
||||
### Resume
|
||||
|
||||
Once you have collected user input, you can **resume** the graph execution using the [Command](../reference/types.md#langgraph.types.Command) primitive. The `Command` primitive provides several options to control and modify the graph's state during resumption:
|
||||
|
||||
```python
|
||||
graph.invoke(Command(resume={"age": "25"}), thread_config)
|
||||
```
|
||||
|
||||
You should see the following output printed by to the `print` function in the `node` function:
|
||||
|
||||
```pycon
|
||||
Value received from interrupt: {'age': '25'}
|
||||
```
|
||||
|
||||
## Interaction Patterns
|
||||
|
||||
1. **Approval**/**Rejection**: Pause the graph before a critical step, such as an API call, to review and approve the action. If the action is rejected.
|
||||
|
||||
2. **Editing**: Pause the graph to review and edit the agent's state. This is useful for correcting mistakes or updating the agent's state.
|
||||
|
||||
|
||||
### Approval/Rejection
|
||||
|
||||
```python
|
||||
def node(state):
|
||||
...
|
||||
# Pause the graph and wait for user input.
|
||||
approval = interrupt(
|
||||
{
|
||||
"question": "Do you approve this action?",
|
||||
# Surface the current state or any other relevant information.
|
||||
"state": state,
|
||||
}
|
||||
)
|
||||
|
||||
if not isinstance(approval, bool):
|
||||
raise ValueError("Approval must be a boolean value.")
|
||||
|
||||
if approval:
|
||||
# Perform the action.
|
||||
if is_approved:
|
||||
# Proceed with the action
|
||||
...
|
||||
else:
|
||||
# Skip the action.
|
||||
# Do something else
|
||||
...
|
||||
|
||||
# Add the node to the graph in an appropriate location
|
||||
# and connect it to the relevant nodes.
|
||||
graph_builder.add_node("human_approval", human_approval)
|
||||
graph = graph_builder.compile(checkpointer=checkpointer)
|
||||
|
||||
...
|
||||
|
||||
# After running the graph and hitting the breakpoint, the graph will pause.
|
||||
# Resume it with either an approval or rejection.
|
||||
thread_config = {"configurable": {"thread_id": "some_id"}}
|
||||
graph.invoke(Command(resume=True), config=thread_config)
|
||||
```
|
||||
|
||||
### Editing
|
||||
|
||||
### Edit
|
||||
|
||||
<figure markdown="1">
|
||||
{: style="max-height:400px"}
|
||||
<figcaption>A human can review and edit the output from the LLM before proceeding. This is particularly
|
||||
critical in applications where the tool calls requested by the LLM may be sensitive or require human oversight.
|
||||
</figcaption>
|
||||
</figure>
|
||||
|
||||
|
||||
```python
|
||||
def node(state):
|
||||
from langgraph.types import interrupt
|
||||
|
||||
def human_editing(state: State):
|
||||
...
|
||||
# Pause the graph and wait for user input.
|
||||
user_input = interrupt(
|
||||
result = interrupt(
|
||||
# Interrupt information to surface to the client.
|
||||
# Can be any JSON serializable value.
|
||||
{
|
||||
"question": "Please provide additional information:",
|
||||
# Surface the current state or any other relevant information.
|
||||
"state": state,
|
||||
"task": "Review the output from the LLM and make any necessary edits.",
|
||||
"llm_output": state["llm_output"]
|
||||
}
|
||||
)
|
||||
|
||||
# Update the state with the edited text
|
||||
return {
|
||||
# some state update based on user input
|
||||
"llm_output": result["edited_text"]
|
||||
}
|
||||
|
||||
|
||||
# Add the node to the graph in an appropriate location
|
||||
# and connect it to the relevant nodes.
|
||||
graph_builder.add_node("human_editing", human_editing)
|
||||
graph = graph_builder.compile(checkpointer=checkpointer)
|
||||
|
||||
...
|
||||
|
||||
# After running the graph and hitting the breakpoint, the graph will pause.
|
||||
# Resume it with the edited text.
|
||||
thread_config = {"configurable": {"thread_id": "some_id"}}
|
||||
graph.invoke(
|
||||
Command(resume={"edited_text": "The edited text"}),
|
||||
config=thread_config
|
||||
)
|
||||
```
|
||||
|
||||
### Input
|
||||
### Multi-turn conversation (Input)
|
||||
|
||||
<figure markdown="1">
|
||||
{: style="max-height:400px"}
|
||||
<figcaption>A <strong>multi-turn conversation</strong> architecture where an <strong>agent</strong> and <strong>human node</strong> cycle back and forth until the agent decides to hand off the conversation to another agent or another part of the system.
|
||||
</figcaption>
|
||||
</figure>
|
||||
|
||||
A **multi-turn conversation** involves multiple back-and-forth interactions between an agent and a human, which can allow the agent to gather additional information from the human in a conversational manner.
|
||||
|
||||
This design pattern is useful in an LLM application consisting of [multiple agents](./multi_agent.md). One or more agents may need to carry out multi-turn conversations with a human, where the human provides input or feedback at different stages of the conversation. For simplicity, the agent implementation below is illustrated as a single node, but in reality
|
||||
it may be part of a larger graph consisting of multiple nodes and include a conditional edge.
|
||||
|
||||
```python
|
||||
from langgraph.types import interrupt
|
||||
|
||||
def node(state):
|
||||
def human_input(state: State):
|
||||
human_message = interrupt("human_input")
|
||||
return {
|
||||
"messages": [
|
||||
{
|
||||
"role": "human",
|
||||
"content": human_message
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
def agent(state: State):
|
||||
# Agent logic
|
||||
...
|
||||
# Pause the graph and wait for user input.
|
||||
human_message = interrupt()
|
||||
return {
|
||||
"messages": [human_message]
|
||||
}
|
||||
|
||||
|
||||
graph_builder.add_node("human_input", human_input)
|
||||
graph_builder.add_edge("human_input", "agent")
|
||||
graph = graph_builder.compile(checkpointer=checkpointer)
|
||||
|
||||
|
||||
# After running the graph and hitting the breakpoint, the graph will pause.
|
||||
# Resume it with the human's input.
|
||||
graph.invoke(
|
||||
Command(resume="hello!"),
|
||||
config=thread_config
|
||||
)
|
||||
```
|
||||
|
||||
###
|
||||
## Additional Resources 📚
|
||||
|
||||
## Use-cases
|
||||
|
||||
### Reviewing Tool Calls
|
||||
|
||||
Some user interaction patterns combine the concepts outlined above.
|
||||
|
||||
For example, many agents rely on [tool calling](https://python.langchain.com/docs/how_to/tool_calling/) to make decisions. Tool calling introduces unique challenges because the agent must get multiple aspects right:
|
||||
|
||||
1. **Selecting the correct tool**: The agent must choose the appropriate tool to call.
|
||||
2. **Providing accurate arguments**: The agent must pass the correct parameters to the tool.
|
||||
3. **Ensuring discretion**: Even if the tool call is technically correct, it might involve sensitive operations that require human approval.
|
||||
|
||||
By addressing these challenges, we can integrate **human-in-the-loop** processes to review and approve tool calls effectively.
|
||||
|
||||
```python
|
||||
# Compile our graph with a checkpointer and a breakpoint before the step to review the tool call from the LLM
|
||||
graph = builder.compile(checkpointer=checkpointer, interrupt_before=["human_review"])
|
||||
|
||||
# Run the graph up to the breakpoint
|
||||
for event in graph.stream(inputs, thread, stream_mode="values"):
|
||||
print(event)
|
||||
|
||||
# Review the tool call and update it, if needed, as the human_review node
|
||||
graph.update_state(thread, {"tool_call": "updated tool call"}, as_node="human_review")
|
||||
|
||||
# Otherwise, approve the tool call and proceed with the graph execution with no edits
|
||||
|
||||
# Continue the graph execution from either:
|
||||
# (1) the forked checkpoint created by human_review or
|
||||
# (2) the checkpoint saved when the tool call was originally made (no edits in human_review)
|
||||
for event in graph.stream(None, thread, stream_mode="values"):
|
||||
print(event)
|
||||
```
|
||||
|
||||
See [the how to review tool calls guide](../how-tos/human_in_the_loop/review-tool-calls.ipynb) for a details.
|
||||
- [**Conceptual Guide: Persistence**](persistence.md#replay): Read the persistence guide for more context on replaying.
|
||||
- [**Conceptual Guide: Breakpoints**](breakpoints.md): Read the breakpoints guide for more context on breakpoints.
|
||||
- [**How to Guides: Human-in-the-loop**](../how-tos/index.md#human-in-the-loop): Learn how to implement human-in-the-loop workflows in LangGraph.
|
||||
|
||||
Reference in New Issue
Block a user