mirror of
https://github.com/langchain-ai/langgraph.git
synced 2026-08-21 07:02:25 +02:00
357 lines
14 KiB
Plaintext
357 lines
14 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"# How to return structured output with a ReAct style agent\n",
|
|
"\n",
|
|
"You might want your agent to return its output in a structured format. For example, if the output of the agent is used by some other downstream software, you may want the output to be in the same structured format every time the agent is invoked to ensure consistency.\n",
|
|
"\n",
|
|
"This notebook will walk through two different options for forcing a function calling agent to structure its output. We will be using a basic [ReAct agent](https://langchain-ai.github.io/langgraph/how-tos/create-react-agent/) (a model node and a tool-calling node) together with a third node at the end that will format response for the user. Both of the options will use the same graph structure as shown in the diagram below, but will have different mechanisms under the hood.\n",
|
|
"\n",
|
|
"\n",
|
|
"\n",
|
|
"**Option 1**\n",
|
|
"\n",
|
|
"\n",
|
|
"\n",
|
|
"The first way you can force your tool calling agent to have structured output is to bind the output you would like as an additional tool for the `agent` node to use. In contrast to the basic ReAct agent, the `agent` node in this case is not selecting between `tools` and `END` but rather selecting between the specific tools it calls. The expected flow in this case is that the LLM in the `agent` node will first select the action tool, and after receiving the action tool output it will call the response tool, which will then route to the `respond` node which simply structures the arguments from the `agent` node tool call.\n",
|
|
"\n",
|
|
"**Pros and Cons**\n",
|
|
"\n",
|
|
"The benefit to this format is that you only need one LLM, and can save money and latency because of this. The downside to this option is that it isn't guaranteed that the single LLM will call the correct tool when you want it to. We can help the LLM by setting `tool_choice` to `any` when we use `bind_tools` which forces the LLM to select at least one tool at every turn, but this is far from a fool proof strategy. In addition, another downside is that the agent might call *multiple* tools, so we need to check for this explicitly in our routing function (or if we are using OpenAI we an set `parallell_tool_calling=False` to ensure only one tool is called at a time).\n",
|
|
"\n",
|
|
"**Option 2**\n",
|
|
"\n",
|
|
"\n",
|
|
"\n",
|
|
"The second way you can force your tool calling agent to have structured output is to use a second LLM (in this case `model_with_structured_output`) to respond to the user. \n",
|
|
"\n",
|
|
"In this case, you will define a basic ReAct agent normally, but instead of having the `agent` node choose between the `tools` node and ending the conversation, the `agent` node will choose between the `tools` node and the `respond` node. The `respond` node will contain a second LLM that uses structured output, and once called will return directly to the user. You can think of this method as basic ReAct with one extra step before responding to the user. \n",
|
|
"\n",
|
|
"**Pros and Cons**\n",
|
|
"\n",
|
|
"The benefit of this method is that it guarantees structured output (as long as `.with_structured_output` works as expected with the LLM). The downside to using this approach is that it requires making an additional LLM call before responding to the user, which can increase costs as well as latency. In addition, by not providing the `agent` node LLM with information about the desired output schema there is a risk that the `agent` LLM will fail to call the correct tools required to answer in the correct output schema.\n",
|
|
"\n",
|
|
"Note that both of these options will follow the exact same graph structure (see the diagram above), in that they are both exact replicas of the basic ReAct architecture but with a `respond` node before the end.\n",
|
|
"\n",
|
|
"## Setup\n",
|
|
"\n",
|
|
"For our setup we need to define how we want to structure our output, define our graph state, and also our tools and the models we are going to use.\n",
|
|
"\n",
|
|
"To use structured output, we will use the `with_structured_output` method from LangChain, which you can read more about [here](https://python.langchain.com/v0.2/docs/how_to/structured_output/).\n",
|
|
"\n",
|
|
"We are going to use a single tool in this example for finding the weather, and will return a structured weather response to the user."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 9,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from pydantic import BaseModel, Field\n",
|
|
"from typing import Literal\n",
|
|
"from langchain_core.tools import tool\n",
|
|
"from langchain_anthropic import ChatAnthropic\n",
|
|
"from langgraph.graph import MessagesState\n",
|
|
"\n",
|
|
"class WeatherResponse(BaseModel):\n",
|
|
" \"\"\"Respond to the user with this\"\"\"\n",
|
|
" temperature: float = Field(description=\"The temperature in fahrenheit\")\n",
|
|
" wind_directon: str = Field(description=\"The direction of the wind in abbreviated form\")\n",
|
|
" wind_speed: float = Field(description=\"The speed of the wind in km/h\")\n",
|
|
"\n",
|
|
"# Inherit 'messages' key from MessagesState, which is a list of chat messages \n",
|
|
"class AgentState(MessagesState):\n",
|
|
" # Final structured response from the agent\n",
|
|
" final_response: WeatherResponse\n",
|
|
"\n",
|
|
"@tool\n",
|
|
"def get_weather(city: Literal[\"nyc\", \"sf\"]):\n",
|
|
" \"\"\"Use this to get weather information.\"\"\"\n",
|
|
" if city == \"nyc\":\n",
|
|
" return \"It is cloudy in NYC, with 5 mph winds in the North-East direction and a temperature of 70 degrees\"\n",
|
|
" elif city == \"sf\":\n",
|
|
" return \"It is 75 degrees and sunny in SF, with 3 mph winds in the South-East direction\"\n",
|
|
" else:\n",
|
|
" raise AssertionError(\"Unknown city\")\n",
|
|
" \n",
|
|
"tools = [get_weather]\n",
|
|
" \n",
|
|
"model = ChatAnthropic(model=\"claude-3-opus-20240229\")\n",
|
|
" \n",
|
|
"model_with_tools = model.bind_tools(tools)\n",
|
|
"model_with_structured_output = model.with_structured_output(WeatherResponse)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Option 1: Bind output as tool\n",
|
|
"\n",
|
|
"Let's now examine how we would use the single LLM option.\n",
|
|
"\n",
|
|
"### Define Graph\n",
|
|
"\n",
|
|
"The graph definition is very similar to the one above, the only difference is we no longer call an LLM in the `response` node, and instead bind the `WeatherResponse` tool to our LLM that already contains the `get_weather` tool."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 10,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from langgraph.graph import StateGraph, END\n",
|
|
"from langgraph.prebuilt import ToolNode\n",
|
|
"\n",
|
|
"tools = [get_weather, WeatherResponse]\n",
|
|
"\n",
|
|
"# Force the model to use tools by passing tool_choice=\"any\" \n",
|
|
"model_with_response_tool = model.bind_tools(tools,tool_choice=\"any\")\n",
|
|
"\n",
|
|
"# Define the function that calls the model\n",
|
|
"def call_model(state: AgentState):\n",
|
|
" response = model_with_response_tool.invoke(state['messages'])\n",
|
|
" # We return a list, because this will get added to the existing list\n",
|
|
" return {\"messages\": [response]}\n",
|
|
"\n",
|
|
"# Define the function that responds to the user\n",
|
|
"def respond(state: AgentState):\n",
|
|
" # Construct the final answer from the arguments of the last tool call\n",
|
|
" response = WeatherResponse(**state['messages'][-1].tool_calls[0]['args'])\n",
|
|
" # We return the final answer\n",
|
|
" return {\"final_response\": response}\n",
|
|
"\n",
|
|
"# Define the function that determines whether to continue or not\n",
|
|
"def should_continue(state: AgentState):\n",
|
|
" messages = state[\"messages\"]\n",
|
|
" last_message = messages[-1]\n",
|
|
" # If there is only one tool call and it is the response tool call we respond to the user\n",
|
|
" if len(last_message.tool_calls) == 1 and last_message.tool_calls[0]['name'] == \"WeatherResponse\":\n",
|
|
" return \"respond\"\n",
|
|
" # Otherwise we will use the tool node again\n",
|
|
" else:\n",
|
|
" return \"continue\"\n",
|
|
"\n",
|
|
"# Define a new graph\n",
|
|
"workflow = StateGraph(AgentState)\n",
|
|
"\n",
|
|
"# Define the two nodes we will cycle between\n",
|
|
"workflow.add_node(\"agent\", call_model)\n",
|
|
"workflow.add_node(\"respond\", respond)\n",
|
|
"workflow.add_node(\"tools\", ToolNode(tools))\n",
|
|
"\n",
|
|
"# Set the entrypoint as `agent`\n",
|
|
"# This means that this node is the first one called\n",
|
|
"workflow.set_entry_point(\"agent\")\n",
|
|
"\n",
|
|
"# We now add a conditional edge\n",
|
|
"workflow.add_conditional_edges(\n",
|
|
" \"agent\",\n",
|
|
" should_continue,\n",
|
|
" {\n",
|
|
" \"continue\": \"tools\",\n",
|
|
" \"respond\": \"respond\",\n",
|
|
" },\n",
|
|
")\n",
|
|
"\n",
|
|
"workflow.add_edge(\"tools\", \"agent\")\n",
|
|
"workflow.add_edge(\"respond\", END)\n",
|
|
"graph = workflow.compile()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Usage\n",
|
|
"\n",
|
|
"Now we can run our graph to check that it worked as intended:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 11,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"answer = graph.invoke(input={\"messages\": [(\"human\", \"what's the weather in SF?\")]})['final_response']"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 12,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"WeatherResponse(temperature=75.0, wind_directon='SE', wind_speed=3.0)"
|
|
]
|
|
},
|
|
"execution_count": 12,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"answer"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Again, the agent returned a `WeatherResponse` object as we expected."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Option 2: 2 LLMs\n",
|
|
"\n",
|
|
"Let's now dive into how we would use a second LLM to force structured output.\n",
|
|
"\n",
|
|
"### Define Graph\n",
|
|
"\n",
|
|
"We can now define our graph:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 13,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from langgraph.graph import StateGraph, END\n",
|
|
"from langgraph.prebuilt import ToolNode\n",
|
|
"from langchain_core.messages import HumanMessage\n",
|
|
"\n",
|
|
"# Define the function that calls the model\n",
|
|
"def call_model(state: AgentState):\n",
|
|
" response = model_with_tools.invoke(state['messages'])\n",
|
|
" # We return a list, because this will get added to the existing list\n",
|
|
" return {\"messages\": [response]}\n",
|
|
"\n",
|
|
"# Define the function that responds to the user\n",
|
|
"def respond(state: AgentState):\n",
|
|
" # We call the model with structured output in order to return the same format to the user every time\n",
|
|
" # state['messages'][-2] is the last ToolMessage in the convo, which we convert to a HumanMessage for the model to use\n",
|
|
" # We could also pass the entire chat history, but this saves tokens since all we care to structure is the output of the tool\n",
|
|
" response = model_with_structured_output.invoke([HumanMessage(content=state['messages'][-2].content)])\n",
|
|
" # We return the final answer\n",
|
|
" return {\"final_response\": response}\n",
|
|
"\n",
|
|
"# Define the function that determines whether to continue or not\n",
|
|
"def should_continue(state: AgentState):\n",
|
|
" messages = state[\"messages\"]\n",
|
|
" last_message = messages[-1]\n",
|
|
" # If there is no function call, then we respond to the user\n",
|
|
" if not last_message.tool_calls:\n",
|
|
" return \"respond\"\n",
|
|
" # Otherwise if there is, we continue\n",
|
|
" else:\n",
|
|
" return \"continue\"\n",
|
|
"\n",
|
|
"# Define a new graph\n",
|
|
"workflow = StateGraph(AgentState)\n",
|
|
"\n",
|
|
"# Define the two nodes we will cycle between\n",
|
|
"workflow.add_node(\"agent\", call_model)\n",
|
|
"workflow.add_node(\"respond\", respond)\n",
|
|
"workflow.add_node(\"tools\", ToolNode(tools))\n",
|
|
"\n",
|
|
"# Set the entrypoint as `agent`\n",
|
|
"# This means that this node is the first one called\n",
|
|
"workflow.set_entry_point(\"agent\")\n",
|
|
"\n",
|
|
"# We now add a conditional edge\n",
|
|
"workflow.add_conditional_edges(\n",
|
|
" \"agent\",\n",
|
|
" should_continue,\n",
|
|
" {\n",
|
|
" \"continue\": \"tools\",\n",
|
|
" \"respond\": \"respond\",\n",
|
|
" },\n",
|
|
")\n",
|
|
"\n",
|
|
"workflow.add_edge(\"tools\", \"agent\")\n",
|
|
"workflow.add_edge(\"respond\", END)\n",
|
|
"graph = workflow.compile()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"\n",
|
|
"### Usage\n",
|
|
"\n",
|
|
"We can now invoke our graph to verify that the output is being structured as desired:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 14,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"answer = graph.invoke(input={\"messages\": [(\"human\", \"what's the weather in SF?\")]})['final_response']"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 15,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"WeatherResponse(temperature=75.0, wind_directon='SE', wind_speed=4.83)"
|
|
]
|
|
},
|
|
"execution_count": 15,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"answer"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"As we can see, the agent returned a `WeatherResponse` object as we expected. If would now be easy to use this agent in a more complex software stack without having to worry about the output of the agent not matching the format expected from the next step in the stack."
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "Python 3 (ipykernel)",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.11.1"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 4
|
|
}
|