mirror of
https://github.com/langchain-ai/langgraph.git
synced 2026-08-24 16:42:24 +02:00
Add prebuilt ToolNode
This commit is contained in:
Nuno Campos
@@ -0,0 +1,622 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "51466c8d-8ce4-4b3d-be4e-18fdbeda5f53",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Chat Agent Executor using prebuilt Tool Node\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"In this example we will build a chat executor that uses tool calling and the prebuilt ToolNode."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "7cbd446a-808f-4394-be92-d45ab818953c",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Setup\n",
|
||||
"\n",
|
||||
"First we need to install the packages required"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"id": "af4ce0ba-7596-4e5f-8bf8-0b0bd6e62833",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"!pip install --quiet -U langchain langchain_openai tavily-python"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "0abe11f4-62ed-4dc4-8875-3db21e260d1d",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Next, we need to set API keys for OpenAI (the LLM we will use) and Tavily (the search tool we will use)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "c903a1cf-2977-4e2d-ad7d-8b3946821d89",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import os\n",
|
||||
"import getpass\n",
|
||||
"\n",
|
||||
"os.environ[\"OPENAI_API_KEY\"] = getpass.getpass(\"OpenAI API Key:\")\n",
|
||||
"os.environ[\"TAVILY_API_KEY\"] = getpass.getpass(\"Tavily API Key:\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "f0ed46a8-effe-4596-b0e1-a6a29ee16f5c",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Optionally, we can set API key for [LangSmith tracing](https://smith.langchain.com/), which will give us best-in-class observability."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "95e25aec-7c9f-4a63-b143-225d0e9a79c3",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"os.environ[\"LANGCHAIN_TRACING_V2\"] = \"true\"\n",
|
||||
"os.environ[\"LANGCHAIN_API_KEY\"] = getpass.getpass(\"LangSmith API Key:\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "21ac643b-cb06-4724-a80c-2862ba4773f1",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Set up the tools\n",
|
||||
"\n",
|
||||
"We will first define the tools we want to use.\n",
|
||||
"For this simple example, we will use a built-in search tool via Tavily.\n",
|
||||
"However, it is really easy to create your own tools - see documentation [here](https://python.langchain.com/docs/modules/agents/tools/custom_tools) on how to do that.\n",
|
||||
"\n",
|
||||
"**MODIFICATION**\n",
|
||||
"\n",
|
||||
"We don't need a ToolExecutor when using ToolNode.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"id": "d7ef57dd-5d6e-4ad3-9377-a92201c1310e",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from langchain_community.tools.tavily_search import TavilySearchResults\n",
|
||||
"\n",
|
||||
"tools = [TavilySearchResults(max_results=1)]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "5497ed70-fce3-47f1-9cad-46f912bad6a5",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Set up the model\n",
|
||||
"\n",
|
||||
"Now we need to load the chat model we want to use.\n",
|
||||
"Importantly, this should satisfy two criteria:\n",
|
||||
"\n",
|
||||
"1. It should work with messages. We will represent all agent state in the form of messages, so it needs to be able to work well with them.\n",
|
||||
"2. It should work with tool calling. This means it should be a model that implements `.bind_tools()`.\n",
|
||||
"\n",
|
||||
"Note: these model requirements are not requirements for using LangGraph - they are just requirements for this one example.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"id": "892b54b9-75f0-4804-9ed0-88b5e5532989",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from langchain_openai import ChatOpenAI\n",
|
||||
"\n",
|
||||
"model = ChatOpenAI(temperature=0)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "a77995c0-bae2-4cee-a036-8688a90f05b9",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"\n",
|
||||
"After we've done this, we should make sure the model knows that it has these tools available to call.\n",
|
||||
"We can do this by converting the LangChain tools into the format for OpenAI function calling, and then bind them to the model class.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"id": "cd3cbae5-d92c-4559-a4aa-44721b80d107",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"model = model.bind_tools(tools)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "8e8b9211-93d0-4ad5-aa7a-9c09099c53ff",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Define the agent state\n",
|
||||
"\n",
|
||||
"The main type of graph in `langgraph` is the `StatefulGraph`.\n",
|
||||
"This graph is parameterized by a state object that it passes around to each node.\n",
|
||||
"Each node then returns operations to update that state.\n",
|
||||
"These operations can either SET specific attributes on the state (e.g. overwrite the existing values) or ADD to the existing attribute.\n",
|
||||
"Whether to set or add is denoted by annotating the state object you construct the graph with.\n",
|
||||
"\n",
|
||||
"For this example, the state we will track will just be a list of messages.\n",
|
||||
"We want each node to just add messages to that list.\n",
|
||||
"Therefore, we will use a `TypedDict` with one key (`messages`) and annotate it so that the `messages` attribute is always added to.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"id": "ea793afa-2eab-4901-910d-6eed90cd6564",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from typing import TypedDict, Annotated, Sequence\n",
|
||||
"import operator\n",
|
||||
"from langchain_core.messages import BaseMessage\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"class AgentState(TypedDict):\n",
|
||||
" messages: Annotated[Sequence[BaseMessage], operator.add]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "e03c5094-9297-4d19-a04e-3eedc75cefb4",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Define the nodes\n",
|
||||
"\n",
|
||||
"We now need to define a few different nodes in our graph.\n",
|
||||
"In `langgraph`, a node can be either a function or a [runnable](https://python.langchain.com/docs/expression_language/).\n",
|
||||
"There are two main nodes we need for this:\n",
|
||||
"\n",
|
||||
"1. The agent: responsible for deciding what (if any) actions to take.\n",
|
||||
"2. **MODIFICATION** The prebuilt ToolNode, given the list of tools. This will take tool calls from the most recent AIMessage, execute them, and return the result as ToolMessages.\n",
|
||||
"\n",
|
||||
"We will also need to define some edges.\n",
|
||||
"Some of these edges may be conditional.\n",
|
||||
"The reason they are conditional is that based on the output of a node, one of several paths may be taken.\n",
|
||||
"The path that is taken is not known until that node is run (the LLM decides).\n",
|
||||
"\n",
|
||||
"1. Conditional Edge: after the agent is called, we should either:\n",
|
||||
" a. If the agent said to take an action, then the function to invoke tools should be called\n",
|
||||
" b. If the agent said that it was finished, then it should finish\n",
|
||||
"2. Normal Edge: after the tools are invoked, it should always go back to the agent to decide what to do next\n",
|
||||
"\n",
|
||||
"Let's define the nodes, as well as a function to decide how what conditional edge to take.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 11,
|
||||
"id": "3b541bb9-900c-40d0-964d-7b5dfee30667",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from langgraph.prebuilt import ToolNode\n",
|
||||
"import json\n",
|
||||
"from langchain_core.messages import FunctionMessage\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"# Define the function that determines whether to continue or not\n",
|
||||
"def should_continue(state):\n",
|
||||
" messages = state[\"messages\"]\n",
|
||||
" last_message = messages[-1]\n",
|
||||
" # If there are no tool calls, then we finish\n",
|
||||
" if not last_message.tool_calls:\n",
|
||||
" return \"end\"\n",
|
||||
" # Otherwise if there is, we continue\n",
|
||||
" else:\n",
|
||||
" return \"continue\"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"# Define the function that calls the model\n",
|
||||
"def call_model(state):\n",
|
||||
" messages = state[\"messages\"]\n",
|
||||
" response = model.invoke(messages)\n",
|
||||
" # We return a list, because this will get added to the existing list\n",
|
||||
" return {\"messages\": [response]}\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"# Define the function to execute tools\n",
|
||||
"tool_node = ToolNode(tools)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "ffd6e892-946c-4899-8cc0-7c9291c1f73b",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Define the graph\n",
|
||||
"\n",
|
||||
"We can now put it all together and define the graph!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 12,
|
||||
"id": "813ae66c-3b58-4283-a02a-36da72a2ab90",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from langgraph.graph import StateGraph, END\n",
|
||||
"\n",
|
||||
"# Define a new graph\n",
|
||||
"workflow = StateGraph(AgentState)\n",
|
||||
"\n",
|
||||
"# Define the two nodes we will cycle between\n",
|
||||
"workflow.add_node(\"agent\", call_model)\n",
|
||||
"workflow.add_node(\"action\", tool_node)\n",
|
||||
"\n",
|
||||
"# Set the entrypoint as `agent`\n",
|
||||
"# This means that this node is the first one called\n",
|
||||
"workflow.set_entry_point(\"agent\")\n",
|
||||
"\n",
|
||||
"# We now add a conditional edge\n",
|
||||
"workflow.add_conditional_edges(\n",
|
||||
" # First, we define the start node. We use `agent`.\n",
|
||||
" # This means these are the edges taken after the `agent` node is called.\n",
|
||||
" \"agent\",\n",
|
||||
" # Next, we pass in the function that will determine which node is called next.\n",
|
||||
" should_continue,\n",
|
||||
" # Finally we pass in a mapping.\n",
|
||||
" # The keys are strings, and the values are other nodes.\n",
|
||||
" # END is a special node marking that the graph should finish.\n",
|
||||
" # What will happen is we will call `should_continue`, and then the output of that\n",
|
||||
" # will be matched against the keys in this mapping.\n",
|
||||
" # Based on which one it matches, that node will then be called.\n",
|
||||
" {\n",
|
||||
" # If `tools`, then we call the tool node.\n",
|
||||
" \"continue\": \"action\",\n",
|
||||
" # Otherwise we finish.\n",
|
||||
" \"end\": END,\n",
|
||||
" },\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"# We now add a normal edge from `tools` to `agent`.\n",
|
||||
"# This means that after `tools` is called, `agent` node is called next.\n",
|
||||
"workflow.add_edge(\"action\", \"agent\")\n",
|
||||
"\n",
|
||||
"# Finally, we compile it!\n",
|
||||
"# This compiles it into a LangChain Runnable,\n",
|
||||
"# meaning you can use it as you would any other runnable\n",
|
||||
"app = workflow.compile()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "547c3931-3dae-4281-ad4e-4b51305594d4",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Use it!\n",
|
||||
"\n",
|
||||
"We can now use it!\n",
|
||||
"This now exposes the [same interface](https://python.langchain.com/docs/expression_language/) as all other LangChain runnables."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 14,
|
||||
"id": "8edb04b9-40b6-46f1-a7a8-4b2d8aba7752",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"/Users/nuno/dev/langgraph/.venv/lib/python3.11/site-packages/langchain_core/messages/ai.py:52: UserWarning: New langchain packages are available that more efficiently handle tool calling. Please upgrade your packages to versions that set message tool calls. e.g., `pip install --upgrade langchain-anthropic`, pip install--upgrade langchain-openai`, etc.\n",
|
||||
" warnings.warn(\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"{'messages': [HumanMessage(content='what is the weather in sf'),\n",
|
||||
" AIMessage(content='', additional_kwargs={'tool_calls': [{'id': 'call_Q7l0TopyJxaly7xM9Vq2aGxO', 'function': {'arguments': '{\"query\":\"weather in San Francisco\"}', 'name': 'tavily_search_results_json'}, 'type': 'function'}]}, response_metadata={'token_usage': {'completion_tokens': 21, 'prompt_tokens': 87, 'total_tokens': 108}, 'model_name': 'gpt-3.5-turbo', 'system_fingerprint': 'fp_b28b39ffa8', 'finish_reason': 'tool_calls', 'logprobs': None}, id='run-d39a73db-e37a-46ce-9476-73fe9eb84b2e-0', tool_calls=[{'name': 'tavily_search_results_json', 'args': {'query': 'weather in San Francisco'}, 'id': 'call_Q7l0TopyJxaly7xM9Vq2aGxO'}]),\n",
|
||||
" ToolMessage(content='[{\"url\": \"https://www.weatherapi.com/\", \"content\": \"{\\'location\\': {\\'name\\': \\'San Francisco\\', \\'region\\': \\'California\\', \\'country\\': \\'United States of America\\', \\'lat\\': 37.78, \\'lon\\': -122.42, \\'tz_id\\': \\'America/Los_Angeles\\', \\'localtime_epoch\\': 1712797953, \\'localtime\\': \\'2024-04-10 18:12\\'}, \\'current\\': {\\'last_updated_epoch\\': 1712797200, \\'last_updated\\': \\'2024-04-10 18:00\\', \\'temp_c\\': 21.1, \\'temp_f\\': 70.0, \\'is_day\\': 1, \\'condition\\': {\\'text\\': \\'Partly cloudy\\', \\'icon\\': \\'//cdn.weatherapi.com/weather/64x64/day/116.png\\', \\'code\\': 1003}, \\'wind_mph\\': 10.5, \\'wind_kph\\': 16.9, \\'wind_degree\\': 300, \\'wind_dir\\': \\'WNW\\', \\'pressure_mb\\': 1017.0, \\'pressure_in\\': 30.04, \\'precip_mm\\': 0.0, \\'precip_in\\': 0.0, \\'humidity\\': 59, \\'cloud\\': 75, \\'feelslike_c\\': 21.1, \\'feelslike_f\\': 70.0, \\'vis_km\\': 16.0, \\'vis_miles\\': 9.0, \\'uv\\': 5.0, \\'gust_mph\\': 16.0, \\'gust_kph\\': 25.8}}\"}]', name='tavily_search_results_json', tool_call_id='call_Q7l0TopyJxaly7xM9Vq2aGxO'),\n",
|
||||
" AIMessage(content='The current weather in San Francisco is partly cloudy with a temperature of 21.1°C (70.0°F). The wind speed is 10.5 mph (16.9 kph) coming from the west-northwest direction. The humidity is at 59% with a visibility of 16.0 km (9.0 miles).', response_metadata={'token_usage': {'completion_tokens': 72, 'prompt_tokens': 466, 'total_tokens': 538}, 'model_name': 'gpt-3.5-turbo', 'system_fingerprint': 'fp_b28b39ffa8', 'finish_reason': 'stop', 'logprobs': None}, id='run-091dcb4c-9424-40fc-aab6-58968d9929a2-0')]}"
|
||||
]
|
||||
},
|
||||
"execution_count": 14,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"from langchain_core.messages import HumanMessage\n",
|
||||
"\n",
|
||||
"inputs = {\"messages\": [HumanMessage(content=\"what is the weather in sf\")]}\n",
|
||||
"app.invoke(inputs)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "5a9e8155-70c5-4973-912c-dc55104b2acf",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"This may take a little bit - it's making a few calls behind the scenes.\n",
|
||||
"In order to start seeing some intermediate results as they happen, we can use streaming - see below for more information on that.\n",
|
||||
"\n",
|
||||
"## Streaming\n",
|
||||
"\n",
|
||||
"LangGraph has support for several different types of streaming.\n",
|
||||
"\n",
|
||||
"### Streaming Node Output\n",
|
||||
"\n",
|
||||
"One of the benefits of using LangGraph is that it is easy to stream output as it's produced by each node.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 15,
|
||||
"id": "f544977e-31f7-41f0-88c4-ec9c27b8cecb",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"/Users/nuno/dev/langgraph/.venv/lib/python3.11/site-packages/langchain_core/messages/ai.py:52: UserWarning: New langchain packages are available that more efficiently handle tool calling. Please upgrade your packages to versions that set message tool calls. e.g., `pip install --upgrade langchain-anthropic`, pip install--upgrade langchain-openai`, etc.\n",
|
||||
" warnings.warn(\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Output from node 'agent':\n",
|
||||
"---\n",
|
||||
"{'messages': [AIMessage(content='', additional_kwargs={'tool_calls': [{'id': 'call_VgzsFE5Cf3sdlrIudeijqVsp', 'function': {'arguments': '{\"query\":\"weather in San Francisco\"}', 'name': 'tavily_search_results_json'}, 'type': 'function'}]}, response_metadata={'token_usage': {'completion_tokens': 21, 'prompt_tokens': 87, 'total_tokens': 108}, 'model_name': 'gpt-3.5-turbo', 'system_fingerprint': 'fp_b28b39ffa8', 'finish_reason': 'tool_calls', 'logprobs': None}, id='run-b3acca13-ccbf-4761-9d0c-1410d5f4f0a9-0', tool_calls=[{'name': 'tavily_search_results_json', 'args': {'query': 'weather in San Francisco'}, 'id': 'call_VgzsFE5Cf3sdlrIudeijqVsp'}])]}\n",
|
||||
"\n",
|
||||
"---\n",
|
||||
"\n",
|
||||
"Output from node 'action':\n",
|
||||
"---\n",
|
||||
"{'messages': [ToolMessage(content='[{\"url\": \"https://www.weatherapi.com/\", \"content\": \"{\\'location\\': {\\'name\\': \\'San Francisco\\', \\'region\\': \\'California\\', \\'country\\': \\'United States of America\\', \\'lat\\': 37.78, \\'lon\\': -122.42, \\'tz_id\\': \\'America/Los_Angeles\\', \\'localtime_epoch\\': 1712797953, \\'localtime\\': \\'2024-04-10 18:12\\'}, \\'current\\': {\\'last_updated_epoch\\': 1712797200, \\'last_updated\\': \\'2024-04-10 18:00\\', \\'temp_c\\': 21.1, \\'temp_f\\': 70.0, \\'is_day\\': 1, \\'condition\\': {\\'text\\': \\'Partly cloudy\\', \\'icon\\': \\'//cdn.weatherapi.com/weather/64x64/day/116.png\\', \\'code\\': 1003}, \\'wind_mph\\': 10.5, \\'wind_kph\\': 16.9, \\'wind_degree\\': 300, \\'wind_dir\\': \\'WNW\\', \\'pressure_mb\\': 1017.0, \\'pressure_in\\': 30.04, \\'precip_mm\\': 0.0, \\'precip_in\\': 0.0, \\'humidity\\': 59, \\'cloud\\': 75, \\'feelslike_c\\': 21.1, \\'feelslike_f\\': 70.0, \\'vis_km\\': 16.0, \\'vis_miles\\': 9.0, \\'uv\\': 5.0, \\'gust_mph\\': 16.0, \\'gust_kph\\': 25.8}}\"}]', name='tavily_search_results_json', tool_call_id='call_VgzsFE5Cf3sdlrIudeijqVsp')]}\n",
|
||||
"\n",
|
||||
"---\n",
|
||||
"\n",
|
||||
"Output from node 'agent':\n",
|
||||
"---\n",
|
||||
"{'messages': [AIMessage(content='The current weather in San Francisco is partly cloudy with a temperature of 70°F (21.1°C). The wind speed is 10.5 mph (16.9 kph) coming from the west-northwest direction. The humidity is at 59%, and the visibility is 9.0 miles.', response_metadata={'token_usage': {'completion_tokens': 65, 'prompt_tokens': 466, 'total_tokens': 531}, 'model_name': 'gpt-3.5-turbo', 'system_fingerprint': 'fp_b28b39ffa8', 'finish_reason': 'stop', 'logprobs': None}, id='run-fa95b6d0-34bb-41e7-8cfd-87b6af69064f-0')]}\n",
|
||||
"\n",
|
||||
"---\n",
|
||||
"\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"inputs = {\"messages\": [HumanMessage(content=\"what is the weather in sf\")]}\n",
|
||||
"for output in app.stream(inputs):\n",
|
||||
" # stream() yields dictionaries with output keyed by node name\n",
|
||||
" for key, value in output.items():\n",
|
||||
" print(f\"Output from node '{key}':\")\n",
|
||||
" print(\"---\")\n",
|
||||
" print(value)\n",
|
||||
" print(\"\\n---\\n\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "2a1b56c5-bd61-4192-8bdb-458a1e9f0159",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Streaming LLM Tokens\n",
|
||||
"\n",
|
||||
"You can also access the LLM tokens as they are produced by each node. \n",
|
||||
"In this case only the \"agent\" node produces LLM tokens.\n",
|
||||
"In order for this to work properly, you must be using an LLM that supports streaming as well as have set it when constructing the LLM (e.g. `ChatOpenAI(model=\"gpt-3.5-turbo-1106\", streaming=True)`)\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 16,
|
||||
"id": "cfd140f0-a5a6-4697-8115-322242f197b5",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"/Users/nuno/dev/langgraph/.venv/lib/python3.11/site-packages/langchain_core/messages/ai.py:52: UserWarning: New langchain packages are available that more efficiently handle tool calling. Please upgrade your packages to versions that set message tool calls. e.g., `pip install --upgrade langchain-anthropic`, pip install--upgrade langchain-openai`, etc.\n",
|
||||
" warnings.warn(\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"content='' additional_kwargs={'tool_calls': [{'index': 0, 'id': 'call_TyyKAi2L0Uymjr3YW2d3ZnIm', 'function': {'arguments': '', 'name': 'tavily_search_results_json'}, 'type': 'function'}]} id='run-7106791e-464e-4aa1-aaee-5d4674fe49e1' invalid_tool_calls=[{'name': 'tavily_search_results_json', 'args': '', 'id': 'call_TyyKAi2L0Uymjr3YW2d3ZnIm', 'error': 'Malformed args.'}] tool_call_chunks=[{'name': 'tavily_search_results_json', 'args': '', 'id': 'call_TyyKAi2L0Uymjr3YW2d3ZnIm', 'index': 0}]\n",
|
||||
"content='' additional_kwargs={'tool_calls': [{'index': 0, 'id': None, 'function': {'arguments': '{\"', 'name': None}, 'type': None}]} id='run-7106791e-464e-4aa1-aaee-5d4674fe49e1' tool_calls=[{'name': '', 'args': {}, 'id': None}] tool_call_chunks=[{'name': None, 'args': '{\"', 'id': None, 'index': 0}]\n",
|
||||
"content='' additional_kwargs={'tool_calls': [{'index': 0, 'id': None, 'function': {'arguments': 'query', 'name': None}, 'type': None}]} id='run-7106791e-464e-4aa1-aaee-5d4674fe49e1' invalid_tool_calls=[{'name': None, 'args': 'query', 'id': None, 'error': 'Malformed args.'}] tool_call_chunks=[{'name': None, 'args': 'query', 'id': None, 'index': 0}]\n",
|
||||
"content='' additional_kwargs={'tool_calls': [{'index': 0, 'id': None, 'function': {'arguments': '\":\"', 'name': None}, 'type': None}]} id='run-7106791e-464e-4aa1-aaee-5d4674fe49e1' invalid_tool_calls=[{'name': None, 'args': '\":\"', 'id': None, 'error': 'Malformed args.'}] tool_call_chunks=[{'name': None, 'args': '\":\"', 'id': None, 'index': 0}]\n",
|
||||
"content='' additional_kwargs={'tool_calls': [{'index': 0, 'id': None, 'function': {'arguments': 'weather', 'name': None}, 'type': None}]} id='run-7106791e-464e-4aa1-aaee-5d4674fe49e1' invalid_tool_calls=[{'name': None, 'args': 'weather', 'id': None, 'error': 'Malformed args.'}] tool_call_chunks=[{'name': None, 'args': 'weather', 'id': None, 'index': 0}]\n",
|
||||
"content='' additional_kwargs={'tool_calls': [{'index': 0, 'id': None, 'function': {'arguments': ' in', 'name': None}, 'type': None}]} id='run-7106791e-464e-4aa1-aaee-5d4674fe49e1' invalid_tool_calls=[{'name': None, 'args': ' in', 'id': None, 'error': 'Malformed args.'}] tool_call_chunks=[{'name': None, 'args': ' in', 'id': None, 'index': 0}]\n",
|
||||
"content='' additional_kwargs={'tool_calls': [{'index': 0, 'id': None, 'function': {'arguments': ' San', 'name': None}, 'type': None}]} id='run-7106791e-464e-4aa1-aaee-5d4674fe49e1' invalid_tool_calls=[{'name': None, 'args': ' San', 'id': None, 'error': 'Malformed args.'}] tool_call_chunks=[{'name': None, 'args': ' San', 'id': None, 'index': 0}]\n",
|
||||
"content='' additional_kwargs={'tool_calls': [{'index': 0, 'id': None, 'function': {'arguments': ' Francisco', 'name': None}, 'type': None}]} id='run-7106791e-464e-4aa1-aaee-5d4674fe49e1' invalid_tool_calls=[{'name': None, 'args': ' Francisco', 'id': None, 'error': 'Malformed args.'}] tool_call_chunks=[{'name': None, 'args': ' Francisco', 'id': None, 'index': 0}]\n",
|
||||
"content='' additional_kwargs={'tool_calls': [{'index': 0, 'id': None, 'function': {'arguments': '\"}', 'name': None}, 'type': None}]} id='run-7106791e-464e-4aa1-aaee-5d4674fe49e1' invalid_tool_calls=[{'name': None, 'args': '\"}', 'id': None, 'error': 'Malformed args.'}] tool_call_chunks=[{'name': None, 'args': '\"}', 'id': None, 'index': 0}]\n",
|
||||
"content='' response_metadata={'finish_reason': 'tool_calls'} id='run-7106791e-464e-4aa1-aaee-5d4674fe49e1'\n",
|
||||
"content='' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content='The' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content=' current' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content=' weather' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content=' in' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content=' San' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content=' Francisco' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content=' is' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content=' as' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content=' follows' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content=':\\n' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content='-' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content=' Temperature' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content=':' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content=' ' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content='21' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content='.' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content='1' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content='°C' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content=' (' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content='70' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content='.' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content='0' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content='°F' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content=')\\n' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content='-' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content=' Condition' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content=':' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content=' Part' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content='ly' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content=' cloudy' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content='\\n' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content='-' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content=' Wind' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content=':' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content=' ' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content='10' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content='.' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content='5' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content=' mph' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content=' from' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content=' W' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content='NW' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content='\\n' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content='-' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content=' Pressure' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content=':' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content=' ' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content='101' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content='7' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content='.' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content='0' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content=' mb' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content='\\n' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content='-' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content=' Hum' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content='idity' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content=':' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content=' ' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content='59' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content='%\\n' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content='-' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content=' Cloud' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content=' Cover' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content=':' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content=' ' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content='75' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content='%\\n' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content='-' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content=' Visibility' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content=':' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content=' ' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content='16' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content='.' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content='0' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content=' km' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content=' (' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content='9' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content='.' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content='0' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content=' miles' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content=')\\n' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content='-' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content=' UV' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content=' Index' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content=':' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content=' ' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content='5' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content='.' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content='0' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content='\\n\\n' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content='For' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content=' more' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content=' details' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content=',' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content=' you' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content=' can' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content=' visit' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content=' [' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content='Weather' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content=' API' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content='](' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content='https' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content='://' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content='www' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content='.weather' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content='api' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content='.com' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content='/' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content=').' id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n",
|
||||
"content='' response_metadata={'finish_reason': 'stop'} id='run-9000bee8-10fe-4712-852b-e043aecb42ec'\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"inputs = {\"messages\": [HumanMessage(content=\"what is the weather in sf?\")]}\n",
|
||||
"\n",
|
||||
"async for output in app.astream_log(inputs, include_types=[\"llm\"]):\n",
|
||||
" # astream_log() yields the requested logs (here LLMs) in JSONPatch format\n",
|
||||
" for op in output.ops:\n",
|
||||
" if op[\"path\"] == \"/streamed_output/-\":\n",
|
||||
" # this is the output from .stream()\n",
|
||||
" ...\n",
|
||||
" elif op[\"path\"].startswith(\"/logs/\") and op[\"path\"].endswith(\n",
|
||||
" \"/streamed_output/-\"\n",
|
||||
" ):\n",
|
||||
" # because we chose to only include LLMs, these are LLM tokens\n",
|
||||
" print(op[\"value\"])"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "08ae8246-11d5-40e1-8567-361e5bef8917",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3 (ipykernel)",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.11.8"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
Reference in New Issue
Block a user