From 7fb7c846e06e608172f590f8128b83e9d7aad49e Mon Sep 17 00:00:00 2001 From: Harrison Chase Date: Mon, 15 Jan 2024 16:10:56 -0800 Subject: [PATCH] cr --- README.md | 31 +++- .../force-calling-a-tool-first.ipynb | 142 ++++++++++++----- .../agent_executor/human-in-the-loop.ipynb | 147 ++++++++++++----- .../agent_executor/managing-agent-steps.ipynb | 149 +++++++++++++----- 4 files changed, 357 insertions(+), 112 deletions(-) diff --git a/README.md b/README.md index 823ffb483..c19cfab94 100644 --- a/README.md +++ b/README.md @@ -419,10 +419,39 @@ Langchain Expression Language allows you to easily define chains (DAGs) but does ## Examples -### ChatExecutor: with function calling +### ChatAgentExecutor: with function calling + +This agent executor takes a list of messages as input and outputs a list of messages. +All agent state is represented as a list of messages. +This specifically uses OpenAI function calling. +This is recommended agent executor for newer chat based models that support function calling. + +- [Getting Started Notebook](examples/chat_agent_executor_with_function_calling/base.ipynb): Walks through creating this type of executor from scratch +- [High Level Entrypoint](examples/chat_agent_executor_with_function_calling/high-level.ipynb): Walks through how to use the high level entrypoint for the chat agent executor. + +**Modifications** + +We also have a lot of examples highlighting how to slightly modify the base chat agent executor. These all build off the [getting started notebook](examples/chat_agent_executor_with_function_calling/base.ipynb) so it is recommended you start with that first. +- [Human-in-the-loop](examples/chat_agent_executor_with_function_calling/human-in-the-loop.ipynb): How to add a human-in-the-loop component +- [Force calling a tool first](examples/chat_agent_executor_with_function_calling/force-calling-a-tool-first.ipynb): How to always call a specific tool first +- [Respond in a specific format](examples/chat_agent_executor_with_function_calling/respond-in-format.ipynb): How to force the agent to respond in a specific format +- [Dynamically returning tool output directly](examples/chat_agent_executor_with_function_calling/dynamically-returning-directly.ipynb): How to dynamically let the agent choose whether to return the result of a tool directly to the user +- [Managing agent steps](examples/chat_agent_executor_with_function_calling/managing-agent-steps.ipynb): How to more explicitly manage intermediate steps that an agent takes ### AgentExecutor +This agent executor uses existing LangChain agents. + +- [Getting Started Notebook](examples/agent_executor/base.ipynb): Walks through creating this type of executor from scratch +- [High Level Entrypoint](examples/agent_executor/high-level.ipynb): Walks through how to use the high level entrypoint for the chat agent executor. + +**Modifications** + +We also have a lot of examples highlighting how to slightly modify the base chat agent executor. These all build off the [getting started notebook](examples/agent_executor/base.ipynb) so it is recommended you start with that first. +- [Human-in-the-loop](examples/agent_executor/human-in-the-loop.ipynb): How to add a human-in-the-loop component +- [Force calling a tool first](examples/agent_executor/force-calling-a-tool-first.ipynb): How to always call a specific tool first +- [Managing agent steps](examples/agent_executor/managing-agent-steps.ipynb): How to more explicitly manage intermediate steps that an agent takes + ## Documentation diff --git a/examples/agent_executor/force-calling-a-tool-first.ipynb b/examples/agent_executor/force-calling-a-tool-first.ipynb index 70712a949..b36121bb5 100644 --- a/examples/agent_executor/force-calling-a-tool-first.ipynb +++ b/examples/agent_executor/force-calling-a-tool-first.ipynb @@ -1,5 +1,29 @@ { "cells": [ + { + "cell_type": "markdown", + "id": "f725852e-71ef-4615-8cac-011a516fbe72", + "metadata": {}, + "source": [ + "# Agent Executor From Scratch\n", + "\n", + "In this notebook we will create an agent with a search tool. However, at the start we will force the agent to call the search tool (and then let it do whatever it wants after). This is useful when you want to force agents to call particular tools, but still want flexibility of what happens after that.\n", + "\n", + "This examples builds off the base agent executor. It is highly recommended you learn about that executor before going through this notebook. You can find documentation for that example [here](./base.ipynb).\n", + "\n", + "Any modifications of that example are called below with **MODIFICATION**, so if you are looking for the differences you can just search for that." + ] + }, + { + "cell_type": "markdown", + "id": "5dace4a9-7c9e-4da2-bf7b-e58d0d05581e", + "metadata": {}, + "source": [ + "## Create the LangChain agent\n", + "\n", + "First, we will create the LangChain agent. For more information on LangChain agents, see [this documentation](https://python.langchain.com/docs/modules/agents/)" + ] + }, { "cell_type": "code", "execution_count": 1, @@ -24,6 +48,21 @@ "agent_runnable = create_openai_functions_agent(llm, tools, prompt)" ] }, + { + "cell_type": "markdown", + "id": "972e58b3-fe3c-449d-b3c4-8fa2217afd07", + "metadata": {}, + "source": [ + "## Define the graph state\n", + "\n", + "We now define the graph state. The state for the traditional LangChain agent has a few attributes:\n", + "\n", + "1. `input`: This is the input string representing the main ask from the user, passed in as input.\n", + "2. `chat_history`: This is any previous conversation messages, also passed in as input.\n", + "3. `intermediate_steps`: This is list of actions and corresponding observations that the agent takes over time. This is updated each iteration of the agent.\n", + "4. `agent_outcome`: This is the response from the agent, either an AgentAction or AgentFinish. The AgentExecutor should finish when this is an AgentFinish, otherwise it should call the requested tools.\n" + ] + }, { "cell_type": "code", "execution_count": 2, @@ -51,6 +90,33 @@ " intermediate_steps: Annotated[list[tuple[AgentAction, str]], operator.add]\n" ] }, + { + "cell_type": "markdown", + "id": "cd27b281-cc9a-49c9-be78-8b98a7d905c4", + "metadata": {}, + "source": [ + "## Define the nodes\n", + "\n", + "We now need to define a few different nodes in our graph.\n", + "In `langgraph`, a node can be either a function or a [runnable](https://python.langchain.com/docs/expression_language/).\n", + "There are two main nodes we need for this:\n", + "\n", + "1. The agent: responsible for deciding what (if any) actions to take.\n", + "2. A function to invoke tools: if the agent decides to take an action, this node will then execute that action.\n", + "\n", + "We will also need to define some edges.\n", + "Some of these edges may be conditional.\n", + "The reason they are conditional is that based on the output of a node, one of several paths may be taken.\n", + "The path that is taken is not known until that node is run (the LLM decides).\n", + "\n", + "1. Conditional Edge: after the agent is called, we should either:\n", + " a. If the agent said to take an action, then the function to invoke tools should be called\n", + " b. If the agent said that it was finished, then it should finish\n", + "2. Normal Edge: after the tools are invoked, it should always go back to the agent to decide what to do next\n", + "\n", + "Let's define the nodes, as well as a function to decide how what conditional edge to take." + ] + }, { "cell_type": "code", "execution_count": 3, @@ -90,10 +156,20 @@ " return \"continue\"" ] }, + { + "cell_type": "markdown", + "id": "02437e83-5485-4827-87e6-7ad1d02cf9be", + "metadata": {}, + "source": [ + "**MODIFICATION**\n", + "\n", + "Here we create a node that returns an AgentAction that just calls the Tavily search with the input" + ] + }, { "cell_type": "code", "execution_count": 4, - "id": "4883e47a-0a15-429c-bf31-1e8afe982a77", + "id": "2ed8463e-73e5-417d-9fab-be6bcee87835", "metadata": {}, "outputs": [ { @@ -114,7 +190,7 @@ { "cell_type": "code", "execution_count": 5, - "id": "fb16db55-ff1a-4e16-94c1-dcb8b2a8f0ba", + "id": "df25d899-2338-4f31-a8bf-0582a2eec325", "metadata": {}, "outputs": [], "source": [ @@ -132,6 +208,20 @@ " return {\"agent_outcome\": action}" ] }, + { + "cell_type": "markdown", + "id": "c0b211f4-0c5c-4792-b18d-cd70907c71e7", + "metadata": {}, + "source": [ + "## Define the graph\n", + "\n", + "We can now put it all together and define the graph!\n", + "\n", + "**MODIFICATION**\n", + "\n", + "We now add a new `first_agent` node which we set as the entrypoint." + ] + }, { "cell_type": "code", "execution_count": 7, @@ -178,17 +268,18 @@ "# This means that after `tools` is called, `agent` node is called next.\n", "workflow.add_edge('action', 'agent')\n", "\n", + "# After the first agent, we want to take an action\n", "workflow.add_edge('first_agent', 'action')\n", "\n", "# Finally, we compile it!\n", "# This compiles it into a LangChain Runnable,\n", "# meaning you can use it as you would any other runnable\n", - "chain = workflow.compile()" + "app = workflow.compile()" ] }, { "cell_type": "code", - "execution_count": 9, + "execution_count": 8, "id": "214ae46e-c297-465d-86db-2b0312ed3530", "metadata": {}, "outputs": [ @@ -196,43 +287,22 @@ "name": "stdout", "output_type": "stream", "text": [ - "Output from node 'first_agent':\n", - "---\n", "{'agent_outcome': AgentActionMessageLog(tool='tavily_search_results_json', tool_input='what is the weather in sf', log='', message_log=[])}\n", - "\n", - "---\n", - "\n", - "Output from node 'action':\n", - "---\n", - "{'intermediate_steps': [(AgentActionMessageLog(tool='tavily_search_results_json', tool_input='what is the weather in sf', log='', message_log=[]), \"[{'url': 'https://weatherspark.com/h/m/557/2024/1/Historical-Weather-in-January-2024-in-San-Francisco-California-United-States', 'content': 'January 2024 Weather History in San Francisco California, United States Daily Precipitation in January 2024 in San Francisco Observed Weather in January 2024 in San Francisco San Francisco Temperature History January 2024 Hourly Temperature in January 2024 in San Francisco Hours of Daylight and Twilight in January 2024 in San FranciscoJanuary 2024 Weather History in San Francisco California, United States. The data for this report comes from the San Francisco International Airport. ... frigid 15°F freezing 32°F very cold 45°F cold 55°F cool 65°F comfortable 75°F warm 85°F hot 95°F sweltering. The hourly reported temperature, color coded into bands. ...'}]\")]}\n", - "\n", - "---\n", - "\n", - "Output from node 'agent':\n", - "---\n", - "{'agent_outcome': AgentFinish(return_values={'output': \"I'm sorry, but I couldn't find the current weather in San Francisco. If you'd like, I can try another source to get the current weather for you.\"}, log=\"I'm sorry, but I couldn't find the current weather in San Francisco. If you'd like, I can try another source to get the current weather for you.\")}\n", - "\n", - "---\n", - "\n", - "Output from node '__end__':\n", - "---\n", - "{'input': 'what is the weather in sf', 'chat_history': [], 'agent_outcome': AgentFinish(return_values={'output': \"I'm sorry, but I couldn't find the current weather in San Francisco. If you'd like, I can try another source to get the current weather for you.\"}, log=\"I'm sorry, but I couldn't find the current weather in San Francisco. If you'd like, I can try another source to get the current weather for you.\"), 'intermediate_steps': [(AgentActionMessageLog(tool='tavily_search_results_json', tool_input='what is the weather in sf', log='', message_log=[]), \"[{'url': 'https://weatherspark.com/h/m/557/2024/1/Historical-Weather-in-January-2024-in-San-Francisco-California-United-States', 'content': 'January 2024 Weather History in San Francisco California, United States Daily Precipitation in January 2024 in San Francisco Observed Weather in January 2024 in San Francisco San Francisco Temperature History January 2024 Hourly Temperature in January 2024 in San Francisco Hours of Daylight and Twilight in January 2024 in San FranciscoJanuary 2024 Weather History in San Francisco California, United States. The data for this report comes from the San Francisco International Airport. ... frigid 15°F freezing 32°F very cold 45°F cold 55°F cool 65°F comfortable 75°F warm 85°F hot 95°F sweltering. The hourly reported temperature, color coded into bands. ...'}]\")]}\n", - "\n", - "---\n", - "\n" + "----\n", + "{'intermediate_steps': [(AgentActionMessageLog(tool='tavily_search_results_json', tool_input='what is the weather in sf', log='', message_log=[]), \"[{'url': 'https://www.whereandwhen.net/when/north-america/california/san-francisco-ca/january/', 'content': 'Best time to go to San Francisco? Weather in San Francisco in january 2024 How was the weather last january? Here is the day by day recorded weather in San Francisco in january 2023: Seasonal average climate and temperature of San Francisco in january The climate of San Francisco in january is tolerableWeather in San Francisco in january 2024. The weather in San Francisco in january comes from statistical datas on the past years. You can view the weather statistics the entire month, but also by using the tabs for the beginning, the middle and the end of the month. ... 15-01-2023 50°F to 52°F. 16-01-2023 45°F to 52°F. 17-01-2023 45°F to ...'}]\")]}\n", + "----\n", + "{'agent_outcome': AgentFinish(return_values={'output': 'The weather in San Francisco in January is typically tolerable, with temperatures ranging from 45°F to 52°F. If you need more specific and up-to-date information about the current weather in San Francisco, I can look it up for you.'}, log='The weather in San Francisco in January is typically tolerable, with temperatures ranging from 45°F to 52°F. If you need more specific and up-to-date information about the current weather in San Francisco, I can look it up for you.')}\n", + "----\n", + "{'input': 'what is the weather in sf', 'chat_history': [], 'agent_outcome': AgentFinish(return_values={'output': 'The weather in San Francisco in January is typically tolerable, with temperatures ranging from 45°F to 52°F. If you need more specific and up-to-date information about the current weather in San Francisco, I can look it up for you.'}, log='The weather in San Francisco in January is typically tolerable, with temperatures ranging from 45°F to 52°F. If you need more specific and up-to-date information about the current weather in San Francisco, I can look it up for you.'), 'intermediate_steps': [(AgentActionMessageLog(tool='tavily_search_results_json', tool_input='what is the weather in sf', log='', message_log=[]), \"[{'url': 'https://www.whereandwhen.net/when/north-america/california/san-francisco-ca/january/', 'content': 'Best time to go to San Francisco? Weather in San Francisco in january 2024 How was the weather last january? Here is the day by day recorded weather in San Francisco in january 2023: Seasonal average climate and temperature of San Francisco in january The climate of San Francisco in january is tolerableWeather in San Francisco in january 2024. The weather in San Francisco in january comes from statistical datas on the past years. You can view the weather statistics the entire month, but also by using the tabs for the beginning, the middle and the end of the month. ... 15-01-2023 50°F to 52°F. 16-01-2023 45°F to 52°F. 17-01-2023 45°F to ...'}]\")]}\n", + "----\n" ] } ], "source": [ - "for output in chain.stream(\n", - " {\"input\": \"what is the weather in sf\", \"chat_history\": []}\n", - "):\n", - " # stream() yields dictionaries with output keyed by node name\n", - " for key, value in output.items():\n", - " print(f\"Output from node '{key}':\")\n", - " print(\"---\")\n", - " print(value)\n", - " print(\"\\n---\\n\")" + "inputs = {\"input\": \"what is the weather in sf\", \"chat_history\": []}\n", + "for s in app.stream(inputs):\n", + " print(list(s.values())[0])\n", + " print(\"----\")" ] }, { diff --git a/examples/agent_executor/human-in-the-loop.ipynb b/examples/agent_executor/human-in-the-loop.ipynb index 3aa623330..c127aee6a 100644 --- a/examples/agent_executor/human-in-the-loop.ipynb +++ b/examples/agent_executor/human-in-the-loop.ipynb @@ -1,5 +1,29 @@ { "cells": [ + { + "cell_type": "markdown", + "id": "f725852e-71ef-4615-8cac-011a516fbe72", + "metadata": {}, + "source": [ + "# Human in the Loop\n", + "\n", + "In this notebook we will go over how to add a human-in-the-loop workflow to the base agent executor. We will use the human to approve\n", + "\n", + "This examples builds off the base agent executor. It is highly recommended you learn about that executor before going through this notebook. You can find documentation for that example [here](./base.ipynb).\n", + "\n", + "Any modifications of that example are called below with **MODIFICATION**, so if you are looking for the differences you can just search for that." + ] + }, + { + "cell_type": "markdown", + "id": "5dace4a9-7c9e-4da2-bf7b-e58d0d05581e", + "metadata": {}, + "source": [ + "## Create the LangChain agent\n", + "\n", + "First, we will create the LangChain agent. For more information on LangChain agents, see [this documentation](https://python.langchain.com/docs/modules/agents/)" + ] + }, { "cell_type": "code", "execution_count": 1, @@ -24,6 +48,21 @@ "agent_runnable = create_openai_functions_agent(llm, tools, prompt)" ] }, + { + "cell_type": "markdown", + "id": "972e58b3-fe3c-449d-b3c4-8fa2217afd07", + "metadata": {}, + "source": [ + "## Define the graph state\n", + "\n", + "We now define the graph state. The state for the traditional LangChain agent has a few attributes:\n", + "\n", + "1. `input`: This is the input string representing the main ask from the user, passed in as input.\n", + "2. `chat_history`: This is any previous conversation messages, also passed in as input.\n", + "3. `intermediate_steps`: This is list of actions and corresponding observations that the agent takes over time. This is updated each iteration of the agent.\n", + "4. `agent_outcome`: This is the response from the agent, either an AgentAction or AgentFinish. The AgentExecutor should finish when this is an AgentFinish, otherwise it should call the requested tools.\n" + ] + }, { "cell_type": "code", "execution_count": 2, @@ -51,10 +90,37 @@ " intermediate_steps: Annotated[list[tuple[AgentAction, str]], operator.add]\n" ] }, + { + "cell_type": "markdown", + "id": "cd27b281-cc9a-49c9-be78-8b98a7d905c4", + "metadata": {}, + "source": [ + "## Define the nodes\n", + "\n", + "We now need to define a few different nodes in our graph.\n", + "In `langgraph`, a node can be either a function or a [runnable](https://python.langchain.com/docs/expression_language/).\n", + "There are two main nodes we need for this:\n", + "\n", + "1. The agent: responsible for deciding what (if any) actions to take.\n", + "2. A function to invoke tools: if the agent decides to take an action, this node will then execute that action.\n", + "\n", + "We will also need to define some edges.\n", + "Some of these edges may be conditional.\n", + "The reason they are conditional is that based on the output of a node, one of several paths may be taken.\n", + "The path that is taken is not known until that node is run (the LLM decides).\n", + "\n", + "1. Conditional Edge: after the agent is called, we should either:\n", + " a. If the agent said to take an action, then the function to invoke tools should be called\n", + " b. If the agent said that it was finished, then it should finish\n", + "2. Normal Edge: after the tools are invoked, it should always go back to the agent to decide what to do next\n", + "\n", + "Let's define the nodes, as well as a function to decide how what conditional edge to take." + ] + }, { "cell_type": "code", - "execution_count": 6, - "id": "d61a970d-edf4-4eef-9678-28bab7c72331", + "execution_count": 3, + "id": "2b757f84-1175-445e-8f8c-e5aeb765a03d", "metadata": {}, "outputs": [], "source": [ @@ -68,8 +134,26 @@ "# Define the agent\n", "def run_agent(data):\n", " agent_outcome = agent_runnable.invoke(data)\n", - " return {\"agent_outcome\": agent_outcome}\n", + " return {\"agent_outcome\": agent_outcome}" + ] + }, + { + "cell_type": "markdown", + "id": "35ace508-d5fe-4139-a0f8-887e38047401", + "metadata": {}, + "source": [ + "**MODIFICATION**\n", "\n", + "We modify the function that is calling the tool to first ask for user approval to continue. Note that this is a simple example and we could modify it to change the tool input, use some other channel besides input, etc." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "2fecf5e0-9604-4992-9c82-b9627466cd32", + "metadata": {}, + "outputs": [], + "source": [ "# Define the function to execute tools\n", "def execute_tools(data):\n", " # Get the most recent agent_outcome - this is the key added in the `agent` above\n", @@ -93,9 +177,19 @@ " return \"continue\"" ] }, + { + "cell_type": "markdown", + "id": "c0b211f4-0c5c-4792-b18d-cd70907c71e7", + "metadata": {}, + "source": [ + "## Define the graph\n", + "\n", + "We can now put it all together and define the graph!" + ] + }, { "cell_type": "code", - "execution_count": 7, + "execution_count": 5, "id": "c4054dde-4618-49b7-998a-daa0c1d6d6c0", "metadata": {}, "outputs": [], @@ -141,12 +235,12 @@ "# Finally, we compile it!\n", "# This compiles it into a LangChain Runnable,\n", "# meaning you can use it as you would any other runnable\n", - "chain = workflow.compile()" + "app = workflow.compile()" ] }, { "cell_type": "code", - "execution_count": 8, + "execution_count": 6, "id": "214ae46e-c297-465d-86db-2b0312ed3530", "metadata": {}, "outputs": [ @@ -154,12 +248,8 @@ "name": "stdout", "output_type": "stream", "text": [ - "Output from node 'agent':\n", - "---\n", "{'agent_outcome': AgentActionMessageLog(tool='tavily_search_results_json', tool_input={'query': 'weather in San Francisco'}, log=\"\\nInvoking: `tavily_search_results_json` with `{'query': 'weather in San Francisco'}`\\n\\n\\n\", message_log=[AIMessage(content='', additional_kwargs={'function_call': {'arguments': '{\"query\":\"weather in San Francisco\"}', 'name': 'tavily_search_results_json'}})])}\n", - "\n", - "---\n", - "\n" + "----\n" ] }, { @@ -173,37 +263,20 @@ "name": "stdout", "output_type": "stream", "text": [ - "Output from node 'action':\n", - "---\n", "{'intermediate_steps': [(AgentActionMessageLog(tool='tavily_search_results_json', tool_input={'query': 'weather in San Francisco'}, log=\"\\nInvoking: `tavily_search_results_json` with `{'query': 'weather in San Francisco'}`\\n\\n\\n\", message_log=[AIMessage(content='', additional_kwargs={'function_call': {'arguments': '{\"query\":\"weather in San Francisco\"}', 'name': 'tavily_search_results_json'}})]), \"[{'url': 'https://weatherspark.com/h/m/557/2024/1/Historical-Weather-in-January-2024-in-San-Francisco-California-United-States', 'content': 'January 2024 Weather History in San Francisco California, United States Daily Precipitation in January 2024 in San Francisco Observed Weather in January 2024 in San Francisco San Francisco Temperature History January 2024 Hourly Temperature in January 2024 in San Francisco Hours of Daylight and Twilight in January 2024 in San FranciscoThis report shows the past weather for San Francisco, providing a weather history for January 2024. It features all historical weather data series we have available, including the San Francisco temperature history for January 2024. You can drill down from year to month and even day level reports by clicking on the graphs.'}]\")]}\n", - "\n", - "---\n", - "\n", - "Output from node 'agent':\n", - "---\n", - "{'agent_outcome': AgentFinish(return_values={'output': \"It seems that I couldn't retrieve the current weather for San Francisco. However, you can easily check the current weather in San Francisco by using a weather website or app.\"}, log=\"It seems that I couldn't retrieve the current weather for San Francisco. However, you can easily check the current weather in San Francisco by using a weather website or app.\")}\n", - "\n", - "---\n", - "\n", - "Output from node '__end__':\n", - "---\n", - "{'input': 'what is the weather in sf', 'chat_history': [], 'agent_outcome': AgentFinish(return_values={'output': \"It seems that I couldn't retrieve the current weather for San Francisco. However, you can easily check the current weather in San Francisco by using a weather website or app.\"}, log=\"It seems that I couldn't retrieve the current weather for San Francisco. However, you can easily check the current weather in San Francisco by using a weather website or app.\"), 'intermediate_steps': [(AgentActionMessageLog(tool='tavily_search_results_json', tool_input={'query': 'weather in San Francisco'}, log=\"\\nInvoking: `tavily_search_results_json` with `{'query': 'weather in San Francisco'}`\\n\\n\\n\", message_log=[AIMessage(content='', additional_kwargs={'function_call': {'arguments': '{\"query\":\"weather in San Francisco\"}', 'name': 'tavily_search_results_json'}})]), \"[{'url': 'https://weatherspark.com/h/m/557/2024/1/Historical-Weather-in-January-2024-in-San-Francisco-California-United-States', 'content': 'January 2024 Weather History in San Francisco California, United States Daily Precipitation in January 2024 in San Francisco Observed Weather in January 2024 in San Francisco San Francisco Temperature History January 2024 Hourly Temperature in January 2024 in San Francisco Hours of Daylight and Twilight in January 2024 in San FranciscoThis report shows the past weather for San Francisco, providing a weather history for January 2024. It features all historical weather data series we have available, including the San Francisco temperature history for January 2024. You can drill down from year to month and even day level reports by clicking on the graphs.'}]\")]}\n", - "\n", - "---\n", - "\n" + "----\n", + "{'agent_outcome': AgentFinish(return_values={'output': \"It seems that I didn't find the current weather information for San Francisco. I recommend checking a reliable weather website or using a weather app to get the most up-to-date information.\"}, log=\"It seems that I didn't find the current weather information for San Francisco. I recommend checking a reliable weather website or using a weather app to get the most up-to-date information.\")}\n", + "----\n", + "{'input': 'what is the weather in sf', 'chat_history': [], 'agent_outcome': AgentFinish(return_values={'output': \"It seems that I didn't find the current weather information for San Francisco. I recommend checking a reliable weather website or using a weather app to get the most up-to-date information.\"}, log=\"It seems that I didn't find the current weather information for San Francisco. I recommend checking a reliable weather website or using a weather app to get the most up-to-date information.\"), 'intermediate_steps': [(AgentActionMessageLog(tool='tavily_search_results_json', tool_input={'query': 'weather in San Francisco'}, log=\"\\nInvoking: `tavily_search_results_json` with `{'query': 'weather in San Francisco'}`\\n\\n\\n\", message_log=[AIMessage(content='', additional_kwargs={'function_call': {'arguments': '{\"query\":\"weather in San Francisco\"}', 'name': 'tavily_search_results_json'}})]), \"[{'url': 'https://weatherspark.com/h/m/557/2024/1/Historical-Weather-in-January-2024-in-San-Francisco-California-United-States', 'content': 'January 2024 Weather History in San Francisco California, United States Daily Precipitation in January 2024 in San Francisco Observed Weather in January 2024 in San Francisco San Francisco Temperature History January 2024 Hourly Temperature in January 2024 in San Francisco Hours of Daylight and Twilight in January 2024 in San FranciscoThis report shows the past weather for San Francisco, providing a weather history for January 2024. It features all historical weather data series we have available, including the San Francisco temperature history for January 2024. You can drill down from year to month and even day level reports by clicking on the graphs.'}]\")]}\n", + "----\n" ] } ], "source": [ - "for output in chain.stream(\n", - " {\"input\": \"what is the weather in sf\", \"chat_history\": []}\n", - "):\n", - " # stream() yields dictionaries with output keyed by node name\n", - " for key, value in output.items():\n", - " print(f\"Output from node '{key}':\")\n", - " print(\"---\")\n", - " print(value)\n", - " print(\"\\n---\\n\")" + "inputs = {\"input\": \"what is the weather in sf\", \"chat_history\": []}\n", + "for s in app.stream(inputs):\n", + " print(list(s.values())[0])\n", + " print(\"----\")" ] }, { diff --git a/examples/agent_executor/managing-agent-steps.ipynb b/examples/agent_executor/managing-agent-steps.ipynb index 7d7ffa498..b53b7e91a 100644 --- a/examples/agent_executor/managing-agent-steps.ipynb +++ b/examples/agent_executor/managing-agent-steps.ipynb @@ -1,5 +1,29 @@ { "cells": [ + { + "cell_type": "markdown", + "id": "f725852e-71ef-4615-8cac-011a516fbe72", + "metadata": {}, + "source": [ + "# Managing Agent Steps\n", + "\n", + "In this notebook we will go over how to build a basic agent executor where we custom handle how to manage the intermediate steps. Normally, all previous steps are passed to the agent at future iterations, but in long-running cases that could lead to an overly large amount of steps that you may want to trim\n", + "\n", + "This examples builds off the base agent executor. It is highly recommended you learn about that executor before going through this notebook. You can find documentation for that example [here](./base.ipynb).\n", + "\n", + "Any modifications of that example are called below with **MODIFICATION**, so if you are looking for the differences you can just search for that." + ] + }, + { + "cell_type": "markdown", + "id": "5dace4a9-7c9e-4da2-bf7b-e58d0d05581e", + "metadata": {}, + "source": [ + "## Create the LangChain agent\n", + "\n", + "First, we will create the LangChain agent. For more information on LangChain agents, see [this documentation](https://python.langchain.com/docs/modules/agents/)" + ] + }, { "cell_type": "code", "execution_count": 1, @@ -24,6 +48,21 @@ "agent_runnable = create_openai_functions_agent(llm, tools, prompt)" ] }, + { + "cell_type": "markdown", + "id": "972e58b3-fe3c-449d-b3c4-8fa2217afd07", + "metadata": {}, + "source": [ + "## Define the graph state\n", + "\n", + "We now define the graph state. The state for the traditional LangChain agent has a few attributes:\n", + "\n", + "1. `input`: This is the input string representing the main ask from the user, passed in as input.\n", + "2. `chat_history`: This is any previous conversation messages, also passed in as input.\n", + "3. `intermediate_steps`: This is list of actions and corresponding observations that the agent takes over time. This is updated each iteration of the agent.\n", + "4. `agent_outcome`: This is the response from the agent, either an AgentAction or AgentFinish. The AgentExecutor should finish when this is an AgentFinish, otherwise it should call the requested tools.\n" + ] + }, { "cell_type": "code", "execution_count": 2, @@ -51,10 +90,37 @@ " intermediate_steps: Annotated[list[tuple[AgentAction, str]], operator.add]\n" ] }, + { + "cell_type": "markdown", + "id": "cd27b281-cc9a-49c9-be78-8b98a7d905c4", + "metadata": {}, + "source": [ + "## Define the nodes\n", + "\n", + "We now need to define a few different nodes in our graph.\n", + "In `langgraph`, a node can be either a function or a [runnable](https://python.langchain.com/docs/expression_language/).\n", + "There are two main nodes we need for this:\n", + "\n", + "1. The agent: responsible for deciding what (if any) actions to take.\n", + "2. A function to invoke tools: if the agent decides to take an action, this node will then execute that action.\n", + "\n", + "We will also need to define some edges.\n", + "Some of these edges may be conditional.\n", + "The reason they are conditional is that based on the output of a node, one of several paths may be taken.\n", + "The path that is taken is not known until that node is run (the LLM decides).\n", + "\n", + "1. Conditional Edge: after the agent is called, we should either:\n", + " a. If the agent said to take an action, then the function to invoke tools should be called\n", + " b. If the agent said that it was finished, then it should finish\n", + "2. Normal Edge: after the tools are invoked, it should always go back to the agent to decide what to do next\n", + "\n", + "Let's define the nodes, as well as a function to decide how what conditional edge to take." + ] + }, { "cell_type": "code", - "execution_count": 3, - "id": "d61a970d-edf4-4eef-9678-28bab7c72331", + "execution_count": 6, + "id": "77e3c059-e31f-4c8f-81bf-edb58688e12b", "metadata": {}, "outputs": [], "source": [ @@ -63,8 +129,26 @@ "\n", "# This a helper class we have that is useful for running tools\n", "# It takes in an agent action and calls that tool and returns the result\n", - "tool_executor = ToolExecutor(tools)\n", + "tool_executor = ToolExecutor(tools)" + ] + }, + { + "cell_type": "markdown", + "id": "4c804a34-d384-4ca9-b9fc-dc86d678ab39", + "metadata": {}, + "source": [ + "**MODIFICATION**\n", "\n", + "Here, we modify the agent to only look at the last five intermediate steps. This is a relatively simple example of shortening the intermediate step history." + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "a9f66a3e-aba1-4893-95b1-a433c7091d5e", + "metadata": {}, + "outputs": [], + "source": [ "# Define the agent\n", "def run_agent(data):\n", " inputs = data.copy()\n", @@ -93,9 +177,19 @@ " return \"continue\"" ] }, + { + "cell_type": "markdown", + "id": "c0b211f4-0c5c-4792-b18d-cd70907c71e7", + "metadata": {}, + "source": [ + "## Define the graph\n", + "\n", + "We can now put it all together and define the graph!" + ] + }, { "cell_type": "code", - "execution_count": 4, + "execution_count": 8, "id": "c4054dde-4618-49b7-998a-daa0c1d6d6c0", "metadata": {}, "outputs": [], @@ -141,12 +235,12 @@ "# Finally, we compile it!\n", "# This compiles it into a LangChain Runnable,\n", "# meaning you can use it as you would any other runnable\n", - "chain = workflow.compile()" + "app = workflow.compile()" ] }, { "cell_type": "code", - "execution_count": 5, + "execution_count": 9, "id": "214ae46e-c297-465d-86db-2b0312ed3530", "metadata": {}, "outputs": [ @@ -154,43 +248,22 @@ "name": "stdout", "output_type": "stream", "text": [ - "Output from node 'agent':\n", - "---\n", "{'agent_outcome': AgentActionMessageLog(tool='tavily_search_results_json', tool_input={'query': 'weather in San Francisco'}, log=\"\\nInvoking: `tavily_search_results_json` with `{'query': 'weather in San Francisco'}`\\n\\n\\n\", message_log=[AIMessage(content='', additional_kwargs={'function_call': {'arguments': '{\"query\":\"weather in San Francisco\"}', 'name': 'tavily_search_results_json'}})])}\n", - "\n", - "---\n", - "\n", - "Output from node 'action':\n", - "---\n", - "{'intermediate_steps': [(AgentActionMessageLog(tool='tavily_search_results_json', tool_input={'query': 'weather in San Francisco'}, log=\"\\nInvoking: `tavily_search_results_json` with `{'query': 'weather in San Francisco'}`\\n\\n\\n\", message_log=[AIMessage(content='', additional_kwargs={'function_call': {'arguments': '{\"query\":\"weather in San Francisco\"}', 'name': 'tavily_search_results_json'}})]), \"[{'url': 'https://weatherspark.com/h/m/557/2024/1/Historical-Weather-in-January-2024-in-San-Francisco-California-United-States', 'content': 'January 2024 Weather History in San Francisco California, United States Daily Precipitation in January 2024 in San Francisco Observed Weather in January 2024 in San Francisco San Francisco Temperature History January 2024 Hourly Temperature in January 2024 in San Francisco Hours of Daylight and Twilight in January 2024 in San FranciscoThis report shows the past weather for San Francisco, providing a weather history for January 2024. It features all historical weather data series we have available, including the San Francisco temperature history for January 2024. You can drill down from year to month and even day level reports by clicking on the graphs.'}]\")]}\n", - "\n", - "---\n", - "\n", - "Output from node 'agent':\n", - "---\n", - "{'agent_outcome': AgentFinish(return_values={'output': \"I'm sorry, I couldn't find the current weather for San Francisco. If you'd like, I can search for the current weather using a different method.\"}, log=\"I'm sorry, I couldn't find the current weather for San Francisco. If you'd like, I can search for the current weather using a different method.\")}\n", - "\n", - "---\n", - "\n", - "Output from node '__end__':\n", - "---\n", - "{'input': 'what is the weather in sf', 'chat_history': [], 'agent_outcome': AgentFinish(return_values={'output': \"I'm sorry, I couldn't find the current weather for San Francisco. If you'd like, I can search for the current weather using a different method.\"}, log=\"I'm sorry, I couldn't find the current weather for San Francisco. If you'd like, I can search for the current weather using a different method.\"), 'intermediate_steps': [(AgentActionMessageLog(tool='tavily_search_results_json', tool_input={'query': 'weather in San Francisco'}, log=\"\\nInvoking: `tavily_search_results_json` with `{'query': 'weather in San Francisco'}`\\n\\n\\n\", message_log=[AIMessage(content='', additional_kwargs={'function_call': {'arguments': '{\"query\":\"weather in San Francisco\"}', 'name': 'tavily_search_results_json'}})]), \"[{'url': 'https://weatherspark.com/h/m/557/2024/1/Historical-Weather-in-January-2024-in-San-Francisco-California-United-States', 'content': 'January 2024 Weather History in San Francisco California, United States Daily Precipitation in January 2024 in San Francisco Observed Weather in January 2024 in San Francisco San Francisco Temperature History January 2024 Hourly Temperature in January 2024 in San Francisco Hours of Daylight and Twilight in January 2024 in San FranciscoThis report shows the past weather for San Francisco, providing a weather history for January 2024. It features all historical weather data series we have available, including the San Francisco temperature history for January 2024. You can drill down from year to month and even day level reports by clicking on the graphs.'}]\")]}\n", - "\n", - "---\n", - "\n" + "----\n", + "{'intermediate_steps': [(AgentActionMessageLog(tool='tavily_search_results_json', tool_input={'query': 'weather in San Francisco'}, log=\"\\nInvoking: `tavily_search_results_json` with `{'query': 'weather in San Francisco'}`\\n\\n\\n\", message_log=[AIMessage(content='', additional_kwargs={'function_call': {'arguments': '{\"query\":\"weather in San Francisco\"}', 'name': 'tavily_search_results_json'}})]), \"[{'url': 'https://en.climate-data.org/north-america/united-states-of-america/california/san-francisco-385/t/january-1/', 'content': 'San Francisco Weather in January San Francisco weather in January San Francisco weather by month // weather averages 9.6 (49.2) 6.2 (43.2) 14 (57.3) 113 San Francisco weather in January // weather averages Airport close to San Francisco you can find all information about the weather in San Francisco in January:Data: 1991 - 2021 Min. Temperature °C (°F), Max. Temperature °C (°F), Precipitation / Rainfall mm (in), Humidity, Rainy days. Data: 1999 - 2019: avg. Sun hours San Francisco weather and climate for further months San Francisco in February San Francisco in March San Francisco in April San Francisco in May San Francisco in June San Francisco in July'}]\")]}\n", + "----\n", + "{'agent_outcome': AgentFinish(return_values={'output': \"The weather in San Francisco varies by month. In January, the average minimum temperature is 9.6°C (49.2°F), and the average maximum temperature is 14°C (57.3°F). The city experiences an average of 113mm of precipitation and has an average of 6 rainy days in January. If you'd like to know more about the weather in other months, feel free to ask!\"}, log=\"The weather in San Francisco varies by month. In January, the average minimum temperature is 9.6°C (49.2°F), and the average maximum temperature is 14°C (57.3°F). The city experiences an average of 113mm of precipitation and has an average of 6 rainy days in January. If you'd like to know more about the weather in other months, feel free to ask!\")}\n", + "----\n", + "{'input': 'what is the weather in sf', 'chat_history': [], 'agent_outcome': AgentFinish(return_values={'output': \"The weather in San Francisco varies by month. In January, the average minimum temperature is 9.6°C (49.2°F), and the average maximum temperature is 14°C (57.3°F). The city experiences an average of 113mm of precipitation and has an average of 6 rainy days in January. If you'd like to know more about the weather in other months, feel free to ask!\"}, log=\"The weather in San Francisco varies by month. In January, the average minimum temperature is 9.6°C (49.2°F), and the average maximum temperature is 14°C (57.3°F). The city experiences an average of 113mm of precipitation and has an average of 6 rainy days in January. If you'd like to know more about the weather in other months, feel free to ask!\"), 'intermediate_steps': [(AgentActionMessageLog(tool='tavily_search_results_json', tool_input={'query': 'weather in San Francisco'}, log=\"\\nInvoking: `tavily_search_results_json` with `{'query': 'weather in San Francisco'}`\\n\\n\\n\", message_log=[AIMessage(content='', additional_kwargs={'function_call': {'arguments': '{\"query\":\"weather in San Francisco\"}', 'name': 'tavily_search_results_json'}})]), \"[{'url': 'https://en.climate-data.org/north-america/united-states-of-america/california/san-francisco-385/t/january-1/', 'content': 'San Francisco Weather in January San Francisco weather in January San Francisco weather by month // weather averages 9.6 (49.2) 6.2 (43.2) 14 (57.3) 113 San Francisco weather in January // weather averages Airport close to San Francisco you can find all information about the weather in San Francisco in January:Data: 1991 - 2021 Min. Temperature °C (°F), Max. Temperature °C (°F), Precipitation / Rainfall mm (in), Humidity, Rainy days. Data: 1999 - 2019: avg. Sun hours San Francisco weather and climate for further months San Francisco in February San Francisco in March San Francisco in April San Francisco in May San Francisco in June San Francisco in July'}]\")]}\n", + "----\n" ] } ], "source": [ - "for output in chain.stream(\n", - " {\"input\": \"what is the weather in sf\", \"chat_history\": []}\n", - "):\n", - " # stream() yields dictionaries with output keyed by node name\n", - " for key, value in output.items():\n", - " print(f\"Output from node '{key}':\")\n", - " print(\"---\")\n", - " print(value)\n", - " print(\"\\n---\\n\")" + "inputs = {\"input\": \"what is the weather in sf\", \"chat_history\": []}\n", + "for s in app.stream(inputs):\n", + " print(list(s.values())[0])\n", + " print(\"----\")" ] }, {