From 7ea5da73c7149c0ad29e58dad192612a16ce776f Mon Sep 17 00:00:00 2001 From: Chester Curme Date: Wed, 24 Jul 2024 11:49:05 -0400 Subject: [PATCH 1/6] add guide --- examples/agent_executor/many_tools.ipynb | 314 +++++++++++++++++++++++ 1 file changed, 314 insertions(+) create mode 100644 examples/agent_executor/many_tools.ipynb diff --git a/examples/agent_executor/many_tools.ipynb b/examples/agent_executor/many_tools.ipynb new file mode 100644 index 000000000..b6e636dc1 --- /dev/null +++ b/examples/agent_executor/many_tools.ipynb @@ -0,0 +1,314 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "5fa317ef-b9a7-4432-ba85-ce71b8dfbdc6", + "metadata": {}, + "source": [ + "# How to handle large numbers of tools\n", + "\n", + "The subset of available tools to call is generally at the discretion of the model (although many providers also enable the user to [specify or constrain the choice of tool](https://python.langchain.com/v0.2/docs/how_to/tool_choice/)). As the number of available tools grows, you may want to limit the scope of the LLM's selection, to decrease token consumption and to help manage sources of error in LLM reasoning.\n", + "\n", + "Here we will demonstrate how to dynamically adjust the tools available to a model. Bottom line up front: like [RAG](https://python.langchain.com/v0.2/docs/concepts/#retrieval) and similar methods, we prefix the model invocation by retrieving over available tools. Although we demonstrate one implementation that searches over tool descriptions, the details of the tool selection can be customized as needed." + ] + }, + { + "cell_type": "markdown", + "id": "1a417013-ddc4-463b-8ea0-0904bd232827", + "metadata": {}, + "source": [ + "## Define the tools" + ] + }, + { + "cell_type": "markdown", + "id": "24708f3b-18b1-4b42-9f6a-0d4827222918", + "metadata": {}, + "source": [ + "Let's consider a toy example in which we have one tool for each company in the S&P 500 index. Each tool will fetch information, and is parameterized by a single integer representing the year.\n", + "\n", + "We first construct a registry that associates a unique identifier with a schema for each tool. We will represent the tools using JSON schema, which can be bound directly to chat models supporting tool calling." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "da30c3f1-127f-4828-8609-94e16719f0be", + "metadata": {}, + "outputs": [], + "source": [ + "import re\n", + "import uuid\n", + "\n", + "from langchain_core.tools import StructuredTool\n", + "\n", + "\n", + "def create_tool(company: str) -> dict:\n", + " \"\"\"Create schema for a placeholder tool.\"\"\"\n", + " formatted_company = re.sub(r\"[^\\w\\s]\", \"\", company).replace(\" \", \"_\")\n", + "\n", + " def company_tool(year: int) -> str:\n", + " return f\"{company} had revenues of $100 in {year}.\"\n", + "\n", + " return StructuredTool.from_function(\n", + " company_tool,\n", + " name=formatted_company,\n", + " description=f\"Information about {company}\",\n", + " )\n", + "\n", + "\n", + "s_and_p_500_companies = [ # Abbreviated list for demonstration purposes\n", + " \"3M\",\n", + " \"A.O. Smith\",\n", + " \"Abbott\",\n", + " \"Accenture\",\n", + " \"Advanced Micro Devices\",\n", + " \"Yum! Brands\",\n", + " \"Zebra Technologies\",\n", + " \"Zimmer Biomet\",\n", + " \"Zoetis\",\n", + "]\n", + "\n", + "tool_registry = {\n", + " str(uuid.uuid4()): create_tool(company) for company in s_and_p_500_companies\n", + "}" + ] + }, + { + "cell_type": "markdown", + "id": "ba17b047-73ed-4385-adc2-f02012db2206", + "metadata": {}, + "source": [ + "## Define the graph" + ] + }, + { + "cell_type": "markdown", + "id": "2055548d-3d14-4aaf-9588-abf70f28b5d6", + "metadata": {}, + "source": [ + "### Tool selection" + ] + }, + { + "cell_type": "markdown", + "id": "8798a0d2-ea93-45bc-ab55-071ab975f2c2", + "metadata": {}, + "source": [ + "We will construct a node that retrieves a subset of available tools given the information in the state-- such as a recent user message. In general, the full scope of [retrieval solutions](https://python.langchain.com/v0.2/docs/concepts/#retrieval) are available for this step. As a simple solution, we index embeddings of tool descriptions in a vector store, and associate user queries to tools via semantic search." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "435b0201-7296-4617-abf8-2c757a71f6b5", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_core.documents import Document\n", + "from langchain_core.vectorstores import InMemoryVectorStore, VectorStore\n", + "from langchain_openai import OpenAIEmbeddings\n", + "\n", + "tool_documents = [\n", + " Document(page_content=tool.description, id=id)\n", + " for id, tool in tool_registry.items()\n", + "]\n", + "\n", + "vector_store = InMemoryVectorStore(embedding=OpenAIEmbeddings())\n", + "upsert_response = vector_store.upsert(tool_documents)\n", + "assert not upsert_response[\"failed\"]" + ] + }, + { + "cell_type": "markdown", + "id": "e9ce366b-b5e7-41e9-b4a9-d775b9be0d09", + "metadata": {}, + "source": [ + "### Incorporating with an agent\n", + "\n", + "We will use a typical React agent graph (e.g., as used in the [quickstart](https://langchain-ai.github.io/langgraph/tutorials/introduction/#part-2-enhancing-the-chatbot-with-tools)), with some modifications:\n", + "\n", + "- We add a `selected_tools` key to the state, which stores our selected subset of tools;\n", + "- We set the entry point of the graph to be a `select_tools` node, which populates this element of the state;\n", + "- We bind the selected subset of tools to the chat model within the `agent` node." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "d319fea9-e8ae-4763-a785-b2bf72239ae4", + "metadata": {}, + "outputs": [], + "source": [ + "from typing import Annotated\n", + "\n", + "from langchain_openai import ChatOpenAI\n", + "from typing_extensions import TypedDict\n", + "\n", + "from langgraph.graph import StateGraph, START, END\n", + "from langgraph.graph.message import add_messages\n", + "from langgraph.prebuilt import ToolNode, tools_condition\n", + "\n", + "\n", + "class State(TypedDict):\n", + " messages: Annotated[list, add_messages]\n", + " selected_tools: list[str]\n", + "\n", + "\n", + "graph_builder = StateGraph(State)\n", + "\n", + "tools = list(tool_registry.values())\n", + "llm = ChatOpenAI()\n", + "\n", + "\n", + "def agent(state: State):\n", + " selected_tools = [tool_registry[id] for id in state[\"selected_tools\"]]\n", + " llm_with_tools = llm.bind_tools(selected_tools)\n", + " return {\"messages\": [llm_with_tools.invoke(state[\"messages\"])]}\n", + "\n", + "\n", + "def select_tools(state: State):\n", + " last_user_message = state[\"messages\"][-1]\n", + " query = last_user_message.content\n", + " tool_documents = vector_store.similarity_search(query)\n", + " return {\"selected_tools\": [document.id for document in tool_documents]}\n", + "\n", + "\n", + "graph_builder.add_node(\"agent\", agent)\n", + "graph_builder.add_node(\"select_tools\", select_tools)\n", + "\n", + "tool_node = ToolNode(tools=tools)\n", + "graph_builder.add_node(\"tools\", tool_node)\n", + "\n", + "graph_builder.add_conditional_edges(\n", + " \"agent\",\n", + " tools_condition,\n", + ")\n", + "graph_builder.add_edge(\"tools\", \"agent\")\n", + "graph_builder.add_edge(\"select_tools\", \"agent\")\n", + "graph_builder.add_edge(START, \"select_tools\")\n", + "graph = graph_builder.compile()" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "35cab3b2-4d03-4cb5-ba10-f7d3a5ad5244", + "metadata": {}, + "outputs": [ + { + "data": { + "image/jpeg": "/9j/4AAQSkZJRgABAQAAAQABAAD/4gHYSUNDX1BST0ZJTEUAAQEAAAHIAAAAAAQwAABtbnRyUkdCIFhZWiAH4AABAAEAAAAAAABhY3NwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAQAA9tYAAQAAAADTLQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAlkZXNjAAAA8AAAACRyWFlaAAABFAAAABRnWFlaAAABKAAAABRiWFlaAAABPAAAABR3dHB0AAABUAAAABRyVFJDAAABZAAAAChnVFJDAAABZAAAAChiVFJDAAABZAAAAChjcHJ0AAABjAAAADxtbHVjAAAAAAAAAAEAAAAMZW5VUwAAAAgAAAAcAHMAUgBHAEJYWVogAAAAAAAAb6IAADj1AAADkFhZWiAAAAAAAABimQAAt4UAABjaWFlaIAAAAAAAACSgAAAPhAAAts9YWVogAAAAAAAA9tYAAQAAAADTLXBhcmEAAAAAAAQAAAACZmYAAPKnAAANWQAAE9AAAApbAAAAAAAAAABtbHVjAAAAAAAAAAEAAAAMZW5VUwAAACAAAAAcAEcAbwBvAGcAbABlACAASQBuAGMALgAgADIAMAAxADb/2wBDAAMCAgMCAgMDAwMEAwMEBQgFBQQEBQoHBwYIDAoMDAsKCwsNDhIQDQ4RDgsLEBYQERMUFRUVDA8XGBYUGBIUFRT/2wBDAQMEBAUEBQkFBQkUDQsNFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBT/wAARCAEvAMcDASIAAhEBAxEB/8QAHQABAAICAwEBAAAAAAAAAAAAAAYHBQgBAwQJAv/EAFoQAAEDBAADAgYKCgwLCQAAAAEAAgMEBQYRBxIhEzEIFBYiQVEVI1VWYXSUlbTTFzI4VHGBk9HS1AkkJTM1NjdCdZGhskNSU2Jyg5KWsbPDGERGV2N2oqTh/8QAGwEBAQADAQEBAAAAAAAAAAAAAAECAwQFBgf/xAAyEQEAAQMABQoGAwEBAAAAAAAAAQIDERIhMVGhBBMUQVJhcZGx0QUVIzNTwYHh8EKi/9oADAMBAAIRAxEAPwD6poiICIiAi8l0udPZrfPW1Ti2CFvMeVpc5x7g1rR1c4nQDR1JIA6lYPyeq8m9vv0ksNK7Zjs9PKWMY30ds5p3I/1gHkG9AO1zu200RMaVU4j/AGxcMzU3220UhZUXClgeOhbLO1pH4iV0+VVl92KD5Sz866qfC8fpIwyCxW2JgAGmUkY7ug9C7fJWy+49B8mZ+ZZ/R7+BqPKqy+7FB8pZ+dPKqy+7FB8pZ+dPJWy+49B8mZ+ZPJWy+49B8mZ+ZPo9/BdR5VWX3YoPlLPzp5VWX3YoPlLPzp5K2X3HoPkzPzJ5K2X3HoPkzPzJ9Hv4Go8qrL7sUHyln51yzJrPI4NZdqFzj6G1LCf+K48lbL7j0HyZn5lw/E7HIwtdZre5p6EGlYQf7E+j38DUyjXB7Q5pDmkbBB2CFyow/AqGhe6ewOdjtWTzftIap3n/ANSD7RwPpIAd36cCdrI2O8yV756Osg8UudLoTQg7Y8HukjPpY7R0e8EEHqFjVRGNKicx5SmNzLIiLSgiIgIiICIiAiIgIiICIiCL3bV2zi025+nU1DA+5SMP86XmEcP4QNyu6+kMPeNiUKMSjxPiTBI/YZX2t0THa6c0MvNrfrImJH+ifUpOui7soiNmPfPFZERFzogFFx4we5ZRcsdo7xJWXa3OnZURU1BUyMEkLS6WNsrYyx8jQDtjXF2xrW+ijPCnwnsb4h8M6zMLhFV2CCgL3VrJ6Cq7OJnbyRRckjoWiZxDBsR8xaXaIB6KI4cLxjnhAGhwuyZbbMVuVzuFRk1DfLcWWqOTlc5tZRVDvTLKGns2OcCHklrCFHMXuedYd4O9wwiz47k9qyyxV8rausprW53aUUlzc6WWgkcDHPL4vIXNaNnYPTYCC8rV4QWA3nEMgyelv27Rj7S+6ulo6iKejby823wPjEo2Oo8zro63pRTO/CxxTGLTY7ja46++UdxvdNanVMNrreyEch2+aJwgIn03q0Rk85Pmk60qNu2G3irsvH1tmxvO6mjyHEKRlrmyOCqqKuvmhNQ2Ro7Tmka7creWJwa7Wy1vL1V7cfrDcW8PcHrbTZay6NxrIbTdam3W2AyVPi0DwJBFEOr3NB3yjr0KC37Pdqe+2mjuVJ23itXC2eLxiB8EnK4bHNHIGvYdHq1wBHcQF7FjccvkeS2SkucVJW0EdS3nbT3KmfTVDBsjT43gOaem9EekLJICjGXatdzsN5Zpr46xlBMevnw1DhGG/lTC78R9ak6jGeN8bpbPb2gmWrutIWgDfSGUVDifUOWF3X4Quix9yInZ1+HXwWNqToiLnQREQEREBERAREQEREBERBisiszrxSRGCRsFwpJRU0c7wSI5QCOoBBLXNc5jgD1a9wBHeuu13ymvgnt9VEKa4saW1NumOzy9xc3YHPGd9HgaPcdEFozKx15x63ZDFHHcKSOp7Il0Uh22SJxGi5jxpzDrptpBW6mqmY0a9np/v933xQgeDZwnaQRw3xYEdxFog/RXH/Zr4T/+W2K/NEH6KkJwYx9KfIr7Ts6AM8cEuh+GRrnH8Z2nkTUe+q/flofqlloW+3wkxG9JKenipKeKCGNsUMTQxkbBprWgaAA9AAXYov5E1Hvqv35aH6pPImo99V+/LQ/VJzdvt8JMRvShFr74LV6yHjHwXtOVX7KLqy51VTVxSCjdFHHyxVMkbdAxk/asG+vera8iaj31X78tD9UnN2+3wkxG94Mi4HcPMuvNRdr3hFgu90qeXtqytt0UssnK0NbzOc0k6a0D8ACx7vBv4UvawO4cYu4MHK0G0wHlGydDzfWSfxrP+RNR76r9+Wh+qQYTMQQ7J789p6a7eIf2iMFObt9vhJiN7utlpxfhbjopbdRW7GrNG8ubT0kTYIu0ce5rGgbc4+gDZPdsrmz0VRdbsL7XwGmLYnQ0NK/7eKNxBc949D3cren80ADvLl2WvC7Vaq0VoilrLgAQKyunfUSt33hrnk8gPqbofAs6pNVNETFvr6/Y1RsERFoQREQEREBERAREQEREBERAREQEREBERBrv4Af3MOPfHbj9NmWxC138AP7mHHvjtx+mzLYhAREQEREBERAREQEREBERAREQEREBERAREQEREBERBrv4Af3MOPfHbj9NmWxC138AP7mHHvjtx+mzLYhAREQEREBERAREQEREBERAREQEREBEXkutzprLbqmuq5Ozpqdhke4NLjoegAdST3ADqSQB1ViJmcQPWihcmR5VUHtKe0Wymid1bHV1rzKB/nckZaD6wC4fCV+PZ3MPvCx/K5vq119Fr3x5wuE3RQj2dzD7wsfyub6tPZ3MPvCx/K5vq06LXvjzgwm60b/ZP+Bbspwm2cSbZTmS42ACjuPINl1E95LHf6uRx7vRK4no1bW+zuYfeFj+VzfVrH5C3IsqsNxst0s9hq7bcKeSkqad9XNqSJ7S1zT7X6QSnRa98ecGHzH/AGPLgpJxT470V7qY3CyYk6O6zyDYDqkO3TR7HcedvP6iInD0r6/rXXwdODF08HDBZccs0FpuDqirkrKmvqKiRskznaDQQI9ANYGt0Om9npzFWn7O5h94WP5XN9WnRa98ecGE3RQj2dzD7wsfyub6tPZ3MPvCx/K5vq06LXvjzgwm6KEezuYfeFj+VzfVrkX3MAetvsh+AVkw3+PsuidFr3x5wYTZFh8eyEXoVMM1OaO4UpAnpi7nAB3yva7Q5mO0dHQ7iCAQQMwuWqmaJ0atqCIixBERAREQEREBERAUT4oHWIP+GuoQd+o1cIKliifFH+KDvj1B9MhXTyb79vxj1ZU7YelFhc2mvVPh19lxuGGpyFlDO63Q1B1G+pDD2TXdR0LtekfhC1ui48ZJj3DGp1kFxyPPKm622yyWm62WCirLPU1JIO4QYmStIDjEXODHEDbyN66JqwxbP3e82/H7bPcbpXU1tt9O3mmq6yZsUUY3rbnuIAGyO8r2LTripeOJb+D3Em35VQ3qoxw2Rk8F0yClt1NUsqhUxgwhtHM9r2Oa7mBLWkFhBJ2FYXFLiVmtx4tVuGYjHfKams9tp66rqbBRW+pqJZZ3SBjXeOzMY2MNj/mBziSRtuhuaQ2DX5dIxjmNc5rXPOmgnRcdE6H4gT+Ja6WfOuKOaXvE8ErqhmBZI+0Vd5vNxbSQVE74YqoU8DYoy6SJjpA5sj9l4bvQXVxPw/J5eLPBOglz64R3Td2jN1pbfRscXCmc7tBG+J7OYsIjI1y6GwGkppdw2SX57RgkEfM3tCOYN31169fjC5YC1oBcXEDRcfSqBqceu1b4ZlVUU+U19BTR4nR1L6OGnpnMkhFZK11OS+IuDHFrnFwIft5AcAABlM4F/otW38WM++xlNxj8ooWWGO7mIYh7HxdkaFtf4oQZ9dr2+gX7DuXem8i92Q8Q+IN2sPFDO7PksFnteE3CupKXHX2+KWKujomgzGeVw7QOkIeG9mW8o5T521jpQNiqW82+uuFbQU1dTVFdQlgqqaKZrpKcvbzM7RoO28zeo3rY6hexa0We/wCSXC+8e71g9JHNktRRWaqttNUaI7R1ua4DR0C4AnQOgSBvorJ4CZa/KscuHjOV12S3Kkq+yqobtao7bXW9/I0mCaFjWje9uDtaIcNF2tmxVkTvHjriNeB3btNHv4fbqn/9/rKmyhOPfyj3f+iaT/nVKmy1cq+5/EekLIiIuNBERAREQEREBERAUT4o/wAUHfHqD6ZCpYsPltkkyHH6mihkbFO4slifINtEkb2yM38HM0bW+xVFF2iqdkTHqsaphh73axe7NXW51TU0TauB8BqaKUxTxBzS3mjeOrXDew4dx0VWMfgzYvU2nIqW93G+ZNW3wUzai7XWsBrIvFyXU5ifGxgjMbnFwIGySeYlT52VOpQGVtlvNPUjo+OG3TVLQfTp8TXNI+Ha/PlnTe5l++ZKv6pehzNc/wDOV0ZRWDgZb6nEsix2/wCR5JltDfKZtLUOvdc174mNDtdkI2Max23b5uXZIbsnQXnvHg/2661dnuUOUZPasjt9ALY+/wBBXRtrK2nB5gyoLo3Mk87bt8gIJJBCmXlnTe5l++ZKv6pPLOm9zL98yVf1ScxX2ZNGdyL5JwNtWRQY5I295Ba75YIDTUeQUNf+33ROAD2Svka9sodytJ52nqNjS4vnA2336x45STZHkcd1sFTJVUOQsrmuuLXyB7ZA572Oa5rmvLS0s0AABoAKU+WdN7mX75kq/qk8s6b3Mv3zJV/VJzFfZk0Z3I4+68Sra40dHiViuVJTnsoa2tyiWOeoY3o2SRraAhr3AAkAkAk9SvxcuFXldkljzGurrjimVUtIKOqZYLg2SGeHtO07CR0kPtjA7ZBDWO84rKY9xYx/LbVHc7GbleLbI5zWVlBa6meJxa4tcA9sZBIcCD16EELJeWdN7mX75kq/qk5i52ZTRlBJfBpxmW7uldc757AOufsw7FvHG+xZq+07Xn7Pk5+XtPbOz5+Tm68q/WT+DbjmU3a8Ty3W/UFpvdQ2qu9goK0RUFxlAaC6VnIXjmDGh/I9odrrtTnyzpvcy/fMlX9UnlnTe5l++ZKv6pOYr7K6M7kduPBa1VWTZBfKG73uw1d8t8VBWMtNYIYyYtCKdg5SWSsaOQOB1ykggr3cOuFlv4cSXmqhuVzvl2vE0c9fdbxO2WonLIxHGDyNY0Na0aADR6e9ZTyzpvcy/fMlX9UuRmNO46ba76TroDZasb/GY9JzFe3RlNGXrx7+Ue7/ANE0n/OqVNlFsStdU+6XC91lO6idVxRU0FLJrtGxRl7ud+u5znSO83Z0Gt3olzRKVxcpmJuat0ekEiIi5UEREBERAREQEREBERAREQEREBERBrv4Af3MOPfHbj9NmWxC138AP7mHHvjtx+mzLYhAREQEREBERAREQEREBERAREQEREBERAREQEREBERBrv4Af3MOPfHbj9NmWxC138AP7mHHvjtx+mzLYhAREQEREBERAREQEREBERAREQEREBERAREQERdbp4mO06RjSPQXAIOxa0+Fj4YVx8F292OnfgJyS1XanfJFchdvFQ2ZjtPiLOwf3NMbt7G+cjXm7WyPjUP+Wj/2gqZ8LfgzTce+CV6sMJide6UeyFpfzDYqo2nlZ+B7S6P4OffoVxI1E8CPwzq21twrg9QYA+6z1lzla66Mu3IYopp3zSy9j2J2ImOe4jnGww92+n0mXz9/YwOBos9DeuJl7gENXUOfa7VHOOVzI2n2+UA+twEYPQjkkHcVv741D/lo/wDaCYkdqLq8ah/y0f8AtBdqYBERQEREBERAREQEREBERAREQEREBERBF89rpoqa1W+Gd9N7KVwpJJoXlkjYxFLK8McOrS4RFvMNEBxIIIBGFOAYw4Dmx21PI9L6KNx79nZI2epJ/GsjxB/hPDP6Yd9Bq11ZNkttw+xVd5u9T4nbaRofNNyOfygkAaa0FxJJA0AT1Xq26qrdqnQnGdfGYZZxGp4/sfYt72rP8gi/RT7H2Le9qz/IIv0Vh8c42YTlWP3m90F+hFuswJuT6yKSkfRjl5tyxzNa9oLeoJHX0bXlxbj7guZTXaO2Xp/7k0gr6+StoqikZTQHenvdMxgAIBI9YBI2ASsufudufNMzvSL7H2Le9qz/ACCL9FPsfYt72rP8gi/RUcsHH/AsnbcTQX3mdQUT7jNHUUdRTyGmYNumjZJG10rB/jMDh1HrC7MZ48YNmFyjoLRezV1MtK+spx4nUMbVQsAL3QPdGGzcuxsRlxHpCc/c7c+Zmd7Pjh/i7SCMbtAI6gihi/RXfj8UWMZXR2qgYKe2V1LNJ4nH0jhkjMejG3WmgteQQNDYaQNlxMJ4G8c7dxssdTV01FWW+rgqKiN8E1HUMjEbKiSKNwmkiYx7nNYHOa0ksJLXAEKbO/lDx34pW/8ARViuq7TMVTmMTwiZWJmdqdoiLx2IiIgIiICIiAiIgIiICIiAiIgIiIIdxB/hPDP6Yd9Bq1DOPNVlFHwzuEmJNrDc+2p2yutsQlrGUpmYKh1Ow7DpRFzlo0evd10prxAYTXYlITpkV3JcdH00lSwf2uA/GvYvTjXao8J9ZWepphWYFV3ql4zxUtkzwWy947bprbV3qmnqqyrnpZZXEcszuf7d0ftL+VxbzcrdaXXe7deeOds4hMqnTUfE64Y5RwUmPG01tpjloqWsE0hZJUta6QySPMfMNBnM0eklbhZFj1uyyx1tnu1K2tttbEYaineSBIw94JBB/qUbwTg1h/DavqK6wWl1NX1EQgkrKmrnq5+yB32Ykme9zWbAPKCBsDp0WrQRS1Di9jy+zZBcKHFOJMOQ0OO3COlfl1TcJmRyTQGN9PC2eV4ke7p1jaWnlHXelnrXjF3huHg1PNprWC00E0VxcaZ48S3aSzlm6e17kAbp2vOAHetgEWWiKW8G2evxq03PB7tYbxbrjbbnc6rx6ooXtoamKWukljdDP9o8lszTyg7HK7YGlaLv5Q8d+KVv/RWYWJDDJxDsPL15KKtc7p3DcA3/AFkLbb1RPhPpKwnKIi8pBERAREQEREBERAREQEREBERARcEga2e/u+FUq+5XjwmMEqo7JXZZwogpb52D6yWkZBWXCmhPn9mHbMbXu2ObvBjIIIJaQzWVcQ7RnV3zPhljlZTVGdW61eNmKuppjSU0rgDB2krNEODjG8cjuYdHN6tOvVinDjJ6PGrbBf8APK+svUcIbV1FBSUscMknpLWvie7XcNk9db0N6Fgw0kMEssrImNml12soaA6QgaBcQOvRdy3UXq7cYpnV/E+q5whvkBcPfne/yFF+rqPcQ4mcM8HveU3nOL1FbbVSvqZT2NCC7Q81jf2v9s52mgekuAVprSL9koq+IGXWawYDhuHZHfbZUfundKy02qephcWuLYYC+NpGwQ97mnr+9FbOk3O7yj2Mpb4HPF+4eExgdxrrhlNwt2SWurMNbRUcNH2Yjft0MjQ6AkAtDmnZ+2jcegICv7yAuHvzvf5Ci/V18vPAJ+yLgPGfHr7bMNyO4YffJfYe41tNbJ30gjfIGGV0obyARSAOc4noGvHTZX11TpNzu8o9jKHDAbgCCcyvR16DBRdf/rrD5Vw/yunitVRhOUQ2+6x18MlzqL3Rtq/ZCkG2ugJby9mAHOe0RhoLh3jmc5WSiwrv3K40ZnV4RHoZQvHuKltyLiLkmFx2670d1skcU8k9ZQPjpamKQDUkMvc4c3M3roksdrYBKmixWVY3RZjjV0sVx7bxC5U0lJUCnmdFIY3tLXae0gg6J7lAqa0ZdwppsBxnFLbJmOMQvdR3e5Xu7E3KmiJBZOHOGpWt2/bR10GBoAGxoRaSLA43nmO5hW3ajsl7obpV2mpdR19PSztfJSzNJBZI0dWnYI6+o+pZ5AREQEREBERAREQEREBRXMc3Nks+Qiw0keUZPaaLxwY7S1cbKmTm5uzBB2WB3K7RI68p0Cei8/F6pzWj4f3Gfh7S0Vblsb4HUlNcCBDK3tmdq1xLmge18/Xf4Oq78a4ZYxjOU3zKrbYqagyK/Fj7lWtHNLKWtaA3ezoeaCQ3QJ847PVBG7bw0dnV9wTiBmNPXWjLrLQOHsJR3Rz6GmqJW6kdyt6PcAXt3vRDtHm00iz0RAREQERVJx08IOg4SCgsdroJcq4gXn2uz4xQnc07jv2yU/4OIaO3n1HXc4gIj4Af3MGPfHbj9NmWxKqjwXOFl14NcEbBi18mp5rvA6eoqfFCTGx80z5SwE9/Lz636SFa6AiIgIiIIRlPDZs1kyd2F1NLg+V3sRvlyCioIpJXSsO2PlaRqToXN87rpx0drw0/EWtxXKcNwjIbbdbtertb+abI7dbiLYamNhMjXOBPZF3I9wB6a161YqIOAQ4dDv0dFyqZteLu4A3qyWTCsdu17suU36eouk9TXTVEdna6MOL28wdysc8E6J2XOJLj0AuZAREQEREBERAREQeG+2z2bslwt3jM1H43TyU/jFM8sli52lvOxwIIcN7BB6EKveFuVYrhvCGnbU59SXm04y72Ir8kutQ2BgnjeIy2V73aDtuY3qepI6klQ7w0sD4j5lwsbVcNMou9jvFnldWS26zzugluTOXRYJGak5mguLWA8rydEFwYW/KDhFilZxa444xZLk+ouFTer1EK+aoeZJZGulDp5HuPVzuXncSep6oPvAiIgIi1w4i8b8k4o5fW8M+Cz4pLnTO7K/5pI3tKKyNPQsjPdLUd+mjoCPgcWBm+NHhB11oySPhzw0t8WVcTqxnM6Fx/admiP/eKx4+1A2CGd52PW0OzHAzwe6HhO+vv93uEuWcQrz593yauHtsp6e1RD/BRDQAaPUN9A0NzfBjghjnA/HJLdZI5aqvrJPGLnea5/aVlxnOy6WaQ9Sdk6HcNn0kk2CgIiICIiAiIgIi8V6s9FkVmr7VcqdlXbq6nkpamnkG2yxPaWvafgIJH40EUz615RX5ThM9hyGls1rpbg+S70c+ue4QcmhEzzT1DuvePwqcL4QcfOElbwQ4s5Dh9ZzyMopyaSdw/f6d/nRP7tbLSNgdzgR6F9WPAN4Q1nB7weLVTXLtI7pfJ3Xypp5BowGVkbWM1rYPZxxkg9Q4uHoQbDoiICIiAiIg4JAGz0ChsmbXW4O7Wx2SnrKA/vdVXVzqYTD/GY1sUhLT6Cdb7wNEEyDJ3mPGrs5pIcKSYgj0HkKjuMtDcbtIaA0CkiAAGgPMC7rFFGhNdUZ146/1herJ5UZb73LP89S/qqoSk8GaS0+ExS8YLVY7PbqhsU7qizx3OTspauRjo3VAd4uOQlr3bbynbjzbB3vYtkjJC4Nc1xaeVwB3o+o/1hfpb/pfjj/17me5jfKjLfe5Z/nqX9VTyoy33uWf56l/VVkl47RebfkFuhuFrrqa5UE4JiqqOZssUgBIPK5pIPUEdD6E+l+OPOr3M9yt+NNl4k8TcdgsdtrafELRPLq7yWWrNRcaqm/nRU8kjImREjYLjvex1ABDrH4TYXimA4HbLRhdBFb7DGzmjYxha97u5zpebzjJsadzedsaOtaXtXl4aHdjrx0Abdq8AAaA/bMh/4kn8a1XqKJtzXTTjExv6875ldsJaiIvPYiIiAiKH8Tszfh9iYKQt9lK1/YUvMNhh1t0hHpDR1+Elo9K22rVV+5FujbI7ct4l2XD5fFqh8tZcC3mFDRND5det2yGsHq5iN+hQibj1cC4+L4tEWb6eMXPkdr4Q2Jw/tKrlrCHSPc90ssrzJLLIdvkee9zj6SV+l9vZ+D8mt04uRpT4zHpgzCwPs83j3q0fzu79XT7PN496tH87u/V1X6Lo+V8i/Hxq900u5CuPGBUPHviDhWVXfHKOmksEuqumFeZBc6cOD2wPcYBytDubro9HvGuoIvX7PN4H/hWj+d3fq6r9E+V8i/Hxq9zS7lgfZ5vHvVo/nd36uuRx5u++uK0mvgu7ifo6rW3XKjvFFFWUFVBXUcw5o6imkEkbxvWw4Eg93oXpT5ZyKdcW+NXuaXctyycc7VVysivFDUWNziB28hbLT7PoL29W/hc1o+FWPFKyaNkkb2yRvAc17TsOB7iCtXFN+EuYyWK8U2P1DybXWlzaQHup5tF3IP8AMeA7Q9DgAPtunj8v+E0UUTd5P1bY7u5dq70RF8mMXlX8WLx8Tm/uFR7Gv4uWr4pF/cCkOVfxYvHxOb+4VHsa/i5avikX9wL0bP2Z8f0vU1awS/3/AIO8LOM+Z+UFZkL7ZkN4hhtlbT0zIH1XjTWNqXujja/ZJ25ocGaJ00dCJfkOf5rwIyCgiyjJBndFc7FdbgYzQQ0j6aqoacTlsRiaNxPbzN0/bgQ3zjvSnjeAGP8Asjlbn112msmTmd9yxySoabfJLM0Nlla3k52vdrew/QPUAFcYrwBsmP3j2TuV3vmYVcdBJa6U5HVMqW0tLJrtI42tY0eeGtDnO5nEAAu1tYREwiCWDM8/xSo4W3zI8pgyG3ZvUxUVXaWW6KCOglnppKiJ1O9g53NYY+R3aF2wd9CodheQZhjngx8LanHaiut1k7Wp9nrnaLdHcK2kpw+YsfHA8ODmc+uchrnBvUDvVy4f4OtgxC+WW4ezF/vUFhY9lltt2rhNS2wObye1NDA4kMJY0yOeWtOhpcUXg+UNkxuOx2LL8tx6ghrp62mbbriwGnbL9tTt543Awg8zmtcHFpcSHd2mJE3wK7w37C7JcYL3FkkVTSRyC7wRiJlWS0bkDB0Zs7PL6O70LJ8M/wCBLh/S1d9Iesbg2F2vh1iNrxuyxPhtdthEMLZHl7yN7LnOPeSSST6yVkuGf8CXD+lq76Q9bK/sVeMftepLkRF5qCIiAqO42VDpc6tsDjtkFtc+MeoyS6d/y2K8VU/HSxSfuVkETdx0vNSVWh3RyFpY8/A17df6zfo6ex8Jrpo5XTpdeY/mY/0LCsUX5kLhG4saHPAPK0nQJ9W/QoeL1nu+uJ2PX/uCX9TX3tVcU7fSZ9GtMlr9U8QuIeU1t/uGM0d0fBb7hUUNFRQUdC+jnML+Q9vJLM2YFxB3yAcoI0Ha62d7NZ9707F/vDL+pryDhDRC91N1o7verCa+dtZXW211wZSzz9OZ5BZzAu0OYtLebXULkvRVexFEzG/bHrCode8nzSuruJVRR5AbLFjNPBVU1AKKGUOeaJkz4pHuBJZzc32unecfO0ABkqHK8l4m5MbfZby3FqO32mir6qWOkjqJZ5qpjnsYBJsCNrWHeupJ1sKaTcO7bPLmD3T1QOURtirNPb7WBAIB2fm9Dyjfnc3X4OixVVwbtj6y21lDdrzZa2joIra+ottS2N1VBGNMbMCwgkddOABGzojotc2r0TnMzGZzr75xjd1bN2B5vByBHBPFNnZ8Wds/6x6sdQKz2K+8ObJbsdxizUd3tFDDyR1V1vDoJyS4khzWUzge/v2PwL1ezWfe9Oxf7wy/qa6LVUW7dNFUTmIiNk+wma6KyodRtgqo/wB9p6iGeM/5zZGuH9oXix6qvNXSyOvVto7ZUB+mR0da6qa5uh1LnRR6O99NHu71JcWsUmUZZa7cxpdCyZlZVO10bDG4O0f9Jwa38Z9RW2u5TRbm5VsiFp2tlURF+WK89wo23CgqaV5IZPE6JxHoDgQf+Kr6ivgxihprZd6Wugq6WNsJlgopp4Zg0AB7HxsLdHW+U6I7iFZKLptXotxNNUZjyXKvPLu0+q4fNdV9Wnl3afVcPmuq+rVhot3SLXYnz/o1K88u7T6rh811X1aeXdp9Vw+a6r6tWGidItdifP8Ao1K+ZmlDUeZTU1zq5z9rDHbagOcfVtzA0fhcQB6SFJcNs09jsYhquQVc081VM2M7ax0sjpOUHQ3y8wbvXXW/Ss4i1XL0V06FMYjxz+oM7hERcqCIiAuuppoqynlgniZPBK0skikaHNe0jRBB6EEehdiJsFH5XwiutjnfNYojdrYSS2lMoFTAP8UF2hI0esuDtaHnnZUKlgraZxbPaLtA8d7ZLbOP6jyaP4traVF9FZ+N3rdOjcpirv2SuqdrVfc/ufcvm+f9BNz+59y+b5/0FtQi6Pn1X4+P9JiGq+5/c+5fN8/6Cbn9z7l83z/oLahE+fVfj4/0Yhqvuf3PuXzfP+gg8YcdC3XMn1C3Tkn/AOC2oRPntX4+P9GIa42XCMkyKVraW0TUMJI5qq6MdTsaPTphHaE/BygesjvV24VhVFhVtdBTuNRVTEPqayQAPmd6PwNGyGtHQfCSSZCi8rlfxK9yuNCdVO6P2vgIiLykf//Z", + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "from IPython.display import Image, display\n", + "\n", + "try:\n", + " display(Image(graph.get_graph().draw_mermaid_png()))\n", + "except Exception:\n", + " # This requires some extra dependencies and is optional\n", + " pass" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "66f62a69-989b-46ce-80b3-97a867e36782", + "metadata": {}, + "outputs": [], + "source": [ + "user_input = \"Can you give me some information about AMD in 2022?\"\n", + "\n", + "result = graph.invoke({\"messages\": [(\"user\", user_input)]})" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "479a459d-6896-4960-aae9-9f1259fb47d1", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "['3a42d402-4f6f-44ce-9ff4-d53d50f5f24a', '42aafb13-52e4-402b-80aa-db5be2e0e40f', '275bcddc-a918-4688-938f-9d2d6f232757', 'abc00b37-e0fa-470f-98d7-f055cc29a11f']\n" + ] + } + ], + "source": [ + "print(result[\"selected_tools\"])" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "376f28fd-3f7f-4ae5-a34c-baef1778e82b", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "================================\u001b[1m Human Message \u001b[0m=================================\n", + "\n", + "Can you give me some information about AMD in 2022?\n", + "==================================\u001b[1m Ai Message \u001b[0m==================================\n", + "Tool Calls:\n", + " Advanced_Micro_Devices (call_oXCnqxGmzQaP4OTEWvPghiKR)\n", + " Call ID: call_oXCnqxGmzQaP4OTEWvPghiKR\n", + " Args:\n", + " year: 2022\n", + "=================================\u001b[1m Tool Message \u001b[0m=================================\n", + "Name: Advanced_Micro_Devices\n", + "\n", + "Advanced Micro Devices had revenues of $100 in 2022.\n", + "==================================\u001b[1m Ai Message \u001b[0m==================================\n", + "\n", + "In 2022, Advanced Micro Devices had revenues of $100.\n" + ] + } + ], + "source": [ + "for message in result[\"messages\"]:\n", + " message.pretty_print()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "871d160e-a2a3-4b13-bbd6-f7426b8f2f8d", + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.4" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} From 590f810b53ac68cac8d0981c74ff85d7ff1f7bc3 Mon Sep 17 00:00:00 2001 From: Chester Curme Date: Wed, 24 Jul 2024 12:45:37 -0400 Subject: [PATCH 2/6] add concluding text --- examples/agent_executor/many_tools.ipynb | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-) diff --git a/examples/agent_executor/many_tools.ipynb b/examples/agent_executor/many_tools.ipynb index b6e636dc1..00aebc53c 100644 --- a/examples/agent_executor/many_tools.ipynb +++ b/examples/agent_executor/many_tools.ipynb @@ -282,12 +282,21 @@ ] }, { - "cell_type": "code", - "execution_count": null, - "id": "871d160e-a2a3-4b13-bbd6-f7426b8f2f8d", + "cell_type": "markdown", + "id": "177aedfa-cec5-45d0-82ad-efc0233aa6b4", "metadata": {}, - "outputs": [], - "source": [] + "source": [ + "## Next steps\n", + "\n", + "This guide provides a minimal implementation for dynamically selecting tools. There is a host of possible improvements and optimizations:\n", + "\n", + "- **Repeating tool selection**: To manage errors from incorrect tool selection, we could revisit the `select_tools` node. Options include:\n", + " - Modify `select_tools` to generate the vector store query using all messages in the state (e.g., with a chat model) and add an edge routing from `tools` to `select_tools`;\n", + " - Equip the agent with a `reselect_tools` tool, allowing it to re-select tools at its discretion.\n", + "- **Optimizing tool selection**: In general, the full scope of [retrieval solutions](https://python.langchain.com/v0.2/docs/concepts/#retrieval) are available for tool selection. Additional options include:\n", + " - Group tools and retrieve over groups;\n", + " - Use a chat model to select tools or groups of tool." + ] } ], "metadata": { From 8d4b95afa8d10c5ffbbad3e20811956492ec9cf4 Mon Sep 17 00:00:00 2001 From: Chester Curme Date: Fri, 26 Jul 2024 11:22:01 -0400 Subject: [PATCH 3/6] add section --- examples/agent_executor/many_tools.ipynb | 177 ++++++++++++++++++++++- 1 file changed, 170 insertions(+), 7 deletions(-) diff --git a/examples/agent_executor/many_tools.ipynb b/examples/agent_executor/many_tools.ipynb index 00aebc53c..6bd9272d2 100644 --- a/examples/agent_executor/many_tools.ipynb +++ b/examples/agent_executor/many_tools.ipynb @@ -110,7 +110,11 @@ "from langchain_openai import OpenAIEmbeddings\n", "\n", "tool_documents = [\n", - " Document(page_content=tool.description, id=id)\n", + " Document(\n", + " page_content=tool.description,\n", + " id=id,\n", + " metadata={\"tool_name\": tool.name},\n", + " )\n", " for id, tool in tool_registry.items()\n", "]\n", "\n", @@ -239,7 +243,7 @@ "name": "stdout", "output_type": "stream", "text": [ - "['3a42d402-4f6f-44ce-9ff4-d53d50f5f24a', '42aafb13-52e4-402b-80aa-db5be2e0e40f', '275bcddc-a918-4688-938f-9d2d6f232757', 'abc00b37-e0fa-470f-98d7-f055cc29a11f']\n" + "['ed32d6f0-76c8-42ad-9ddb-eb28ab73cba3', 'ab4b0a2d-e535-419d-8d8b-d1e1cc6a410b', 'da70a795-fd64-474d-be7f-d39314255f6c', '3f605f81-bcd0-4bae-848b-fcae0b2115be']\n" ] } ], @@ -262,8 +266,8 @@ "Can you give me some information about AMD in 2022?\n", "==================================\u001b[1m Ai Message \u001b[0m==================================\n", "Tool Calls:\n", - " Advanced_Micro_Devices (call_oXCnqxGmzQaP4OTEWvPghiKR)\n", - " Call ID: call_oXCnqxGmzQaP4OTEWvPghiKR\n", + " Advanced_Micro_Devices (call_RhGe3Zhp9ENphSC8Pusd7ZWe)\n", + " Call ID: call_RhGe3Zhp9ENphSC8Pusd7ZWe\n", " Args:\n", " year: 2022\n", "=================================\u001b[1m Tool Message \u001b[0m=================================\n", @@ -281,6 +285,167 @@ " message.pretty_print()" ] }, + { + "cell_type": "markdown", + "id": "3bd847ef-4627-4fc2-99f9-c2b17cf83f95", + "metadata": {}, + "source": [ + "## Repeating tool selection\n", + "\n", + "To manage errors from incorrect tool selection, we could revisit the `select_tools` node. One option for implementing this is to modify `select_tools` to generate the vector store query using all messages in the state (e.g., with a chat model) and add an edge routing from `tools` to `select_tools`.\n", + "\n", + "We implement this change below. For demonstration purposes, we simulate an error in the initial tool selection by adding a `hack_remove_tool_condition` to the `select_tools` node, which removes the correct tool on the first iteration of the node. Note that on the second iteration, the agent finishes the run as it has access to the correct tool." + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "1954a5f1-91e4-4b32-9be9-c8bc1cc43cb5", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_core.messages import HumanMessage, SystemMessage, ToolMessage\n", + "from langchain_core.pydantic_v1 import BaseModel, Field\n", + "\n", + "\n", + "class QueryForTools(BaseModel):\n", + " \"\"\"Generate a query for additional tools.\"\"\"\n", + "\n", + " query: str = Field(..., description=\"Query for additional tools.\")\n", + "\n", + "\n", + "def select_tools(state: State):\n", + " last_message = state[\"messages\"][-1]\n", + " hack_remove_tool_condition = False\n", + " if isinstance(last_message, HumanMessage):\n", + " query = last_message.content\n", + " hack_remove_tool_condition = True\n", + " else:\n", + " assert isinstance(last_message, ToolMessage)\n", + " system = SystemMessage(\n", + " \"Given this conversation, generate a query for additional tools. \"\n", + " \"The query should be a short string containing what type of information \"\n", + " \"is needed. If no further information is needed, \"\n", + " \"set more_information_needed False and populate a blank string for the query.\"\n", + " )\n", + " input_messages = [system] + state[\"messages\"]\n", + " response = llm.bind_tools(\n", + " [QueryForTools], tool_choice=True\n", + " ).invoke(input_messages)\n", + " query = response.tool_calls[0][\"args\"][\"query\"]\n", + " tool_documents = vector_store.similarity_search(query)\n", + " if hack_remove_tool_condition:\n", + " # Remove needed tool\n", + " selected_tools = [\n", + " document.id\n", + " for document in tool_documents\n", + " if document.metadata[\"tool_name\"] != \"Advanced_Micro_Devices\"\n", + " ]\n", + " else:\n", + " selected_tools = [document.id for document in tool_documents]\n", + " return {\"selected_tools\": selected_tools}\n", + "\n", + "\n", + "graph_builder = StateGraph(State)\n", + "graph_builder.add_node(\"agent\", agent)\n", + "graph_builder.add_node(\"select_tools\", select_tools)\n", + "\n", + "tool_node = ToolNode(tools=tools)\n", + "graph_builder.add_node(\"tools\", tool_node)\n", + "\n", + "graph_builder.add_conditional_edges(\n", + " \"agent\",\n", + " tools_condition,\n", + ")\n", + "graph_builder.add_edge(\"tools\", \"select_tools\")\n", + "graph_builder.add_edge(\"select_tools\", \"agent\")\n", + "graph_builder.add_edge(START, \"select_tools\")\n", + "graph = graph_builder.compile()" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "9110789a-843a-4c21-aeff-8841b24f7674", + "metadata": {}, + "outputs": [ + { + "data": { + "image/jpeg": "/9j/4AAQSkZJRgABAQAAAQABAAD/4gHYSUNDX1BST0ZJTEUAAQEAAAHIAAAAAAQwAABtbnRyUkdCIFhZWiAH4AABAAEAAAAAAABhY3NwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAQAA9tYAAQAAAADTLQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAlkZXNjAAAA8AAAACRyWFlaAAABFAAAABRnWFlaAAABKAAAABRiWFlaAAABPAAAABR3dHB0AAABUAAAABRyVFJDAAABZAAAAChnVFJDAAABZAAAAChiVFJDAAABZAAAAChjcHJ0AAABjAAAADxtbHVjAAAAAAAAAAEAAAAMZW5VUwAAAAgAAAAcAHMAUgBHAEJYWVogAAAAAAAAb6IAADj1AAADkFhZWiAAAAAAAABimQAAt4UAABjaWFlaIAAAAAAAACSgAAAPhAAAts9YWVogAAAAAAAA9tYAAQAAAADTLXBhcmEAAAAAAAQAAAACZmYAAPKnAAANWQAAE9AAAApbAAAAAAAAAABtbHVjAAAAAAAAAAEAAAAMZW5VUwAAACAAAAAcAEcAbwBvAGcAbABlACAASQBuAGMALgAgADIAMAAxADb/2wBDAAMCAgMCAgMDAwMEAwMEBQgFBQQEBQoHBwYIDAoMDAsKCwsNDhIQDQ4RDgsLEBYQERMUFRUVDA8XGBYUGBIUFRT/2wBDAQMEBAUEBQkFBQkUDQsNFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBT/wAARCAEvAMcDASIAAhEBAxEB/8QAHQABAAIDAAMBAAAAAAAAAAAAAAYHBAUIAgMJAf/EAFYQAAEDBAADAwQKDAsFCAMAAAECAwQABQYRBxIhCBMxFiJBURQVVVZhdJSVtNMXIzI2OFJUcbLR0tQkMzVTYnJ1gZGToQlCkrGzJUNGc3aDlsSFoqP/xAAbAQEBAAMBAQEAAAAAAAAAAAAAAQMEBQIGB//EADMRAQABAwAEDAYCAwAAAAAAAAABAgMREiExUQQTFEFSU2FxkaHB0QUVIzOx4YHwMqLC/9oADAMBAAIRAxEAPwD6p0pSgUpSgUpSgVjTLnDt+vZUtiNvqO+cCP8Ama0HeS80KzGlPW2wglAfj+ZImkHRKF/7jXQgKHnL8UlKQFLyomB47C2W7LBU4SVKedZS44onxJWraifzmtjQoo1XJ17o9f7K4jnZXlVZfdiB8pR+unlVZfdiB8pR+unkrZfceB8mR+qnkrZfceB8mR+qn0e3yXUeVVl92IHylH66eVVl92IHylH66eStl9x4HyZH6qeStl9x4HyZH6qfR7fI1HlVZfdiB8pR+unlVZfdiB8pR+unkrZfceB8mR+qnkrZfceB8mR+qn0e3yNQMpspOhd4G/jKP11nx5TMxvvGHm32/DnbUFD/ABFYAxayg7Fogb+LI/VWBI4e2Bxzv41ubtcwDSZds/grw9XnI1zD+irY8dg7NMWZ55jwn1hNSR0qP264zbTPZtV3dMovbES5cgQH9DfduAdEu62egCVAEgDRSJBWKuiaJJ1FKUrwhSlKBSlKBSlKBSlKBSlKBUbz59ftI1b2llty6SWoHOkkFKFq+2kEdQe7DmiPA68PGpJUYzwdxHs0877uDdI7rhA3pKyWSfzDvdn1AE1nsfdp/uvm81jakbDDcZhtllCWmm0hCEIGglIGgAPQK9lKVgQqD5pxswzh9kESx328GLdpTIkIisxH5CkNFfIHHO6QoNoKgQFL5QSD16VOK5u7Rwu1jzZi/wCB2XLhxIRbmo8SbaraZVouTXfqPsOaT5iAnalc5KCkObCz9yAnWN9oO1X/AI15Rw8VBnsSrSY7bEoQJS25Di23HHOdfc920lIQAlSl6Xs8pPhW1xLj9gWcZR5O2a/d/eFJcW1HehvxxIDf3ZZW42lLvL6eQq0OvhUJtkq9YN2iM7dkY3dpTOWwrUbZcoUJyRBbeYaebcRIdSNMgKUk7VraTsVUuFW7LLtxB4P5Bf7RxCnZJbrpIGTSrtHeTboTr8R9kJjMg933XOsDvWklIQAVr6igve5dqDBUWLIZ1onS769Zo8tx9iHbJikJdjlSVtLcSyUtq5k60rrykLAKetSTg7xSg8X8DtmRQ2JUVb7DKpLEmG/HDbymkLUlBebR3qBz6DiAUq10NV3wfwa6p7OuX2F22PW273SbkIbYmMlhay9KkhpZCgDpSVIIV4FOiOlSrs4ZA/cuFOO2ibYL3j9ysVrh26Wxebe5F5nm2UoX3SlDTidoPnJ2NEeugtGlKUGmy+0rvOOzWGSEzEo76K4f+7fR5zSv7lBP5xsemsqwXZu/WK3XNocrU2M3JQPUFpCh/wA68r3dG7JZp1wd2W4rC31BI2SEpJ0B6T0rCwq1OWLDrFbXv46JBYYX/WS2lJ/1FbG2zr36vDX6LzN1SlK10KUpQKUpQKUpQKUpQKUpQKx7hAj3WBJhS2kvxZLamXWl+C0KGlA/nBNZFKsTMTmBG7Vd3LI8zZ709p/7iJOcJCJid6SCo9A9rW0/73VSemwnVX7gVw6ym7ybrecGx66XOSoKfmTLa0664QAAVKUkk9AB19VTKdAjXSI7EmR2pcV1PK4w+gLQseopPQio/wCQMZjpAul4tjfUhpictaBv1Jc5wkfAND4Kz/Tua5nE92r9LqlHj2bOE58eG+LH/wDEMfs1MsYxKyYTaUWvH7TCsttQpS0xIDCWWgpR2SEpAGya1vkTI99V+/zmfqqeRMj31X7/ADmfqqcXb6flJiN6UUqL+RMj31X7/OZ+qqpcDvWQ5J2hOKOFS8ouos+NRrW7CU2poOlUhlS3OdXd6PUDWgNfDTi7fT8pMRvdBVGsx4aYlxDMQ5RjVqyExOf2ObnDbf7nm1zcvMDrfKnevHQ9VeHkTI99V+/zmfqqeRMj31X7/OZ+qpxdvp+UmI3tAOzfwpDakDhxi4QohRT7UsaJG9HXL8J/xNbzE+GGF8OHJcvHMZs2OLebCZD1vhtxytCeulFIGwPHrXmnCpAIPlTfTr0F5nr/APyryb4fWtxxC7i7NvakHaU3OUt5oH191sN7+Hl3TQtRtr8I98GIepTyc8fYEfTmOsOpeXJG9TXEKCkJb9CmgoAlXgogAbG6ldfgAA0Ogr9rHXXpYiNUQFKUrGhSlKBSlKBSlKBSlKBSlKBSlKBSlKBSlKBXO/CT8Mnj58SsH0VddEVzvwk/DJ4+fErB9FXQdEUpSgUpSgUpSgUpSgUpSgUpSgUpSgUpSgUpSgUpSgUpSgUpSgVzvwk/DJ4+fErB9FXXRFc78JPwyePnxKwfRV0HRFKUoFKUoFKUoFKUoFKUoFKUoFKUoFKUoFKUoFKVDpGX3W5OuqsUCG9CbWpsS50hbYeUk8qihKUK2jYICiRvWwCkhRy27VVz/FcZTGlQj29zD8gsfyt76unt7mH5BY/lb31dZ+S1748YMJvSoR7e5h+QWP5W99XT29zD8gsfyt76unJa98eMGEW7V/BRHHrglfMbabC7uyPZ9qUTrllthXINnoOcKW2SfAOE+ivjhwu4XXbilxPsmEwWls3K4zBFX3jZ3HSCS6tSeh0hIUojx0k19uPb3MPyCx/K3vq6pzAOz09w7435fxMt0CzG55AjQiKkOhqIpZCn1oPd7JcWAo+ragOitByWvfHjBh0Ni2OQsOxi0WC2oU3brVDZgxkKVzFLTSAhAJ9J0kVtKhHt7mH5BY/lb31dPb3MPyCx/K3vq6clr3x4wYTelQj29zD8gsfyt76unt7mH5BY/lb31dOS1748YMJvSoR7e5h+QWP5W99XWys2US3Li1brxDZhynwox3YzxdZe5dkp2UpKVgDm5SDsb0To681cGrpjOqe6YMJLSlK1UKUpQKUpQKUpQKUpQKUpQKrnhyebAsfJ8TBaJ/PyirGqueG/3gY98RZ/QFdDg/2q++PxUvMkdK50tfFPJsZ4j5U3nF/l20xFXGXacZVamkxLnAZbUttcaYBzLdCAFLQVbHXzAButTwz4h8Y8ok4bkSrbep9ovjsd+fDkwbYzbIsN9IUXI7rckyT3YUlQ7xKisA7SknQaSOlbTebff4CJ1rnRrlCcUpKJMR5LrailRSoBSSQSFJUk+ogj0VmVyJiPEi+4V2fuHVnxiPJcveSX+6QW3ojDLzzDaJcx1xTSH3G2lOaQAAtQHUnSiAk717ijxYxGwTYF1gS2pV1vFss2PXzJIkNt5LspxSHlPMw3lNqDQSFJI5eYrAI6dZFY6cccQy2pxxSUISNqUo6AHrJryrmvtG4fldm7OGbovGfzMgJEVxDi7ZFjrSkPJC2zyI0UqKkkdAocmuYgmr/xi0zrJZWIdyvcrIZjZUV3GYyy067tRI2llCEDQIHRI6Drs7NeonXgbNbiGgCtSUAkJBUdbJ8BXlVE9qaz3C7TOE6YF+l2NSsxjMhcVlhwhamHyl37ahQ5kcqgB9ye8OwdJ17JNxzrNuJWQYbY80XjjGJWuAX7iq2x5Ei5y5CHFBbiVJ5ENgNdUtpSSpStEAACaWvAvKsOTebfDuMK3yJ0ZifNCzFiuPJS6+EAFfIknauUEE6B1sbrnLBuL+acdpeH2e13lvCnn8aVfbrcIUNqUt54SlxQ0yl4KSlvnacWSQo6KACPGsTD84umc8VeEq74th+9WiflNlmSozfdtSXIwba75Kdnl5glJIB0CTrpTS3DqStJfTq/4gR4+2x6+r+CyKqLg3m+VTeI1zsueZHJgZATLcZxV+0tMxXGEvANPw5SRzPIDeuYFSlAq6hPL1ty/fy9iH9rf/VkVmtTmZ7p/ErG1P6UpXJQpSlApSlApSlApSlApSlAqueG/wB4GPfEWf0BVjVXEFT+EQm7TLt0+QxFHdxpUCIuSh1oHzNhtJKVAaBBHo2CRXQ4NroqojbmPX3eo1xhGm+BNqezhnJbpfb/AJAqLIkyYVrusxL0KG4+lSHC2jkCtci1JCVKUlIOgB0rwwTgPbuHVzhuWjJsnFlgqcMPHX7iF2+OFhQ5Up5OdSU8x5UrWoJOiB0FenIu05w0xC7v2q+5RHst0Y5S7BuDa2H2+ZIUnmQtIUNpII2OoINYkLtacJrlPjQYmZwpU2S4lliMwFrcdWogJSlIGySSAAPHdZ+Ir6MmjO57nOzZiy8Yk2FE28sQvbdV7tq2pvK7Z5SlKUVRF8u0DmWs8quYeer0Gti9wOtV0wi5Yzf73f8AJ2Z0hEsz7pO3LYeRyltbK20oDRQUJUORIG9k72dyfyzje5l++ZJf1VPLON7mX75kl/VU4ivoyaM7kWa4G2+ViGRY5f8AJMjy2BfI6Yz6r1NStbSE75S13aEJQrZ3za2SlOydCkW2ZzgMVm1WKOjOogT3irplWQmPLCydd3ytQlJKQAkhWwSVHY6bMp8s43uZfvmSX9VTyzje5l++ZJf1VOIudGU0ZRa+YNcuLuNJg5nARi0yDPZn26Xjl4VJeYeb2UupW5HbCVDak8pQoEE1iXzs/W+9SmZzeV5TarwYCbZOulunNtSLkwkqKRI+1FJUCteloSlQ5iAQKmnlnG9zL98yS/qqwb3xPsmNWqTdLu3dLXbYqO8fmTLTKaZaT61LU2AB8JNOIr6MmjKO3Ps9445Ex1uwzbthkmwwVWyFNx+Slp72IrlKmXC4haXElSQragSFdQQSTXnE7PmL2uFhrFteudsexWW5MhzI0v7e+t07kB9Sge8S8SefY2d9CnpW+sXFGx5RaY10syLndrZJTzMTYNqkvMujZG0rS2QRsEdD6Kz/ACzje5l++ZJf1VOIr6K6M7kYx3ghbrHm0bKJeQZDkU+EiQi3tXqal5qAHyO97oBCVdQkJ88q0BoaqVX7+XsQ/tb/AOrIrw8s43uZfvmSX9VWTbo0nJ75bJZhSYNutrypHeTGi0t90trbSlKFecEgLUoqIHUJA5tq5bFE2omqqMRifwRExtTmlKVx3kpSlApSlApSlApSlApSlArCvN6t+O2uTc7rOj223RUFx+XLdS000keJUpRAA/PVQcWe1JYsEvqcRxm3yeIHEN7aWcbshC1NH8aS71SwkeneyAQSNdaidl7NWT8YrpFyPj1eW7ulpYeh4LaFqbtEI+jvSDuQsb8SSPEbUk6oOOu3GLVx4vc7inw7x25SMaskduDfcpcYbjQZrnepaZWyFBLjywXA2pXnHlS0NJSjapn/ALMns5i93mRxVvsQqg25ao1kQ6PNdkaIdf16QgHlSeo5lK8Civo0rHrUuwrsarZDVZVxzEVbTHR7GUwU8paLeuXkKSRy61rpqvTimK2nB8bttgsUFu22e3MJjxYrWyG0JHTqSSo+kqJJJJJJJJoNtSlKBSlKBXrkR2pcd1h9pDzDqShxtxIUlaSNEEHoQR6K9lKDk/JeGmVdkrIJuZ8K4L+QcOZbpkX7AmySuJ+NJgerQ8Wx6BrqnXd9C8MeKGNcYMQh5LitybuVskjWx0cZWPum3E+KFjfUH4CNggmV1zVxM4E5HwzzGXxQ4Jpaj3l895fsOcVyQb4gbJUgeDb/AFJChrZJPiVBYdK0quuCfHTHOOeOO3CzqdhXOGv2PdLJOT3cy3PjYU26g9R1B0rwOj4EECxaBSlKBSlKBSlKBSlKBSlKCOcQuIVh4V4fcMnyaeLbZYCUqekFClkcyglKQlIJJKlAAAeJrnwXXi92qPNtSJ3Bvhg943B9IF+ujR/m0+EZBH+949QQVgkVve39+C7knxy3fTWa6JoILwm4J4dwSsRteJWdqAHNGTMX9slS1fjOunzlnZJ14DZ0AKnVKUClKUClKUClKUClKUClKUFEcauzxNvWStcRuGlxbxTifCRoyCP4JeGhr+Dy0D7oEAAL8RoepJTveA/Htni4zc7Nd7TIxXPbCUtXvHpYPMwo+DrSvBxpWtpUPg9BSVW1XO2C/hxcT/8A0za/0lUHRNKUoFKUoFKUoFKUoFKUoOdu39+C7knxy3fTWa6Jrnbt/fgu5J8ct301muiaBSlKBSlKBSlKBSlKBSlKBSvFbiGwOdSU7/GOq8PZTP8APN/8Qq4kV32heK9z4JcLrjmVtxnyrTbltqlQRMMZSGFHlU6FBtzfKSnY0PNKjvzevzzsX+0VNl445PxE+x933t3bItu9rfbrl7nuSTz957HPNzb8OUa9Zr6i3iFbb/aZtruCWJcCawuNIYcIKXG1pKVJPwEEivllwj7GchvtlzcMvDfsrE8afF2ekvjzJkTYVGQTrlJcJSlQH4roH3NMSPqFhF/k5Xhlgvcy3Ls8u5W+PMetzi+dUVbjaVqaKtJ2UlRTvQ3rwHhW7r1eymf51v8A4hT2Uz/PN/8AEKYke2leKFpcG0qCh6wd15VApSlApSlApSlBzt2/vwXck+OW76azXRNc7dv78F3JPjlu+ms10TQKUpQKUpQKUpQKUpQKwb5cTaLLcJwSFGLHcf5T4HlSTr/Ss6tJnH3l3/8As+R/01VktxFVdMTvWNqIW/DrTdoLE28W+LebjIbS69KnMIeWpRG9DmHmpG9BI0APRWR9j7Fve1Z/kDX7NbOy/wAjwP8AyG/0RUOx/jtg2UZb5M26+d5elKdQ3HeiPspfU3vvA044hKHSnRJCFHQBPorrVXrkVTEVTH8rmW++x9i3vas/yBr9mn2PsW97Vn+QNfs1GHe0LgLeWN40m+KkXh2cLYhqPCkOtqk8wCmg8lstlSd7UArzQCVa0a907j5gVtyhePychabuLclMJxXcPGM3IJADK5AR3SXNkDkKwdnWt144+5058UzO9IfsfYt72rP8ga/Zp9j7Fve1Z/kDX7NRm69obh9ZLvNtk3IAzKgSxBmn2HIU1EePLyh5wNlDQPOnSlqCT10To6wxx6taOOM/h09CnIfjw4zyJjcGS4hx55S/MJS0UIQEpSe9KuUlRTsFChTj7nTnxMzvSW7We3YVDVerLCj2qRGUguCG0lpD7fOApC0pGlDROtjYOiCKs+q64h/eZc/6if001YtYOEzNVFNVWucz6LOuClKVz3kpSlApSlBzt2/vwXck+OW76azXRNc7dv78F3JPjlu+ms10TQKUpQKUpQKUpQKUpQK0mcfeXf8A+z5H/TVW7rT5gyuTiV7ZbHM45BfQkD0ktqArLa+5T3wsbWns+/aaDrx9jo1v+qK5EscHL73lPDK75DaM8m5Xb8k73IHZjDybVCStD7IEZoHuy2C4j7a2lWkBRWobrrqxLS7Y7ctB5kqjtkH1jlFZ1b1yM1SjjDHA9YcksGG5Sm5WDDLDmrtytU6ZjsxDkp9cl4x2nZfKY4Sp18kOJUedPIDyEq1mYPwthQYMjh/nWMcRrnNdvD6XJFtuE/2kmMOylOokkoeSwgALClpICuZJPKSav1js88Po+UeUCcfC7kJhuADsyQ5HTJKufvgwpwtBfMebmCN76+NWNWKKN45iyXDb3K4R9pOEix3B2XdrxNdt0dMRZcmJMGKlCmU624CpKgCnfVJHiKliHblhvaHbusywXmfa8gxy3Wtqfb4S5DceS1IeK0yCn+KHK8lXOrQ0FddirxpXrREd4h/eZc/6if001YtV5n6C7iU9tPVbndtpGvFSnEgD+8kVYdThH2qO+f8AleYpSlaCFKUoFKUoOdu39+C7knxy3fTWa6Jrnbt/fgu5J8ct301muiaBSlKBSlKBSlKBSlfhISCSdAeJNB+1Ds/4t4zwzuGNwL9NcYnZDOTbrbHZYW6t50lIPRIOkp5gSo9ADWFceIr0/iXL4fQrPfYsg2hU5eTNxEmFEUslLSQpe0rXsLOtEbRog+dy5nDDAZeDYbarVe8hmZpd4anHl3q6pSXlOuFRWUeJQnz1JSNkhJ5dkCgjGJ8McyiXzJ3rrlrVvssiepdmtlhhtgRY3o7xT6XPOUTspQAkEEjorlTJ/IC4e/O9/wCRC/d6mVK2o4Td3+UT6LmUN8gLh7873/kQv3etPmOC5Yxid4dxnLbhIyFuI6u3szmIfcOSAkltK9MJISVaBOxrdWVSnKbnZ4R7GVdYvg2TSMatTt+yy5Rb4uI0qexEbhKZbkFA7xLZMfZSFb0T6NVs/IC4e/O9/wCRC/d60PAWFjeJ2S8YRYcnm5LIxy4Oonm4qUp+K68tT3dFRSOYDnOjtX56tCnKbnZ4R7GUZtmDNxJrMqfdJ98dYPOymd3QQ0r8cIabQCr1E7I9Gqk1KVhruVXJzVJnJSlKxoUpSgUpSg527f34LuSfHLd9NZromqD7ddnlXjstZqIiO8dhpjTyj1oZktOL/wAEpUf7quzHb5GybH7ZeIaueHcIrUtlXrQ4gKSf8CKDYUpSgUpSgUpUYyLN4MC5u43b7jbHc1ft7023WaZK7tT/ACDQUrQKgjmIBIBOgogHlOg3Nxvlus70JqfPiwnZrwjRUSHktl90gkNoBPnK0CdDr0NVVPx699oPHM2xXOsduGF457ZJiwH7fdQJNyitr89S+QaS25ykcp3tC/QQFHa45wtVl9uwjIeKdpst44g2DvJDMmC2sR4jzigr7WlRPMUhLelK3pSeZOj1qzqDHt8Fm1wI0OOFhiO0llsOOKcVypAA2pRJUdDxJJPpNZFKUClKUClKUFaWuciw8e7tZIOCGHFvVnReJ2YR2z3cqU253CIzpCNBaWwVDa/AnQ8TVl1C83g5VJyzC37DkEK1WiPMdN5t8pCSq4MlvSUNkpJCkq87oU/DvwqaUClKUClKUClKUClKUGoy/G42ZYnerBM6xLrCegvdN+Y42UK/0UaqDsS5HIvvZyxqDP6XXH1P2GY2Tvu1xnVNpT/c2G/8avaucuz4fIrtDcdMGV9rjyLlHyqCn0LEtv8AhCgPUHEpTQdG0pSgUqJ8WOIULhRw2yPLrhymPaYa5AbUrXeua022D61rKUj4VCvk3wH7aeecNLheLWL1b4luyi+IulwvFxt6pS4LzjiBJkhCCFOBSEjmQeY+aCjlUTzB9Uc5ziZOt+UY9w9uNjuHEe3RW3UWy4SdJj94dJW6EgkeaCoD0+bvQUDW3xzB4cK4x8kulutb2cO25mDcLzDi92p7lG1JQSSoI5ySATvQSDvQ17cTxzF4r07JMdhWwO5F3U6TdbclBFwBQO7c7xO+dJToggkHmJ8VEmR0ClKUClKUClKUClKwr1Z4WRWafarlHRLt06O5Fkx3BtLrS0lK0n4CCR/fQVtxfi4S/n/C1eUTJsa9NXhxVgbiglt6T3R5ku6SdJ5fWR19NWtXwg4+cJJvBDizkOHzOdxEJ8mI+ofx8dfnNL8NbKSNgeCgR6K+rHYN4QzOD3Z4tUa5d43dL4+q+SY7g0WC6htKEa1sHu22yQeoUVD0UHQ9KUoFKUoFKUoPwkAbPQVDXM2utwV3tjskeZAP8XKnTlRg8PxkJS04Sk+gnW/EDRBMgydZbxq7KSSFCI8QR6DyGo7jKQnG7SEgJAiNAADQHmCt6xRRoTXVGdeOf0wvNk8qMt97ln+enf3Wq4l4PlrnH6BxOiwrPFcbsLlhmW32xdUJTZe71C+89jjlKVf0TsdNirXQ4hwqCVJUUnlUAd6PqP8AiK8qz/S6uP8Ab3M9jW+VGW+9yz/PTv7rTyoy33uWf56d/da2VYdovNvyC3M3C1zo1ygPglqVDeS604ASDyqSSD1BHQ+in0urjxq9zPYpXtTcMs77QvDtjF47lvxuAJiJUxMSSqY7LShKuVsBaGUjSiFaKupSnqNVA+F3+zM4VRokO6Xi83nMEuI2WVqEGOv0EKbSO9SoHYI7wEEEHqK6yrF4aHdjnjoAm7TwABoD+EuH/mSf76xXqKJtzXTTjExv5875ldsNriGH2XAcat+PY7bWLRZoDfdRocZPKhCdkk+skklRUdlRJJJJJrcUpXPeSlKUClKh/E7M14fYkCIU+2k1fcReYbCDranCPSEjr8JKR6ay2rVV+5FujbI9uW8S7Lh7vsaQt2ZcCnmEGEkLd161bISgermI36KhD3Hq4FR9j4s0Ub6eyLnyK18IS0of6mq5SghTi1LU666suOuuHa3FnxUo+kmvKvt7Pwfg1unFyNKe+Y/GDMLA+zzePerD+d1fu9Ps83j3qw/ndX7vVf0rY+V8C6vzq900uxCuPGBQePfEHCsqu+OQ4zlgd1LjCeXBc44UFpYWosDlSFc3XR6LWNdQRev2ebwP/CsP53V+71X9KfK+BdX51e5pdiwPs83j3qw/ndX7vX6OPN331xWJr4Luon6PVa265Q7xCamQJTE6G8OZuRGcDjaxvWwoEg+HorJp8s4FOuLfnV7ml2LcsnHO1S3UNXiDIsalEDv3Cl2Ps+grT1T+dSUj4asdp1DzaHG1pcbWApK0nYUD4EGuXKm/CXMXLFeI2PyFk2uaVJiA+Ed7RVyD+gsBWh6FAAfddOPw/wCE0UUTd4PzbY7Oxdq76UpXyY1eVfexePib36BqPY197lq+KNfoCpDlX3sXj4m9+gaj2Nfe5avijX6Aro2fsz3+i8zlrBL/AH/g7ws4z5n5QTMhXbMhvDLNsmx4yGFyvZSUJkrU22leyTtSQoI0TpI6ES/Ic/zXgRkEBrKMkGdwrnYrrcC2YDMRcaVBjh8paLSRtpaeZOl7UCE+cd6qeJ4AY/7Y5Wpc67PWTJy+u5Y45ISbe468kJddSnk50rVrewvQPUAGvzFeANkx+8e2dyu98zCW3ActcU5HKRJTFiua7xttKUJHnhKQpSuZRAAKtbrxETCIJYMzz/FJHC2+ZHlLGQ27N5LUKXaUW5phuA6/GckNKjrQOdSUFvkV3hVsHfQ1DsLyDMMc7MfC2TjsidbrJ3sn2+udotzdwmxI4W8ULbYWFBSOfXOQlSgnqB41cuH9nWwYhfLLcPbi/wB6YsKFosttu04PRbYFJ5PtSQgKJCCUJLillKToar8hdnyDZMbbsdiy/LcegMzn5sZNuuKAY6Xfuo6edtQLIPMpKVBRSVEhXhpiRN8Cu7N+wuyXFi9tZI1JiNuC7sNhpEslI24EDojZ2eX0eHorZ8M/5EuH9rTvpC61uDYXa+HWI2vG7K0tm121kMspcWVrI3sqUo+JJJJPrJrZcM/5EuH9rTvpC6yV/Yq749V5kupSlc1ClKUCqO42SFO51bWFHaGLapbY9Rcd0r/poq8aqfjpYnP+ysgaTtuLzRJWh4NuFJQs/Aladf8Aub9HTsfCa6aOF06XPmP5mP7CwrGleLhUG1FCQpYB5Uk6BPq36Kh4vWe764nY9f8AqB39zr72quKdv4mfwxplXP0niFxDymbf7hjMO6LYt9wkQYUJiHBXDfLK+Q9+468l4FRB3yAcoI0Fa62d7dZ9707F/wDIXf3OsQcIYQvcm6w7verCZ76Zk622ucERX3+nMsgo5gVaHMUlPNrqK1L0VXsRRMxv2x+YVDr3k+aTp3EqRDyA2VrGY7EqNAEJl0KWYSHltOLUCSjm5vudK84+doADZQcryXibkxt9lvKcWh2+0wp8p1uI3IdfelIUtCAHNgNpSg711JOtipo9w7tr7uYLU/KByhtLUzS0/awGAwO783oeUb87m6/B0rVSuDdsXMtsyDdrzZZsOA1bVyLbJS2qUw2NIS8CggkddKABGzojpWObV6JzmZjM519s4xu5tm7AxuzkCOCeKbOz7GVs/wDuLqx6gVnsV94c2S3Y7jFmh3e0QWeRuVdbwph8kqJIUlEZQPj47H5qyvbrPvenYv8A5C7+51sWqot26aKonMREbJ9hM69EyQqGliU3/Gx5DL7Z/pJcSof6isLHpV5lxXFXq2w7ZIC9IbhzVSkqTodSpTTejvfTR8PGpLi1icyjLLXbkJKmUPImSla6JZbUFaP9ZQSn+8+o1lruU0W5uVbIhadrpWlKV+WKx7hDTcIEmKskIfaU0oj0BQIP/Oq+hXwYxBjWy7xZzEuK2lkusQnn2XgkABaFtoKdHW+U6I8CKsmlbNq9FuJpqjMeC5V55d2n1XD5rlfV08u7T6rh81yvq6sOlZuUWuhPj+jUrzy7tPquHzXK+rp5d2n1XD5rlfV1YdKcotdCfH9GpXyM0gyPMjRrnLfP3LLdtkBSj6tqQEj86iAPSRUlw2zP2OxhmVyCW8+9KeS2dpQp1xTnKDob5eYJ3rrrfpreUrFcvRXToUxiO/PpBncUpStVClKUCvXJjNTI7rD7SH2HUlDjTiQpK0kaIIPQgj0V7KU2Cj8r4RXWxvresTRu1sJJTFLoElgfigq0HEj1lQVrQ887NQp1ibGUUv2i7MLHily2vj/A8mj/AHbrqWlfRWfjd63To3KYq7dkrqna5X2/7n3L5vf/AGKbf9z7l83v/sV1RStj59V1fn+kxDlfb/ufcvm9/wDYpt/3PuXze/8AsV1RSnz6rq/P9GIcr7f9z7l83v8A7FB7IUdC3XMn1C3Pkn/9K6opT57V1fn+jEOcbLhGSZE6lMW0PQWSRzSrohUdCR6dII7wn4OUD1keNXbhWFQsKtqmI6jIlPELkzHAAt5Xo/MkbISkdB8JJJkNK5XC/iV7hcaE6qd0eq9xSlK5SP/Z", + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "from IPython.display import Image, display\n", + "\n", + "try:\n", + " display(Image(graph.get_graph().draw_mermaid_png()))\n", + "except Exception:\n", + " # This requires some extra dependencies and is optional\n", + " pass" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "id": "bee04c3d-0e36-4443-b0c8-10986a5f6e39", + "metadata": {}, + "outputs": [], + "source": [ + "user_input = \"Can you give me some information about AMD in 2022?\"\n", + "\n", + "result = graph.invoke({\"messages\": [(\"user\", user_input)]})" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "id": "92ba9195-52d4-46c3-b811-8e00a9d61480", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "================================\u001b[1m Human Message \u001b[0m=================================\n", + "\n", + "Can you give me some information about AMD in 2022?\n", + "==================================\u001b[1m Ai Message \u001b[0m==================================\n", + "Tool Calls:\n", + " Accenture (call_ytesTjST6vxetxsQQmVKDjyJ)\n", + " Call ID: call_ytesTjST6vxetxsQQmVKDjyJ\n", + " Args:\n", + " year: 2022\n", + "=================================\u001b[1m Tool Message \u001b[0m=================================\n", + "Name: Accenture\n", + "\n", + "Accenture had revenues of $100 in 2022.\n", + "==================================\u001b[1m Ai Message \u001b[0m==================================\n", + "Tool Calls:\n", + " Advanced_Micro_Devices (call_16WW5BuJtX0uLylLcawxrDcI)\n", + " Call ID: call_16WW5BuJtX0uLylLcawxrDcI\n", + " Args:\n", + " year: 2022\n", + "=================================\u001b[1m Tool Message \u001b[0m=================================\n", + "Name: Advanced_Micro_Devices\n", + "\n", + "Advanced Micro Devices had revenues of $100 in 2022.\n", + "==================================\u001b[1m Ai Message \u001b[0m==================================\n", + "\n", + "In 2022, AMD had revenues of $100 and Accenture had revenues of $100.\n" + ] + } + ], + "source": [ + "for message in result[\"messages\"]:\n", + " message.pretty_print()" + ] + }, { "cell_type": "markdown", "id": "177aedfa-cec5-45d0-82ad-efc0233aa6b4", @@ -290,9 +455,7 @@ "\n", "This guide provides a minimal implementation for dynamically selecting tools. There is a host of possible improvements and optimizations:\n", "\n", - "- **Repeating tool selection**: To manage errors from incorrect tool selection, we could revisit the `select_tools` node. Options include:\n", - " - Modify `select_tools` to generate the vector store query using all messages in the state (e.g., with a chat model) and add an edge routing from `tools` to `select_tools`;\n", - " - Equip the agent with a `reselect_tools` tool, allowing it to re-select tools at its discretion.\n", + "- **Repeating tool selection**: Here, we repeated tool selection by modifying the `select_tools` node. Another option is to equip the agent with a `reselect_tools` tool, allowing it to re-select tools at its discretion.\n", "- **Optimizing tool selection**: In general, the full scope of [retrieval solutions](https://python.langchain.com/v0.2/docs/concepts/#retrieval) are available for tool selection. Additional options include:\n", " - Group tools and retrieve over groups;\n", " - Use a chat model to select tools or groups of tool." From f124ce2f7295ffe503ce34f2d9c109fb197ca6e1 Mon Sep 17 00:00:00 2001 From: Chester Curme Date: Mon, 5 Aug 2024 13:28:06 -0400 Subject: [PATCH 4/6] move file --- examples/{agent_executor/many_tools.ipynb => many-tools.ipynb} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename examples/{agent_executor/many_tools.ipynb => many-tools.ipynb} (100%) diff --git a/examples/agent_executor/many_tools.ipynb b/examples/many-tools.ipynb similarity index 100% rename from examples/agent_executor/many_tools.ipynb rename to examples/many-tools.ipynb From ad31da06d128a6f2bd1193d8829ba8787d06ef2f Mon Sep 17 00:00:00 2001 From: Chester Curme Date: Mon, 5 Aug 2024 13:34:30 -0400 Subject: [PATCH 5/6] update script + index + mkdocs --- docs/_scripts/copy_notebooks.py | 1 + docs/docs/how-tos/index.md | 1 + docs/mkdocs.yml | 1 + 3 files changed, 3 insertions(+) diff --git a/docs/_scripts/copy_notebooks.py b/docs/_scripts/copy_notebooks.py index b0f06cb67..00c0f1464 100644 --- a/docs/_scripts/copy_notebooks.py +++ b/docs/_scripts/copy_notebooks.py @@ -43,6 +43,7 @@ _MANUAL = { "tool-calling.ipynb", "tool-calling-errors.ipynb", "pass-config-to-tools.ipynb", + "many-tools.ipynb", "dynamic-returning-direct.ipynb", "managing-agent-steps.ipynb", "respond-in-format.ipynb", diff --git a/docs/docs/how-tos/index.md b/docs/docs/how-tos/index.md index b5cedfc97..f301ff934 100644 --- a/docs/docs/how-tos/index.md +++ b/docs/docs/how-tos/index.md @@ -60,6 +60,7 @@ These guides show how to use different streaming modes. - [How to handle tool calling errors](tool-calling-errors.ipynb) - [How to pass graph state to tools](pass-run-time-values-to-tools.ipynb) - [How to pass config to tools](pass-config-to-tools.ipynb) +- [How to handle large numbers of tools](many-tools.ipynb) ## State Management diff --git a/docs/mkdocs.yml b/docs/mkdocs.yml index 7ea4ae6f4..a55211bcf 100644 --- a/docs/mkdocs.yml +++ b/docs/mkdocs.yml @@ -157,6 +157,7 @@ nav: - Handle tool calling errors: how-tos/tool-calling-errors.ipynb - Pass graph state to tools: how-tos/pass-run-time-values-to-tools.ipynb - Pass config to tools: how-tos/pass-config-to-tools.ipynb + - Handle large numbers of tools: how-tos/many-tools.ipynb - State Management: - Use Pydantic model as state: how-tos/state-model.ipynb - Use a context object in state: how-tos/state-context-key.ipynb From 14d448b74d5ea38e8da93310554eb46e980d96a2 Mon Sep 17 00:00:00 2001 From: Chester Curme Date: Mon, 5 Aug 2024 13:45:47 -0400 Subject: [PATCH 6/6] remove usage of upsert --- examples/many-tools.ipynb | 25 ++++++++++++------------- 1 file changed, 12 insertions(+), 13 deletions(-) diff --git a/examples/many-tools.ipynb b/examples/many-tools.ipynb index 6bd9272d2..b415f5065 100644 --- a/examples/many-tools.ipynb +++ b/examples/many-tools.ipynb @@ -119,8 +119,7 @@ "]\n", "\n", "vector_store = InMemoryVectorStore(embedding=OpenAIEmbeddings())\n", - "upsert_response = vector_store.upsert(tool_documents)\n", - "assert not upsert_response[\"failed\"]" + "document_ids = vector_store.add_documents(tool_documents)" ] }, { @@ -243,7 +242,7 @@ "name": "stdout", "output_type": "stream", "text": [ - "['ed32d6f0-76c8-42ad-9ddb-eb28ab73cba3', 'ab4b0a2d-e535-419d-8d8b-d1e1cc6a410b', 'da70a795-fd64-474d-be7f-d39314255f6c', '3f605f81-bcd0-4bae-848b-fcae0b2115be']\n" + "['3b7d1528-6007-4473-a92f-b9b3341c3bfe', '8d77b753-c58a-41bf-9649-ad1a7326bc27', '514a6fc3-03d1-4e73-b410-c39309ad7b2f', '83c5cc8f-5111-46ed-874a-e0b883265ff6']\n" ] } ], @@ -266,8 +265,8 @@ "Can you give me some information about AMD in 2022?\n", "==================================\u001b[1m Ai Message \u001b[0m==================================\n", "Tool Calls:\n", - " Advanced_Micro_Devices (call_RhGe3Zhp9ENphSC8Pusd7ZWe)\n", - " Call ID: call_RhGe3Zhp9ENphSC8Pusd7ZWe\n", + " Advanced_Micro_Devices (call_Htbv7Imx4BwSsYWhZvSSs6yW)\n", + " Call ID: call_Htbv7Imx4BwSsYWhZvSSs6yW\n", " Args:\n", " year: 2022\n", "=================================\u001b[1m Tool Message \u001b[0m=================================\n", @@ -392,7 +391,7 @@ }, { "cell_type": "code", - "execution_count": 14, + "execution_count": 18, "id": "bee04c3d-0e36-4443-b0c8-10986a5f6e39", "metadata": {}, "outputs": [], @@ -404,8 +403,8 @@ }, { "cell_type": "code", - "execution_count": 15, - "id": "92ba9195-52d4-46c3-b811-8e00a9d61480", + "execution_count": 19, + "id": "6906fb50-435c-4473-bbb6-5353433b9199", "metadata": {}, "outputs": [ { @@ -417,8 +416,8 @@ "Can you give me some information about AMD in 2022?\n", "==================================\u001b[1m Ai Message \u001b[0m==================================\n", "Tool Calls:\n", - " Accenture (call_ytesTjST6vxetxsQQmVKDjyJ)\n", - " Call ID: call_ytesTjST6vxetxsQQmVKDjyJ\n", + " Accenture (call_L82JRUyIFilhzeTmPnNbPeVD)\n", + " Call ID: call_L82JRUyIFilhzeTmPnNbPeVD\n", " Args:\n", " year: 2022\n", "=================================\u001b[1m Tool Message \u001b[0m=================================\n", @@ -427,8 +426,8 @@ "Accenture had revenues of $100 in 2022.\n", "==================================\u001b[1m Ai Message \u001b[0m==================================\n", "Tool Calls:\n", - " Advanced_Micro_Devices (call_16WW5BuJtX0uLylLcawxrDcI)\n", - " Call ID: call_16WW5BuJtX0uLylLcawxrDcI\n", + " Advanced_Micro_Devices (call_k3zR9zS98gjiejmNgq6aVsXL)\n", + " Call ID: call_k3zR9zS98gjiejmNgq6aVsXL\n", " Args:\n", " year: 2022\n", "=================================\u001b[1m Tool Message \u001b[0m=================================\n", @@ -437,7 +436,7 @@ "Advanced Micro Devices had revenues of $100 in 2022.\n", "==================================\u001b[1m Ai Message \u001b[0m==================================\n", "\n", - "In 2022, AMD had revenues of $100 and Accenture had revenues of $100.\n" + "In 2022, Advanced Micro Devices (AMD) had revenues of $100.\n" ] } ],