{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "## Initialization\n", "\n", "### Initializing client\n", "\n", "To get started we need to initialize our client. The process for initializing our client is almost identical for both the local deployment and cloud deployment using Langsmith." ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "from langgraph_sdk import get_client\n", "\n", "# If you deployed using Langsmith use this option\n", "# Find this url on your Langsmith deployment page\n", "example_deployed_url = (\n", " \"https://ht-unhealthy-buffalo25-39d00f953458585aa9f7b5a4fa-g3ps4aazkq-uc.a.run.app\"\n", ")\n", "\n", "# If you deployed locally using langgraph up -c langgraph.json use this option\n", "# This is the default URL, and you can just call get_client() to use it\n", "example_local_url = \"http://localhost:8123\"\n", "\n", "client = get_client(url=\"whatever-your-url-is\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Selecting an Assistant\n", "\n", "To select an assistant we can search the assistants that are hosted on our client, and then select the one we want," ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [], "source": [ "assistants = await client.assistants.search()\n", "assistants = [a for a in assistants if not a[\"config\"]]\n", "assistant = assistants[0]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In our example we are only hosting a single assistant, but you have the option to host many, in which case you will most likely want to do more filtering than just selecting the first one. Each assistant is a JSON object with the following format, allowing you to select based on a variety of parameters." ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'assistant_id': 'fe096781-5601-53d2-b2f6-0d3403f7e9ca',\n", " 'graph_id': 'agent',\n", " 'created_at': '2024-06-11T20:12:45.862108+00:00',\n", " 'updated_at': '2024-06-11T20:12:45.862108+00:00',\n", " 'config': {},\n", " 'metadata': {'created_by': 'system'}}" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "assistant" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Creating a thread\n", "\n", "Threads are what we will actually use to run our graphs (assistants). Each thread will update the same state for the graph, meaning we can run the graph multiple times while the state will persist. We can also look back at our thread history, add meta data to different steps of our thread, and update the thread state manually if we wish. We will dive into all of those topics later in this article, but for now let’s just see how to start a thread:" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "thread = await client.threads.create()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can examine the structure of our thread, which similar to the assistants object provides us with some information about the thread itself, including its id, timestamps, and metadata:" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'thread_id': '6c2e8002-5712-4388-bed6-0747e9a86e31',\n", " 'created_at': '2024-06-19T15:58:59.243657+00:00',\n", " 'updated_at': '2024-06-19T15:58:59.243657+00:00',\n", " 'metadata': {}}" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "thread" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now we are ready to actually use our graph!\n", "\n", "## Invoking the graph\n", "\n", "The graph used in this example is a simple example of a StateGraph, but it allows us to show most of the API functionality. The state of our graph is defined as follows:" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [], "source": [ "from typing import Annotated, TypedDict\n", "\n", "from langchain_core.messages import AnyMessage\n", "\n", "from langgraph.graph import add_messages\n", "\n", "\n", "def update_user_info(old_info, new_info):\n", " if \"name\" not in new_info or new_info[\"age\"] == -1:\n", " return old_info\n", " return new_info\n", "\n", "\n", "class UserInformation(TypedDict):\n", " age: int\n", " name: str\n", "\n", "\n", "class AgentState(TypedDict):\n", " messages: Annotated[list[AnyMessage], add_messages]\n", " user_info: Annotated[UserInformation, update_user_info]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "It is important to note that our member variables can be updated by using the `Annotated` class. This is especially important when we make API calls that will update our variables. This is also a good example that your graph can hold much more information than just messages. In our example we use a very simple `UserInformation` class, but you can imagine holding much richer information in your state.\n", "\n", "Our graph looks like follows:\n", "\n", "
\n", " \n", "
\n", "\n", "The workflow is as follows: first the user inputs some message, our llm decides how to configure the call to our tool `get_user_info` , and after getting the results of the tool call we respond to our user using another LLM.\n", "\n", "### Simple Invocation\n", "\n", "Ok, now that we have set up our client, assistant, and thread we can actually invoke the graph above. Let’s first define the function we will use to invoke the graph, since we don’t want to have to rewrite this code every single run." ] }, { "cell_type": "code", "execution_count": 48, "metadata": {}, "outputs": [], "source": [ "async def run_input(client, thread, assistant, input, metadata={}):\n", " # client.runs.stream will stream the results of running our graph\n", " async for chunk in client.runs.stream(\n", " thread[\"thread_id\"],\n", " assistant[\"assistant_id\"],\n", " input=input,\n", " config={\"configurable\": metadata},\n", " stream_mode=\"updates\",\n", " ):\n", " if chunk.data and \"run_id\" not in chunk.data:\n", " print(chunk.data)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let’s now see what happens to our graph when we run it with a simple sentence:" ] }, { "cell_type": "code", "execution_count": 28, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{'llm': {'messages': [{'content': '', 'additional_kwargs': {'tool_calls': [{'index': 0, 'id': 'call_ioeUDw39bXQ2nap5f593xZMW', 'function': {'arguments': '{\"age\":26,\"name\":\"Bagatur\"}', 'name': 'PersonalInfo'}, 'type': 'function'}]}, 'response_metadata': {'finish_reason': 'stop'}, 'type': 'ai', 'name': None, 'id': 'run-75a0d9db-df99-4b0c-b356-f24e76f5ca5a', 'example': False, 'tool_calls': [{'name': 'PersonalInfo', 'args': {'age': 26, 'name': 'Bagatur'}, 'id': 'call_ioeUDw39bXQ2nap5f593xZMW'}], 'invalid_tool_calls': [], 'usage_metadata': None}]}}\n", "{'get_user_info': {'messages': [{'content': 'Hello! My name is Bagatur and I am 26 years old.', 'additional_kwargs': {}, 'response_metadata': {}, 'type': 'human', 'name': None, 'id': 'abafa365-1280-439d-b2ac-d4c547284ff9', 'example': False}, {'content': '', 'additional_kwargs': {'tool_calls': [{'index': 0, 'id': 'call_E6NK18NXOHFkp8CeIuF7iI3K', 'function': {'arguments': '{\"age\":26,\"name\":\"Bagatur\"}', 'name': 'PersonalInfo'}, 'type': 'function'}]}, 'response_metadata': {'finish_reason': 'stop'}, 'type': 'ai', 'name': None, 'id': 'run-c018c6a2-b93a-4787-aae3-46df0ef24c5b', 'example': False, 'tool_calls': [{'name': 'PersonalInfo', 'args': {'age': 26, 'name': 'Bagatur'}, 'id': 'call_E6NK18NXOHFkp8CeIuF7iI3K'}], 'invalid_tool_calls': [], 'usage_metadata': None}, {'content': 'Hello Bagatur! How can I assist you today?', 'additional_kwargs': {}, 'response_metadata': {'finish_reason': 'stop'}, 'type': 'ai', 'name': None, 'id': 'run-fb1ac0ca-b1f8-454b-ba3f-849c266dcc05', 'example': False, 'tool_calls': [], 'invalid_tool_calls': [], 'usage_metadata': None}, {'content': 'Hello! What is my name?', 'additional_kwargs': {}, 'response_metadata': {}, 'type': 'human', 'name': None, 'id': '8bf0637b-8214-4c85-85ff-873ec193a7b8', 'example': False}, {'content': '', 'additional_kwargs': {'tool_calls': [{'index': 0, 'id': 'call_6EFvGDGWHuoOKm4p7dq3qazN', 'function': {'arguments': '{\"age\":-1,\"name\":\"John Doe\"}', 'name': 'PersonalInfo'}, 'type': 'function'}]}, 'response_metadata': {'finish_reason': 'stop'}, 'type': 'ai', 'name': None, 'id': 'run-30a91dfa-4526-4bb7-b4a2-017e5c530ed2', 'example': False, 'tool_calls': [{'name': 'PersonalInfo', 'args': {'age': -1, 'name': 'John Doe'}, 'id': 'call_6EFvGDGWHuoOKm4p7dq3qazN'}], 'invalid_tool_calls': [], 'usage_metadata': None}, {'content': 'Your name is Bagatur. How can I assist you today?', 'additional_kwargs': {}, 'response_metadata': {'finish_reason': 'stop'}, 'type': 'ai', 'name': None, 'id': 'run-f3dcd3ad-b438-4011-9353-83978758ee81', 'example': False, 'tool_calls': [], 'invalid_tool_calls': [], 'usage_metadata': None}, {'content': 'Hello! My name is Bagatur and I am 26 years old.', 'additional_kwargs': {}, 'response_metadata': {}, 'type': 'human', 'name': None, 'id': '88e7bb12-552d-4607-8bc5-ce2b342f0604', 'example': False}, {'content': '', 'additional_kwargs': {'tool_calls': [{'index': 0, 'id': 'call_ioeUDw39bXQ2nap5f593xZMW', 'function': {'arguments': '{\"age\":26,\"name\":\"Bagatur\"}', 'name': 'PersonalInfo'}, 'type': 'function'}]}, 'response_metadata': {'finish_reason': 'stop'}, 'type': 'ai', 'name': None, 'id': 'run-75a0d9db-df99-4b0c-b356-f24e76f5ca5a', 'example': False, 'tool_calls': [{'name': 'PersonalInfo', 'args': {'age': 26, 'name': 'Bagatur'}, 'id': 'call_ioeUDw39bXQ2nap5f593xZMW'}], 'invalid_tool_calls': [], 'usage_metadata': None}], 'user_info': {'age': 26, 'name': 'Bagatur'}}}\n", "{'respond_to_user': {'messages': [{'content': 'Hello Bagatur! How can I assist you today?', 'additional_kwargs': {}, 'response_metadata': {'finish_reason': 'stop'}, 'type': 'ai', 'name': None, 'id': 'run-49176b69-aae5-41ac-8c1e-7bab373b2b9b', 'example': False, 'tool_calls': [], 'invalid_tool_calls': [], 'usage_metadata': None}]}}\n" ] } ], "source": [ "input = {\n", " \"messages\": [\n", " {\"role\": \"user\", \"content\": \"Hello! My name is Bagatur and I am 26 years old.\"}\n", " ]\n", "}\n", "\n", "await run_input(client, thread, assistant, input)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can see our graph ran as expected, by calling our three nodes sequentially. Let’s now examine the state to take a further look under the hood. We can get the state by using the following command: " ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "dict_keys(['values', 'next', 'config', 'metadata', 'created_at', 'parent_config'])" ] }, "execution_count": 17, "metadata": {}, "output_type": "execute_result" } ], "source": [ "state = await client.threads.get_state(thread_id=thread[\"thread_id\"])\n", "state.keys()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Our state variable contains a variety of important information. Here is a quick summary of the keys and what they represent:\n", "\n", "- `values` contains the actual state values, so in our case you could call `state['values']['messages']` or `state['values']['user_info']` and get the actual values of each of the state variables.\n", "- `next` tells us what action in the graph is next at the current state. Since we just finished running our graph and reached the end node, it is currently empty because there is no next action to take. However, if you go through the state at each point of the run you will see that the `next` value goes from `__start__` → `llm` → `get_user_info` →`respond_to_user` .\n", "- `metadata` stores the metadata associated with our state. This is data that is outside of the agent state, but is important to keep track of across multiple runs. An example of this is shown in the next section.\n", "- `config` tells us what the configuration of the state is. This is important for when we want to run a query starting at a previous state instead of the one we are at. An example of this is shown in the Invoking from a previous checkpoint section\n", "\n", "### Invoking with Metadata\n", "\n", "Let’s create a new thread to reset our state and start fresh." ] }, { "cell_type": "code", "execution_count": 49, "metadata": {}, "outputs": [], "source": [ "thread = await client.threads.create()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now let’s add some metadata to our request. In this example we are going to treat each run of our assistant as a separate “node”. For each run, we will pass in a “node_id” as well as a “parent_node” in the metadata. This way we can easily go “back in time” and rerun our graph from a previous checkpoint. \n", "\n", "> NOTE: The reason we add this metadata instead of using the `parent_config` attribute is because `parent_config` tracks every individual step of a run, not the entire run itself.\n", ">" ] }, { "cell_type": "code", "execution_count": 50, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{'llm': {'messages': [{'content': '', 'additional_kwargs': {'tool_calls': [{'index': 0, 'id': 'call_VrA7UKg2w99BvIpsqrwjoawk', 'function': {'arguments': '{\"age\":26,\"name\":\"Bagatur\"}', 'name': 'PersonalInfo'}, 'type': 'function'}]}, 'response_metadata': {'finish_reason': 'stop'}, 'type': 'ai', 'name': None, 'id': 'run-067609c1-4f1a-4d6d-bc1c-224a29153d37', 'example': False, 'tool_calls': [{'name': 'PersonalInfo', 'args': {'age': 26, 'name': 'Bagatur'}, 'id': 'call_VrA7UKg2w99BvIpsqrwjoawk'}], 'invalid_tool_calls': [], 'usage_metadata': None}]}}\n", "{'get_user_info': {'messages': [{'content': 'Hello! My name is Bagatur and I am 26 years old.', 'additional_kwargs': {}, 'response_metadata': {}, 'type': 'human', 'name': None, 'id': '05faf88c-9f85-462e-84ee-4667de35625d', 'example': False}, {'content': '', 'additional_kwargs': {'tool_calls': [{'index': 0, 'id': 'call_VrA7UKg2w99BvIpsqrwjoawk', 'function': {'arguments': '{\"age\":26,\"name\":\"Bagatur\"}', 'name': 'PersonalInfo'}, 'type': 'function'}]}, 'response_metadata': {'finish_reason': 'stop'}, 'type': 'ai', 'name': None, 'id': 'run-067609c1-4f1a-4d6d-bc1c-224a29153d37', 'example': False, 'tool_calls': [{'name': 'PersonalInfo', 'args': {'age': 26, 'name': 'Bagatur'}, 'id': 'call_VrA7UKg2w99BvIpsqrwjoawk'}], 'invalid_tool_calls': [], 'usage_metadata': None}], 'user_info': {'age': 26, 'name': 'Bagatur'}}}\n", "{'respond_to_user': {'messages': [{'content': 'Hello Bagatur! How can I assist you today?', 'additional_kwargs': {}, 'response_metadata': {'finish_reason': 'stop'}, 'type': 'ai', 'name': None, 'id': 'run-48041247-fe04-419f-9f12-511e28c5f8aa', 'example': False, 'tool_calls': [], 'invalid_tool_calls': [], 'usage_metadata': None}]}}\n" ] } ], "source": [ "input = {\n", " \"messages\": [\n", " {\"role\": \"user\", \"content\": \"Hello! My name is Bagatur and I am 26 years old.\"}\n", " ]\n", "}\n", "\n", "metadata = {\"node_id\": 1, \"parent_node\": None}\n", "\n", "await run_input(client, thread, assistant, input, metadata)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Using our `run_input` function makes it easy to pass in metadata and you can inspect the function as well as the API docs to see exactly how metadata gets passed.\n", "\n", "We can continue our thread by creating a second node as follows:" ] }, { "cell_type": "code", "execution_count": 51, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{'llm': {'messages': [{'content': '', 'additional_kwargs': {'tool_calls': [{'index': 0, 'id': 'call_b2T2e0doVrh6uB1aHLIQglYR', 'function': {'arguments': '{\"age\":-1,\"name\":\"John Doe\"}', 'name': 'PersonalInfo'}, 'type': 'function'}]}, 'response_metadata': {'finish_reason': 'stop'}, 'type': 'ai', 'name': None, 'id': 'run-808dca3a-0188-4c6b-9fc7-0037949fd820', 'example': False, 'tool_calls': [{'name': 'PersonalInfo', 'args': {'age': -1, 'name': 'John Doe'}, 'id': 'call_b2T2e0doVrh6uB1aHLIQglYR'}], 'invalid_tool_calls': [], 'usage_metadata': None}]}}\n", "{'get_user_info': {'messages': [{'content': 'Hello! My name is Bagatur and I am 26 years old.', 'additional_kwargs': {}, 'response_metadata': {}, 'type': 'human', 'name': None, 'id': '05faf88c-9f85-462e-84ee-4667de35625d', 'example': False}, {'content': '', 'additional_kwargs': {'tool_calls': [{'index': 0, 'id': 'call_VrA7UKg2w99BvIpsqrwjoawk', 'function': {'arguments': '{\"age\":26,\"name\":\"Bagatur\"}', 'name': 'PersonalInfo'}, 'type': 'function'}]}, 'response_metadata': {'finish_reason': 'stop'}, 'type': 'ai', 'name': None, 'id': 'run-067609c1-4f1a-4d6d-bc1c-224a29153d37', 'example': False, 'tool_calls': [{'name': 'PersonalInfo', 'args': {'age': 26, 'name': 'Bagatur'}, 'id': 'call_VrA7UKg2w99BvIpsqrwjoawk'}], 'invalid_tool_calls': [], 'usage_metadata': None}, {'content': 'Hello Bagatur! How can I assist you today?', 'additional_kwargs': {}, 'response_metadata': {'finish_reason': 'stop'}, 'type': 'ai', 'name': None, 'id': 'run-48041247-fe04-419f-9f12-511e28c5f8aa', 'example': False, 'tool_calls': [], 'invalid_tool_calls': [], 'usage_metadata': None}, {'content': 'Hello! What is my name?', 'additional_kwargs': {}, 'response_metadata': {}, 'type': 'human', 'name': None, 'id': '0970d6dd-e071-49bf-bf71-5345185b8291', 'example': False}, {'content': '', 'additional_kwargs': {'tool_calls': [{'index': 0, 'id': 'call_b2T2e0doVrh6uB1aHLIQglYR', 'function': {'arguments': '{\"age\":-1,\"name\":\"John Doe\"}', 'name': 'PersonalInfo'}, 'type': 'function'}]}, 'response_metadata': {'finish_reason': 'stop'}, 'type': 'ai', 'name': None, 'id': 'run-808dca3a-0188-4c6b-9fc7-0037949fd820', 'example': False, 'tool_calls': [{'name': 'PersonalInfo', 'args': {'age': -1, 'name': 'John Doe'}, 'id': 'call_b2T2e0doVrh6uB1aHLIQglYR'}], 'invalid_tool_calls': [], 'usage_metadata': None}], 'user_info': {'age': -1, 'name': 'John Doe'}}}\n", "{'respond_to_user': {'messages': [{'content': 'Your name is Bagatur. How can I assist you today, Bagatur?', 'additional_kwargs': {}, 'response_metadata': {'finish_reason': 'stop'}, 'type': 'ai', 'name': None, 'id': 'run-9ada26f0-951e-4dd5-a2c5-4357bdeeac9d', 'example': False, 'tool_calls': [], 'invalid_tool_calls': [], 'usage_metadata': None}]}}\n" ] } ], "source": [ "input = {\"messages\": [{\"role\": \"user\", \"content\": \"Hello! What is my name?\"}]}\n", "metadata = {\"node_id\": 2, \"parent_node\": 1}\n", "\n", "await run_input(client, thread, assistant, input, metadata)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Perfect! The state persisted across separate runs, and the LLM remembers the name of our user. In a future we will explore non-sequential runs, i.e. not having each run just follow the last one but choosing which checkpoint we start our run from.\n", "\n", "## Querying and Updating the thread\n", "\n", "### Getting checkpoints by metadata\n", "\n", "Let’s say we want to start a new run from a previous state (not the current state). This state lives somewhere in our history, so we can utilize the `get_history` function to try and find it." ] }, { "cell_type": "code", "execution_count": 52, "metadata": {}, "outputs": [], "source": [ "history = await client.threads.get_history(thread_id=thread[\"thread_id\"])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This is helpful for inspecting the specifics of our current thread, but remember that the history contains all the intermediate steps a graph takes. In our case, where the graph has 5 nodes (remember that Start and End both count as nodes), our history array grows quickly. Luckily, there is a way to query by using metadata. For example if we wanted to start a run from Node 1(from the example from above) we need to find the state from the end of run with metadata node_id:1 , which we can do like so:" ] }, { "cell_type": "code", "execution_count": 53, "metadata": {}, "outputs": [], "source": [ "node_1_history = await client.threads.get_history(\n", " thread_id=thread[\"thread_id\"], metadata={\"node_id\": 1}\n", ")\n", "# At the end of the run there will be no 'next' for the graph to execute\n", "node_1_end_of_run = [h for h in node_1_history if h[\"next\"] == []][0]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now let’s explore how we could use this information to create a new branch in our thread.\n", "\n", "### Invoking from a previous checkpoint\n", "\n", "The following diagram describes what we would like to happen:\n", "\n", "
\n", " \n", "
\n", "Basically, we want to have 3 runs of our graph, but instead of having them sequentially - we want both the second and third run to originate from the same state. We can do this by utilizing the code we used above, and passing additional metadata to our run." ] }, { "cell_type": "code", "execution_count": 54, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{'llm': {'messages': [{'content': '', 'additional_kwargs': {'tool_calls': [{'index': 0, 'id': 'call_yC7HUAQfcLzheepMD8SnAojR', 'function': {'arguments': '{\"age\":-1}', 'name': 'PersonalInfo'}, 'type': 'function'}]}, 'response_metadata': {'finish_reason': 'stop'}, 'type': 'ai', 'name': None, 'id': 'run-b6a653e2-11ea-4f82-aa6a-ef321deed9ce', 'example': False, 'tool_calls': [{'name': 'PersonalInfo', 'args': {'age': -1}, 'id': 'call_yC7HUAQfcLzheepMD8SnAojR'}], 'invalid_tool_calls': [], 'usage_metadata': None}]}}\n", "{'get_user_info': {'messages': [{'content': 'Hello! My name is Bagatur and I am 26 years old.', 'additional_kwargs': {}, 'response_metadata': {}, 'type': 'human', 'name': None, 'id': '05faf88c-9f85-462e-84ee-4667de35625d', 'example': False}, {'content': '', 'additional_kwargs': {'tool_calls': [{'index': 0, 'id': 'call_VrA7UKg2w99BvIpsqrwjoawk', 'function': {'arguments': '{\"age\":26,\"name\":\"Bagatur\"}', 'name': 'PersonalInfo'}, 'type': 'function'}]}, 'response_metadata': {'finish_reason': 'stop'}, 'type': 'ai', 'name': None, 'id': 'run-067609c1-4f1a-4d6d-bc1c-224a29153d37', 'example': False, 'tool_calls': [{'name': 'PersonalInfo', 'args': {'age': 26, 'name': 'Bagatur'}, 'id': 'call_VrA7UKg2w99BvIpsqrwjoawk'}], 'invalid_tool_calls': [], 'usage_metadata': None}, {'content': 'Hello Bagatur! How can I assist you today?', 'additional_kwargs': {}, 'response_metadata': {'finish_reason': 'stop'}, 'type': 'ai', 'name': None, 'id': 'run-48041247-fe04-419f-9f12-511e28c5f8aa', 'example': False, 'tool_calls': [], 'invalid_tool_calls': [], 'usage_metadata': None}, {'content': 'Hello! What is my age?', 'additional_kwargs': {}, 'response_metadata': {}, 'type': 'human', 'name': None, 'id': 'e94fa535-f821-4bfb-9d96-68c414adad8d', 'example': False}, {'content': '', 'additional_kwargs': {'tool_calls': [{'index': 0, 'id': 'call_yC7HUAQfcLzheepMD8SnAojR', 'function': {'arguments': '{\"age\":-1}', 'name': 'PersonalInfo'}, 'type': 'function'}]}, 'response_metadata': {'finish_reason': 'stop'}, 'type': 'ai', 'name': None, 'id': 'run-b6a653e2-11ea-4f82-aa6a-ef321deed9ce', 'example': False, 'tool_calls': [{'name': 'PersonalInfo', 'args': {'age': -1}, 'id': 'call_yC7HUAQfcLzheepMD8SnAojR'}], 'invalid_tool_calls': [], 'usage_metadata': None}], 'user_info': {'age': -1}}}\n", "{'respond_to_user': {'messages': [{'content': 'You are 26 years old, Bagatur! How can I assist you today?', 'additional_kwargs': {}, 'response_metadata': {'finish_reason': 'stop'}, 'type': 'ai', 'name': None, 'id': 'run-e3b6e627-b4b1-4249-a1bc-a6359ad7b961', 'example': False, 'tool_calls': [], 'invalid_tool_calls': [], 'usage_metadata': None}]}}\n" ] } ], "source": [ "input = {\"messages\": [{\"role\": \"user\", \"content\": \"Hello! What is my age?\"}]}\n", "metadata = {\n", " **{\"node_id\": 3, \"parent_node\": 1},\n", " **{\n", " \"thread_ts\": node_1_end_of_run[\"checkpoint_id\"],\n", " \"thread_id\": thread[\"thread_id\"],\n", " },\n", "}\n", "\n", "await run_input(client, thread, assistant, input, metadata)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To check that everything actually worked as planned, let’s check our current state and check that message history to ensure that the message we passed to Node 2 is nowhere to be found." ] }, { "cell_type": "code", "execution_count": 55, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "False" ] }, "execution_count": 55, "metadata": {}, "output_type": "execute_result" } ], "source": [ "\"Hello! What is my name?\" in [\n", " message[\"content\"] for message in state[\"values\"][\"messages\"]\n", "]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Great! This has worked as expected. Being able to go back to previous states and execute new graph runs from those checkpoints is a great way to develop flexible applications that don’t require reloading or restarting everything when an error is detected or a user changes their mind.\n", "\n", "### Updating/Patching the thread state\n", "\n", "Lastly, let’s discuss the ability to manually change both the thread state as well as the metadata for a given state. Let’s say we incorrectly inputted data to the LLM and we want to rectify it. \n", "\n", "Continuing our previous example, let’s say the user mistyped their age and we want to let the graph know that without actually running it. In this case we can rectify this by using `update_state`" ] }, { "cell_type": "code", "execution_count": 56, "metadata": {}, "outputs": [], "source": [ "new_state = await client.threads.update_state(\n", " thread_id=thread[\"thread_id\"], values={\"user_info\": {\"name\": \"Bagatur\", \"age\": 35}}\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let’s make sure that the state did in fact update and ask our LLM again how old we are by invoking the graph again:" ] }, { "cell_type": "code", "execution_count": 57, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{'llm': {'messages': [{'content': '', 'additional_kwargs': {'tool_calls': [{'index': 0, 'id': 'call_Ph3tYdt2UNdwqAF3kVL198Bg', 'function': {'arguments': '{\"age\":-1}', 'name': 'PersonalInfo'}, 'type': 'function'}]}, 'response_metadata': {'finish_reason': 'stop'}, 'type': 'ai', 'name': None, 'id': 'run-373f3e22-8549-4f77-9ff2-05133099bb09', 'example': False, 'tool_calls': [{'name': 'PersonalInfo', 'args': {'age': -1}, 'id': 'call_Ph3tYdt2UNdwqAF3kVL198Bg'}], 'invalid_tool_calls': [], 'usage_metadata': None}]}}\n", "{'get_user_info': {'messages': [{'content': 'Hello! My name is Bagatur and I am 26 years old.', 'additional_kwargs': {}, 'response_metadata': {}, 'type': 'human', 'name': None, 'id': '05faf88c-9f85-462e-84ee-4667de35625d', 'example': False}, {'content': '', 'additional_kwargs': {'tool_calls': [{'index': 0, 'id': 'call_VrA7UKg2w99BvIpsqrwjoawk', 'function': {'arguments': '{\"age\":26,\"name\":\"Bagatur\"}', 'name': 'PersonalInfo'}, 'type': 'function'}]}, 'response_metadata': {'finish_reason': 'stop'}, 'type': 'ai', 'name': None, 'id': 'run-067609c1-4f1a-4d6d-bc1c-224a29153d37', 'example': False, 'tool_calls': [{'name': 'PersonalInfo', 'args': {'age': 26, 'name': 'Bagatur'}, 'id': 'call_VrA7UKg2w99BvIpsqrwjoawk'}], 'invalid_tool_calls': [], 'usage_metadata': None}, {'content': 'Hello Bagatur! How can I assist you today?', 'additional_kwargs': {}, 'response_metadata': {'finish_reason': 'stop'}, 'type': 'ai', 'name': None, 'id': 'run-48041247-fe04-419f-9f12-511e28c5f8aa', 'example': False, 'tool_calls': [], 'invalid_tool_calls': [], 'usage_metadata': None}, {'content': 'Hello! What is my age?', 'additional_kwargs': {}, 'response_metadata': {}, 'type': 'human', 'name': None, 'id': 'e94fa535-f821-4bfb-9d96-68c414adad8d', 'example': False}, {'content': '', 'additional_kwargs': {'tool_calls': [{'index': 0, 'id': 'call_yC7HUAQfcLzheepMD8SnAojR', 'function': {'arguments': '{\"age\":-1}', 'name': 'PersonalInfo'}, 'type': 'function'}]}, 'response_metadata': {'finish_reason': 'stop'}, 'type': 'ai', 'name': None, 'id': 'run-b6a653e2-11ea-4f82-aa6a-ef321deed9ce', 'example': False, 'tool_calls': [{'name': 'PersonalInfo', 'args': {'age': -1}, 'id': 'call_yC7HUAQfcLzheepMD8SnAojR'}], 'invalid_tool_calls': [], 'usage_metadata': None}, {'content': 'You are 26 years old, Bagatur! How can I assist you today?', 'additional_kwargs': {}, 'response_metadata': {'finish_reason': 'stop'}, 'type': 'ai', 'name': None, 'id': 'run-e3b6e627-b4b1-4249-a1bc-a6359ad7b961', 'example': False, 'tool_calls': [], 'invalid_tool_calls': [], 'usage_metadata': None}, {'content': 'Hello! What is my age?', 'additional_kwargs': {}, 'response_metadata': {}, 'type': 'human', 'name': None, 'id': '24ac02be-7b08-4e91-9d20-90507ed25391', 'example': False}, {'content': '', 'additional_kwargs': {'tool_calls': [{'index': 0, 'id': 'call_Ph3tYdt2UNdwqAF3kVL198Bg', 'function': {'arguments': '{\"age\":-1}', 'name': 'PersonalInfo'}, 'type': 'function'}]}, 'response_metadata': {'finish_reason': 'stop'}, 'type': 'ai', 'name': None, 'id': 'run-373f3e22-8549-4f77-9ff2-05133099bb09', 'example': False, 'tool_calls': [{'name': 'PersonalInfo', 'args': {'age': -1}, 'id': 'call_Ph3tYdt2UNdwqAF3kVL198Bg'}], 'invalid_tool_calls': [], 'usage_metadata': None}], 'user_info': {'age': -1}}}\n", "{'respond_to_user': {'messages': [{'content': 'My apologies for the confusion earlier. You are 35 years old, Bagatur! How can I assist you today?', 'additional_kwargs': {}, 'response_metadata': {'finish_reason': 'stop'}, 'type': 'ai', 'name': None, 'id': 'run-298306df-34a5-459a-a8c3-a0f16440c429', 'example': False, 'tool_calls': [], 'invalid_tool_calls': [], 'usage_metadata': None}]}}\n" ] } ], "source": [ "input = {\"messages\": [{\"role\": \"user\", \"content\": \"Hello! What is my age?\"}]}\n", "\n", "await run_input(client, thread, assistant, input)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Voila! The LLM knows our users age updated without us having to prompt it at all.\n", "\n", "The last thing we will talk about is patching the thread, which is used when we want to update the metadata of a state. For example, say we actually wanted to update our last state to have `node_id:4` instead of `node_id:3`. To do this, we can call:" ] }, { "cell_type": "code", "execution_count": 58, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'configurable': {'thread_id': '4f044e5a-6f6e-4663-923e-6333c052ce9f',\n", " 'thread_ts': '1ef2e5d6-e39f-6d26-800e-22245f8220a7'}}" ] }, "execution_count": 58, "metadata": {}, "output_type": "execute_result" } ], "source": [ "await client.threads.patch_state(thread_id=thread[\"thread_id\"], metadata={\"node_id\": 4})" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can check that this worked by checking the metadata of our state" ] }, { "cell_type": "code", "execution_count": 59, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Current node id is 4\n" ] } ], "source": [ "state = await client.threads.get_state(thread_id=thread[\"thread_id\"])\n", "print(f\"Current node id is {state['metadata']['node_id']}\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Perfect! The patch worked as expected." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.9" } }, "nbformat": 4, "nbformat_minor": 2 }