examples: update HITL notebook (#543)

* update notebook

* update

* verify via ToolMessage

* update

* revert change to legacy notebook

* clean up

examples/human-in-the-loop.ipynb

@@ -102,7 +102,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 4,
+   "execution_count": 2,
    "id": "6098e5cb",
    "metadata": {},
    "outputs": [],
@@ -135,7 +135,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 5,
+   "execution_count": 3,
    "id": "d7ef57dd-5d6e-4ad3-9377-a92201c1310e",
    "metadata": {},
    "outputs": [],
@@ -168,7 +168,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 6,
+   "execution_count": 4,
    "id": "5cf3331e-ccb3-41c8-aeb9-a840a94d41e7",
    "metadata": {},
    "outputs": [],
@@ -193,7 +193,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 7,
+   "execution_count": 5,
    "id": "892b54b9-75f0-4804-9ed0-88b5e5532989",
    "metadata": {},
    "outputs": [],
@@ -215,7 +215,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 8,
+   "execution_count": 6,
    "id": "cd3cbae5-d92c-4559-a4aa-44721b80d107",
    "metadata": {},
    "outputs": [],
@@ -252,7 +252,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 9,
+   "execution_count": 7,
    "id": "3b541bb9-900c-40d0-964d-7b5dfee30667",
    "metadata": {},
    "outputs": [],
@@ -315,7 +315,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 10,
+   "execution_count": 8,
    "id": "812b4e70-4956-4415-8880-db48b3dcbad2",
    "metadata": {},
    "outputs": [],
@@ -371,7 +371,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 11,
+   "execution_count": 9,
    "id": "6845ed6a-d155-4105-9160-28849877248b",
    "metadata": {},
    "outputs": [],
@@ -393,7 +393,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 12,
+   "execution_count": 10,
    "id": "79d29875-8aa8-434c-9f20-1c58346a6249",
    "metadata": {},
    "outputs": [],
@@ -414,7 +414,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 13,
+   "execution_count": 11,
    "id": "cd13cc10-c74c-415c-871d-ca2a1e547cff",
    "metadata": {},
    "outputs": [
@@ -447,7 +447,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 14,
+   "execution_count": 12,
    "id": "cfd140f0-a5a6-4697-8115-322242f197b5",
    "metadata": {},
    "outputs": [
@@ -475,7 +475,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 15,
+   "execution_count": 13,
    "id": "08ae8246-11d5-40e1-8567-361e5bef8917",
    "metadata": {},
    "outputs": [
@@ -500,7 +500,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 16,
+   "execution_count": 14,
    "id": "273d56a8-f40f-4a51-a27f-7c6bb2bda0ba",
    "metadata": {},
    "outputs": [
@@ -513,8 +513,8 @@
       "what's the weather in sf now?\n",
       "==================================\u001b[1m Ai Message \u001b[0m==================================\n",
       "Tool Calls:\n",
-      "  search (call_yBVzSaJu8hvdunRIklHNLGhV)\n",
-      " Call ID: call_yBVzSaJu8hvdunRIklHNLGhV\n",
+      "  search (call_bxEBI37XzVUvfLKZB2JMewRk)\n",
+      " Call ID: call_bxEBI37XzVUvfLKZB2JMewRk\n",
       "  Args:\n",
       "    query: weather in San Francisco\n"
      ]
@@ -540,7 +540,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 17,
+   "execution_count": 15,
    "id": "51923913-20f7-4ee1-b9ba-d01f5fb2869b",
    "metadata": {},
    "outputs": [
@@ -554,7 +554,7 @@
       "[\"It's sunny in San Francisco, but you better look out if you're a Gemini 😈.\"]\n",
       "==================================\u001b[1m Ai Message \u001b[0m==================================\n",
       "\n",
-      "It seems like it's sunny in San Francisco at the moment. Enjoy the weather! If you need more specific details, feel free to ask.\n"
+      "The current weather in San Francisco is sunny. Enjoy the sunshine!\n"
      ]
     }
    ],
@@ -562,6 +562,395 @@
     "for event in app.stream(None, thread, stream_mode=\"values\"):\n",
     "    event[\"messages\"][-1].pretty_print()"
    ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "89bdf865-9e87-4080-b482-061c132d6a5c",
+   "metadata": {},
+   "source": [
+    "## Conversational human-in-the-loop\n",
+    "\n",
+    "Suppose that upon interruption, we wish to intervene in the agent's action. How should we implement an intervention?\n",
+    "\n",
+    "There are multiple options, and the ideal option may depend on the specifics of your application and capabilities of your chosen LLM. Note that many chat models require that messages with tool calls be immediately followed by a tool message containing the result of the tool call. So our intervention may:\n",
+    "\n",
+    "1. Update the parameters of the tool call before proceeding normally (see this [how-to guide](https://langchain-ai.github.io/langgraph/how-tos/time-travel/#pause-before-tools) for an example);\n",
+    "2. Add a tool message to the conversation history indicating the user's desired intervention (see an example [here](https://langchain-ai.github.io/langgraph/tutorials/customer-support/customer-support/#state-assistant));\n",
+    "3. Catch the tool call message, replacing it with a `AIMessage` asking for verification and only adding the tool call message to the conversation history if approved.\n",
+    "\n",
+    "Below we demonstrate the third option, supporting a conversational human-in-the-loop experience in which the user can instruct the LLM to modify tool calls before execution via a typical chat interface. We include two implementations-- one in which we interrupt and manually update the state, and one in which we customize the state of the underlying graph.\n",
+    "\n",
+    "### Manually update state\n",
+    "\n",
+    "Once the graph execution is interrupted, we are free to issue arbitrary updates to the state. Below, if a tool call is generated, we will:\n",
+    "\n",
+    "1. Append a \"verification\" `AIMessage` to the state asking for user approval;\n",
+    "2. Receive user input and append it to the state as a `HumanMessage`;\n",
+    "3. If approved, append the tool call message to the state and resume execution;\n",
+    "4. Otherwise, resume execution from the new user input."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 16,
+   "id": "5454f436-d56e-4499-9381-06192aca1b56",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import json\n",
+    "from typing import Optional\n",
+    "\n",
+    "from langchain_core.messages import AIMessage\n",
+    "\n",
+    "\n",
+    "# Helper function to construct message asking for verification\n",
+    "def generate_verification_message(message: AIMessage) -> None:\n",
+    "    \"\"\"Generate \"verification message\" from message with tool calls.\"\"\"\n",
+    "    serialized_tool_calls = json.dumps(\n",
+    "        message.tool_calls,\n",
+    "        indent=2,\n",
+    "    )\n",
+    "    return AIMessage(\n",
+    "        content = (\n",
+    "            \"I plan to invoke the following tools, do you approve?\\n\\n\"\n",
+    "            \"Type 'y' if you do, anything else to stop.\\n\\n\"\n",
+    "            f\"{serialized_tool_calls}\"\n",
+    "        ),\n",
+    "        id=message.id,\n",
+    "    )\n",
+    "\n",
+    "\n",
+    "# Helper function to stream output from the graph\n",
+    "def stream_app_catch_tool_calls(inputs, thread) -> Optional[AIMessage]:\n",
+    "    \"\"\"Stream app, catching tool calls.\"\"\"\n",
+    "    tool_call_message = None\n",
+    "    for event in app.stream(inputs, thread, stream_mode=\"values\"):\n",
+    "        message = event[\"messages\"][-1]\n",
+    "        if isinstance(message, AIMessage) and message.tool_calls:\n",
+    "            tool_call_message = message\n",
+    "        else:\n",
+    "            message.pretty_print()\n",
+    "\n",
+    "    return tool_call_message"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 17,
+   "id": "bca5e7e3-9203-4032-b51e-3ea3e6e50de1",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "================================\u001b[1m Human Message \u001b[0m=================================\n",
+      "\n",
+      "what's the weather in sf now?\n",
+      "==================================\u001b[1m Ai Message \u001b[0m==================================\n",
+      "\n",
+      "I plan to invoke the following tools, do you approve?\n",
+      "\n",
+      "Type 'y' if you do, anything else to stop.\n",
+      "\n",
+      "[\n",
+      "  {\n",
+      "    \"name\": \"search\",\n",
+      "    \"args\": {\n",
+      "      \"query\": \"weather in San Francisco\"\n",
+      "    },\n",
+      "    \"id\": \"call_fwf8h8Km90CxA7rfaRJypFAB\"\n",
+      "  }\n",
+      "]\n"
+     ]
+    },
+    {
+     "name": "stdin",
+     "output_type": "stream",
+     "text": [
+      " can you specify sf in CA?\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "================================\u001b[1m Human Message \u001b[0m=================================\n",
+      "\n",
+      "can you specify sf in CA?\n",
+      "==================================\u001b[1m Ai Message \u001b[0m==================================\n",
+      "\n",
+      "I plan to invoke the following tools, do you approve?\n",
+      "\n",
+      "Type 'y' if you do, anything else to stop.\n",
+      "\n",
+      "[\n",
+      "  {\n",
+      "    \"name\": \"search\",\n",
+      "    \"args\": {\n",
+      "      \"query\": \"weather in San Francisco, California\"\n",
+      "    },\n",
+      "    \"id\": \"call_AKIFrAtiunH0AZmLxJE0WSRR\"\n",
+      "  }\n",
+      "]\n"
+     ]
+    },
+    {
+     "name": "stdin",
+     "output_type": "stream",
+     "text": [
+      " y\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "================================\u001b[1m Human Message \u001b[0m=================================\n",
+      "\n",
+      "y\n",
+      "=================================\u001b[1m Tool Message \u001b[0m=================================\n",
+      "Name: search\n",
+      "\n",
+      "[\"It's sunny in San Francisco, but you better look out if you're a Gemini 😈.\"]\n",
+      "==================================\u001b[1m Ai Message \u001b[0m==================================\n",
+      "\n",
+      "The current weather in San Francisco, California is sunny. Enjoy the sunshine!\n"
+     ]
+    }
+   ],
+   "source": [
+    "import uuid\n",
+    "\n",
+    "thread = {\"configurable\": {\"thread_id\": \"3\"}}\n",
+    "\n",
+    "tool_call_message = stream_app_catch_tool_calls(\n",
+    "    {\"messages\": [HumanMessage(\"what's the weather in sf now?\")]},\n",
+    "    thread,\n",
+    ")\n",
+    "\n",
+    "while tool_call_message:\n",
+    "    verification_message = generate_verification_message(tool_call_message)\n",
+    "    verification_message.pretty_print()\n",
+    "    input_message = HumanMessage(input())\n",
+    "    if input_message.content == \"exit\":\n",
+    "        break\n",
+    "    input_message.pretty_print()\n",
+    "\n",
+    "    # First we update the state with the verification message and the input message.\n",
+    "    # note that `generate_verification_message` sets the message ID to be the same\n",
+    "    # as the ID from the original tool call message. Updating the state with this\n",
+    "    # message will overwrite the previous tool call.\n",
+    "    snapshot = app.get_state(thread)\n",
+    "    snapshot.values[\"messages\"] += [verification_message, input_message]\n",
+    "\n",
+    "    if input_message.content == \"y\":\n",
+    "        tool_call_message.id = str(uuid.uuid4())\n",
+    "        # If verified, we append the tool call message to the state\n",
+    "        # and resume execution.\n",
+    "        snapshot.values[\"messages\"] += [tool_call_message]\n",
+    "        app.update_state(thread, snapshot.values, as_node=\"agent\")\n",
+    "    else:\n",
+    "        # Otherwise, resume execution from the input message.\n",
+    "        app.update_state(thread, snapshot.values, as_node=\"__start__\")\n",
+    "\n",
+    "    tool_call_message = stream_app_catch_tool_calls(None, thread)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "0f11f99d-f4e6-4ed8-a931-3058c76bfd92",
+   "metadata": {},
+   "source": [
+    "### Customize the state\n",
+    "\n",
+    "Alternatively, we can handle the verification inside the graph, without interrupting execution. We only need to make two changes to the original graph:\n",
+    "\n",
+    "1. We add a key to the state where we will cache tool calls generated by the LLM;\n",
+    "2. When calling the LLM, if a tool call message is generated we will cache it and generate a verification message instead. If the tool call is verified, we will return the cached message."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 18,
+   "id": "03232f16-d6fe-46d0-afa0-a6f0d0bf16de",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "class State(TypedDict):\n",
+    "    messages: Annotated[list, add_messages]\n",
+    "    tool_call_message: Optional[AIMessage]\n",
+    "\n",
+    "\n",
+    "def call_model(state):\n",
+    "    messages = state[\"messages\"]\n",
+    "    if messages[-1].content == \"y\":\n",
+    "        return {\n",
+    "            \"messages\": [state[\"tool_call_message\"]],\n",
+    "            \"tool_call_message\": None,\n",
+    "        }\n",
+    "    else:\n",
+    "        response = model.invoke(messages)\n",
+    "        if response.tool_calls:\n",
+    "            verification_message = generate_verification_message(response)\n",
+    "            response.id = str(uuid.uuid4())\n",
+    "            return {\n",
+    "                \"messages\": [verification_message],\n",
+    "                \"tool_call_message\": response,\n",
+    "            }\n",
+    "        else:\n",
+    "            return {\n",
+    "                \"messages\": [response],\n",
+    "                \"tool_call_message\": None,\n",
+    "            }"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "7b07c467-b4aa-4764-8251-be77aea968fb",
+   "metadata": {},
+   "source": [
+    "We then build and compile the graph exactly as before:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 19,
+   "id": "502dc688-c926-407e-8759-8c9e39eb4257",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "workflow = StateGraph(State)\n",
+    "\n",
+    "workflow.add_node(\"agent\", call_model)\n",
+    "workflow.add_node(\"action\", call_tool)\n",
+    "\n",
+    "workflow.set_entry_point(\"agent\")\n",
+    "\n",
+    "workflow.add_conditional_edges(\n",
+    "    \"agent\",\n",
+    "    should_continue,\n",
+    "    {\n",
+    "        \"continue\": \"action\",\n",
+    "        \"end\": END,\n",
+    "    },\n",
+    ")\n",
+    "\n",
+    "workflow.add_edge(\"action\", \"agent\")\n",
+    "\n",
+    "app = workflow.compile(checkpointer=memory)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 20,
+   "id": "2f01c0e8-51f4-419c-b220-381ea2519437",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "================================\u001b[1m Human Message \u001b[0m=================================\n",
+      "\n",
+      "what's the weather in sf?\n",
+      "==================================\u001b[1m Ai Message \u001b[0m==================================\n",
+      "\n",
+      "I plan to invoke the following tools, do you approve?\n",
+      "\n",
+      "Type 'y' if you do, anything else to stop.\n",
+      "\n",
+      "[\n",
+      "  {\n",
+      "    \"name\": \"search\",\n",
+      "    \"args\": {\n",
+      "      \"query\": \"weather in San Francisco\"\n",
+      "    },\n",
+      "    \"id\": \"call_Nanzshz5kQZc0FWJcD2hkYXn\"\n",
+      "  }\n",
+      "]\n"
+     ]
+    }
+   ],
+   "source": [
+    "thread = {\"configurable\": {\"thread_id\": \"4\"}}\n",
+    "\n",
+    "inputs = [HumanMessage(content=\"what's the weather in sf?\")]\n",
+    "for event in app.stream({\"messages\": inputs}, thread, stream_mode=\"values\"):\n",
+    "    event[\"messages\"][-1].pretty_print()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 21,
+   "id": "2ba9d2c7-1929-4e48-afee-d631865da33d",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "================================\u001b[1m Human Message \u001b[0m=================================\n",
+      "\n",
+      "can you specify sf in CA?\n",
+      "==================================\u001b[1m Ai Message \u001b[0m==================================\n",
+      "\n",
+      "I plan to invoke the following tools, do you approve?\n",
+      "\n",
+      "Type 'y' if you do, anything else to stop.\n",
+      "\n",
+      "[\n",
+      "  {\n",
+      "    \"name\": \"search\",\n",
+      "    \"args\": {\n",
+      "      \"query\": \"weather in San Francisco, California\"\n",
+      "    },\n",
+      "    \"id\": \"call_qOnskgB8E72ReGOroSBPdu3v\"\n",
+      "  }\n",
+      "]\n"
+     ]
+    }
+   ],
+   "source": [
+    "inputs = [HumanMessage(content=\"can you specify sf in CA?\")]\n",
+    "for event in app.stream({\"messages\": inputs}, thread, stream_mode=\"values\"):\n",
+    "    event[\"messages\"][-1].pretty_print()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 22,
+   "id": "cc6d7e6e-7144-4fcb-97ab-799e6524730d",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "================================\u001b[1m Human Message \u001b[0m=================================\n",
+      "\n",
+      "y\n",
+      "==================================\u001b[1m Ai Message \u001b[0m==================================\n",
+      "Tool Calls:\n",
+      "  search (call_qOnskgB8E72ReGOroSBPdu3v)\n",
+      " Call ID: call_qOnskgB8E72ReGOroSBPdu3v\n",
+      "  Args:\n",
+      "    query: weather in San Francisco, California\n",
+      "=================================\u001b[1m Tool Message \u001b[0m=================================\n",
+      "Name: search\n",
+      "\n",
+      "[\"It's sunny in San Francisco, but you better look out if you're a Gemini 😈.\"]\n",
+      "==================================\u001b[1m Ai Message \u001b[0m==================================\n",
+      "\n",
+      "The weather in San Francisco, California is sunny. Enjoy the sunshine!\n"
+     ]
+    }
+   ],
+   "source": [
+    "inputs = [HumanMessage(content=\"y\")]\n",
+    "for event in app.stream({\"messages\": inputs}, thread, stream_mode=\"values\"):\n",
+    "    event[\"messages\"][-1].pretty_print()"
+   ]
   }
  ],
  "metadata": {
@@ -580,7 +969,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.11.2"
+   "version": "3.10.4"
   }
  },
  "nbformat": 4,