{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "15c4bd28",
   "metadata": {},
   "source": [
    "# How to stream from the final node"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "964686a6-8fed-4360-84d2-958c48186008",
   "metadata": {},
   "source": [
    "A common use case is streaming from an agent is to stream LLM tokens from inside the final node. This guide demonstrates how you can do this.\n",
    "\n",
    "## Setup\n",
    "\n",
    "First let's install our required packages and set our API keys"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "c04a3f8e-0bc9-430b-85db-3edfa026d2cd",
   "metadata": {},
   "outputs": [],
   "source": [
    "%%capture --no-stderr\n",
    "%pip install -U langgraph langchain-openai langchain-community"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c87e4a47-4099-4d1a-907c-a99fa857165a",
   "metadata": {},
   "outputs": [],
   "source": [
    "import getpass\n",
    "import os\n",
    "\n",
    "\n",
    "def _set_env(var: str):\n",
    "    if not os.environ.get(var):\n",
    "        os.environ[var] = getpass.getpass(f\"{var}: \")\n",
    "\n",
    "\n",
    "_set_env(\"OPENAI_API_KEY\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eb79e50b",
   "metadata": {},
   "source": [
    "<div class=\"admonition tip\">\n",
    "    <p class=\"admonition-title\">Set up <a href=\"https://smith.langchain.com\">LangSmith</a> for LangGraph development</p>\n",
    "    <p style=\"padding-top: 5px;\">\n",
    "        Sign up for LangSmith to quickly spot issues and improve the performance of your LangGraph projects. LangSmith lets you use trace data to debug, test, and monitor your LLM apps built with LangGraph — read more about how to get started <a href=\"https://docs.smith.langchain.com\">here</a>. \n",
    "    </p>\n",
    "</div>    "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "17f994ca-28e7-4379-a1c9-8c1682773b5f",
   "metadata": {},
   "source": [
    "## Define model and tools"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "5e62618d-0e0c-483c-acd3-40a26e61894a",
   "metadata": {},
   "outputs": [],
   "source": [
    "from typing import Literal\n",
    "from langchain_community.tools.tavily_search import TavilySearchResults\n",
    "from langchain_core.runnables import ConfigurableField\n",
    "from langchain_core.tools import tool\n",
    "from langchain_openai import ChatOpenAI\n",
    "from langgraph.prebuilt import create_react_agent\n",
    "from langgraph.prebuilt import ToolNode\n",
    "\n",
    "\n",
    "@tool\n",
    "def get_weather(city: Literal[\"nyc\", \"sf\"]):\n",
    "    \"\"\"Use this to get weather information.\"\"\"\n",
    "    if city == \"nyc\":\n",
    "        return \"It might be cloudy in nyc\"\n",
    "    elif city == \"sf\":\n",
    "        return \"It's always sunny in sf\"\n",
    "    else:\n",
    "        raise AssertionError(\"Unknown city\")\n",
    "\n",
    "\n",
    "tools = [get_weather]\n",
    "model = ChatOpenAI(model_name=\"gpt-3.5-turbo\", temperature=0)\n",
    "final_model = ChatOpenAI(model_name=\"gpt-3.5-turbo\", temperature=0)\n",
    "\n",
    "model = model.bind_tools(tools)\n",
    "# NOTE: this is where we're adding a tag that we'll can use later to filter the model stream events to only the model called in the final node.\n",
    "# This is not necessary if you call a single LLM but might be important in case you call multiple models within the node and want to filter events\n",
    "# from only one of them.\n",
    "final_model = final_model.with_config(tags=[\"final_node\"])\n",
    "tool_node = ToolNode(tools=tools)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9acef997-5dd6-4108-baf1-c4d6be3e4999",
   "metadata": {},
   "source": [
    "## Define graph"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "8c7339d2-1835-4b5a-a99c-a60e150280af",
   "metadata": {},
   "outputs": [],
   "source": [
    "from typing import TypedDict, Annotated\n",
    "\n",
    "from langgraph.graph import END, StateGraph, START\n",
    "from langgraph.graph.message import MessagesState\n",
    "from langchain_core.messages import BaseMessage, SystemMessage, HumanMessage\n",
    "\n",
    "\n",
    "def should_continue(state: MessagesState) -> Literal[\"tools\", \"final\"]:\n",
    "    messages = state[\"messages\"]\n",
    "    last_message = messages[-1]\n",
    "    # If the LLM makes a tool call, then we route to the \"tools\" node\n",
    "    if last_message.tool_calls:\n",
    "        return \"tools\"\n",
    "    # Otherwise, we stop (reply to the user)\n",
    "    return \"final\"\n",
    "\n",
    "\n",
    "def call_model(state: MessagesState):\n",
    "    messages = state[\"messages\"]\n",
    "    response = model.invoke(messages)\n",
    "    # We return a list, because this will get added to the existing list\n",
    "    return {\"messages\": [response]}\n",
    "\n",
    "\n",
    "def call_final_model(state: MessagesState):\n",
    "    messages = state[\"messages\"]\n",
    "    last_ai_message = messages[-1]\n",
    "    response = final_model.invoke(\n",
    "        [\n",
    "            SystemMessage(\"Rewrite this in the voice of Al Roker\"),\n",
    "            HumanMessage(last_ai_message.content),\n",
    "        ]\n",
    "    )\n",
    "    # overwrite the last AI message from the agent\n",
    "    response.id = last_ai_message.id\n",
    "    return {\"messages\": [response]}\n",
    "\n",
    "workflow = StateGraph(MessagesState)\n",
    "\n",
    "workflow.add_node(\"agent\", call_model)\n",
    "workflow.add_node(\"tools\", tool_node)\n",
    "# add a separate final node\n",
    "workflow.add_node(\"final\", call_final_model)\n",
    "\n",
    "workflow.add_edge(START, \"agent\")\n",
    "workflow.add_conditional_edges(\n",
    "    \"agent\",\n",
    "    should_continue,\n",
    ")\n",
    "\n",
    "workflow.add_edge(\"tools\", \"agent\")\n",
    "workflow.add_edge(\"final\", END)\n",
    "\n",
    "app = workflow.compile()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "2ab6d079-ba06-48ba-abe5-e72df24407af",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/jpeg": "/9j/4AAQSkZJRgABAQAAAQABAAD/4gHYSUNDX1BST0ZJTEUAAQEAAAHIAAAAAAQwAABtbnRyUkdCIFhZWiAH4AABAAEAAAAAAABhY3NwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAQAA9tYAAQAAAADTLQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAlkZXNjAAAA8AAAACRyWFlaAAABFAAAABRnWFlaAAABKAAAABRiWFlaAAABPAAAABR3dHB0AAABUAAAABRyVFJDAAABZAAAAChnVFJDAAABZAAAAChiVFJDAAABZAAAAChjcHJ0AAABjAAAADxtbHVjAAAAAAAAAAEAAAAMZW5VUwAAAAgAAAAcAHMAUgBHAEJYWVogAAAAAAAAb6IAADj1AAADkFhZWiAAAAAAAABimQAAt4UAABjaWFlaIAAAAAAAACSgAAAPhAAAts9YWVogAAAAAAAA9tYAAQAAAADTLXBhcmEAAAAAAAQAAAACZmYAAPKnAAANWQAAE9AAAApbAAAAAAAAAABtbHVjAAAAAAAAAAEAAAAMZW5VUwAAACAAAAAcAEcAbwBvAGcAbABlACAASQBuAGMALgAgADIAMAAxADb/2wBDAAMCAgMCAgMDAwMEAwMEBQgFBQQEBQoHBwYIDAoMDAsKCwsNDhIQDQ4RDgsLEBYQERMUFRUVDA8XGBYUGBIUFRT/2wBDAQMEBAUEBQkFBQkUDQsNFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBT/wAARCAEvALUDASIAAhEBAxEB/8QAHQABAAIDAAMBAAAAAAAAAAAAAAYHBAUIAgMJAf/EAFwQAAEEAQIDAgYHEQwFDQAAAAEAAgMEBQYRBxIhMUEIExQWIlEVVVZhdpTRFyMyNzhCYnF1kZOVsbTS09QJNTZSU3J0gZKhsrMYM0NUwSQnV4KDhZaio6TC8PH/xAAbAQEBAAMBAQEAAAAAAAAAAAAAAQIDBAUGB//EADQRAQABAgEICAUFAQEAAAAAAAABAhEDBBIhMVFhkaEFExRBcbHB0RUyUlPhIzNCgfCy8f/aAAwDAQACEQMRAD8A+qaIiAiLAzWYjwtNsz45LEsj2wwVoADJNI7sa0EgeskkgAAuJABIypiaptAz1rptRYmu/llydOJ38V9hgP5VqPM450eO1NMMk5w/e6Nzm0ovseT/AGp7i6TffqQ1gPKNjHpDBQt5Y8LjmN332bUjA3+8t2bhU6KpmfD/AHouh5+dWF9uKHxlnyp51YX24ofGWfKnmrhfaeh8WZ8ieauF9p6HxZnyJ+jv5LoPOrC+3FD4yz5U86sL7cUPjLPlTzVwvtPQ+LM+RPNXC+09D4sz5E/R38jQedWF9uKHxlnyp51YX24ofGWfKnmrhfaeh8WZ8ieauF9p6HxZnyJ+jv5Gh5xajxM7w2LKUpHHuZYYT+VbFamTSOCmZySYXHPb28rqsZH5FrPMpuD+faZm9iHt6+Qbk0pfsTF2R/zo+UjpuHAcpZuFVoiZjx/3omhKUWuwmZZmaz3+Jkq2YXmKxVm254XjuO3QggggjoQQR2rYrTVTNM2lBERYgiIgIiICIiAiIgKL1dsvr+9JJs6LD1o4IWn62abd8jvVvyCIA9o5njpud5QoxhB5HrjUtd+4daZWvRnbo5pYYjsfWDCN/VzD1rowvlrnvt6xHldY70nREXOj8JABJOwCqiTwnNC3tL6mzGByM2eGDoTX3x16NoMsMYeXeKTxREjS/ZpfHzgb7noCrWkDXRuDm87SCC3bfcerZco8NMPqOcas0VpTCarxXDqzpm5HWx2sqHkzsXkJDyx1qszvSlhLXvJG72s5Rs/rsgtjS3hIaUy/CjF64ykl3EVLLa8U0UmLucwsyRNkMULDCHzt9I7Pja5rtjse1bb/AEgOH40I3WbtSQM015Y3HvvPhlb4mw6QRiOVhZzxEOcN+dreUHc7DqqUZqzVtjgfw5w1PT+udOVsO+hi9VihiJo8m2COq5rvJPRLpGGZkYdJDu4NduO/aKYzQebl0bq7GR6U1UK1ziVhczWhzkE1mxPRc+mHzSPcXl2whkL+Zxcwbc/KeiC6dU+FfpjT2pdGUIqeYt47PvuCW4MHkBJA2GMuBZD5OXy8z9h6I6N9Ls6q7WOD2NcN9iNxuNlTHHZmRwfEXhXrKDB5XO4nBXb8eQiwtR1uzE2xUdHHIIWbuc0PABIB233VxULYv0a9oRSwCaNsginYWSM3G+zmnq1w36g9hQe9ERBF8jtiNd4myzZseXikozjr6ckbTLE71dGtnB9fM31KUKMaib5bq/SlVm5dXmsZF+w6BjYHw9T3elYb9vY+oqTroxflond6z6LPcIiLnQREQEREBERAREQFpdQYmexPTyePEfspR5hGJXFrZon7eMicR2B3K0g9dnMYdiAQd0izpqmibwamhbPhdfYa7jrlWK7WlYYL2LvxAuaD2xyxnft6+sOHUEggqKf6NfCf/o30sP8AuiD9FTPNaVxeoJI5rlbe1E0tjtwSOhsRgnchsrCHgb7HYHboFrjoiUdI9S52Ju+/KLLH/wB7mE/3rbm4VWmKrePvHsuhpcf4PXDDFX612lw+01VuVpWzQzw4qFr43tILXNIbuCCAQR6lYKi/mTY91We/DQ/qk8ybHuqz34aH9UnV4f18pLRtShFWHFLGZPR3DLV2foapzRvYrD3L1cTSQuYZIoXvbzARjcbtG43WJwYrZbXvCPRmpcnqnMtyOXw9W9YED4mxiSSJr3coMZ2G5Ow3KdXh/XyktG1bSr/JeD5wxzGRtX73D/TVy9alfPPYnxcL5JZHEuc9zi3ckkkkntJW48ybHuqz34aH9UnmTY91We/DQ/qk6vD+vlJaNrQP8G/hTK4F/DjS7yAG7uxMB6AbAfQ9wAClJlwXD3CUsfVr18bTib4ijjKUQaX7dkcMTe37QHQbk7AErGGiJSNpNTZ2Vu++3lMbf72xg/3rPwuk8XgJpJ6lcutyN5ZLlmV887x6jI8lxHvb7Jm4VOmar+HvPsaHrwGKsMt2stkWsZk7jWsMTHcza8LSSyMHvPpEuI7XE9wC3iItVdU1zeU1iIiwBERAREQEREBERAREQEREBERBA+Pn0iuI3wbyX5rItd4Mf1OfDH4N4/8AN2LY8fPpFcRvg3kvzWRa7wY/qc+GPwbx/wCbsQWaiIgIiICIiAiIgIiICIiAiIgIiICIiAiIgIiIIHx8+kVxG+DeS/NZFrvBj+pz4Y/BvH/m7FsePn0iuI3wbyX5rItd4Mf1OfDH4N4/83Ygs1ERAREQEREBERAREQEREBERAREQEREBEWo1DqFuEZXjigNy/aeWV6wdy82w3c5ztjysaOpdse4AEkA5U0zXObTrG3RQk53V5O4x+EaPUbkx297fxXVfns7rD/cMH8bm/Vrq7LXtjjC2cF/uo3Ax2L1Ni+KONrk1coGY/LFo35LDGbQyH+dG3k9Q8U3vcsH9y54JyZ7XWT4l3o3NoYJr6OOPUCS3LGWyEHvDInkEH+Wae5dq8WdIZvjDw6zmj81j8KKGUgMRlZalLoXgh0crd4/omPDXD7XXosfgzobN8E+G2F0dhqOFlqY6ItdYksyh9iVxLpJHbR9rnEnbuGw7AnZa9scYLLsRQj2d1h/uGD+Nzfq1+jO6wB64/CH3vLJhv/6Sdlr2xxgsmyLTae1D7M+U1565pZGqQJ63NzgB2/K9jthzMdsdjsDuCCAQQtyuaqmaJzataCIiwBERAREQEREBERAREQEREBQrUx/5xNPN7vYvIns7/G0vlKmqhWpvpjad+5WR/wA6kuvJf3f6q/5lYbJFXHHTVc+mNLUY6GevYPLZG9HUpjFYyPIXLby1zjDDFJ6AOzS4vf6LQ07qmqXHfW9fhlk4MrdOJzNLWsGl7GocrRgjko1JWxSeUzxRvdAJGiUM6Hxe5aT3hbZqiEdWIuSZ+Nut9L47VlGpn5tdWZtWUNMYTOV6dTl+e1vHTFkbTFFJMzZ7DzPDC/k6N9IHN1BxH4v6G4fa5v34srFXo0qtnF5rUVHHRWG2DZZHLC6KpK+N7CxwIdytI9IeoqZ8DqleMcjJo2vjc17HDdrmncEesFc/5rI6xxGrtU6FymsJsxBkNG2c1WyPsfWgmpTRyiJ7GBrOVzHCRpAeHOG30R7VJ/BVxF/F8BtEyXc7azMdrC0Zq8VmGGMU4zXZtCwxsaXNHreXO9ZKyibzYWRgDtxHyY9eJrb+/wDPp/lP31NlCcD9MfJfcmt/nTKbLVlX7keEeTKRERcbEREQEREBERAREQEREBERAUK1N9MbTv3KyP8AnUlNVF9X4u37IY3NUoDckpMmglrM2Ej4ZTGXFm/a5piYeXcbjfv2B6smmKcWL7J5xMLGtF+InDWhxGrYsWL2QxGQxVsXaGTxUrY7NaXkcwlpe1zSHMe5pa5pBB7OxVlrbwfWYXh9qmjp12o9SWs9kK2Qu1X5ivFNNMzlbJIJJ4nRu5wxpdHIDGeUBoYFYWM4z6WzWdyWEx9m3fzOM2F7H1KE809QnsErGMJYe7qtz551vazPfiS3+qXbOBXP8ZXNlS3DfgxqDU+jc/priBDkKWmvH1Z9P1rNup7KY2aPmc6Zs1KNkbCH8hZtuRs7fodlOrPAelldD57TGZ1XqjP18z4kTW8ldjkniEbw5oiAiEbNyOvobnvPYpPkOIeMxFCzevVcxSpVo3TT2bGHtRxxRtG7nucYwGtABJJ6ABfmM4j4rN46rkMdXy9+hajbNXtVcRakimjcN2vY5sZDmkEEEHYqdRX9MmbOx+3OHmLv6+Zq2d9iS8MPLhDWLm+TugklZK4lvLzc28YG/NtsT071g8LuFlXhPiZMVjs3msnimtZHTp5WwyZlGJvNyxQkMa7l2dt6Rcdmt69FtvPOt7WZ78SW/wBUh1pVDXE4zOAAEkuwtpo6e+Y9lepr15spmyzMD9MfJfcmt/nTKbKr+Cet8FxYiyustP5GK9Qe8YpjGfRxGBzy7xg7WucZAQ3r6AY764gWguLKZicTR3RHkSIiLlQREQEREBERAREQEREBEWNkslVw2OtX707KtKrE+eeeU7MjjaC5zie4AAn+pBkqoMnra1x209ag4Ra4p4uXF51lHK5Z9B0+0cez5o4OcBj3Hdo5urSObY9hXuxecyHHMaE1lofV8+H0PFZnsXarsYWT5ZrSY2M5pR6MRIeSQ3cgtIIIBFoUcdUxcLoadWGpE6R8ro4Iwxpe9xc9xA73OJJPaSSSg/KuLp0bFuxXqQQWLbxJZlija187w0NDnkDdxDWtG57gB3LKREHFf7ptxz8y+G1Ph/jLHi8tqX55c5Ds6Oix3UeseMeA31FrJAe1a/8AcweOvnPojI8NspYDsjgd7eN53elJTe/02jvPi5HdvqlaB0apZ4XHgZaO4iV9dcUMrnNSDO08LNar04bUApxmtWJYwMdCXBhLOZwD+pe4gjfpg+Bv4G+jtG4vh7xZoZnUR1Daw8dyWnLah8ic6xWIe0sEIeWjxhLQX9C1pJOyDsZERBAOI/DnL6hxFGHRmp5NA34MqzJTWKFOKSO31PjWTRkDn5w4ncnq4AnfZZOneKlLUPEnUuivYjMUMjhIobBt26bmVLcMgGz4Zex2zuZux23LHbb7HabLUau0tR1vpfKYDJiY47JV31bArzOhkLHDY7PaQR0P/wBCDboq0oP1FwyyOhNHYzBZTV2mX131Lup7uSZJbqSMbvG+drgC9rgHAuB7eUAb7A2Dj8nTy1cz0bcF2Bsj4jJXkbI0PY4te3cHbdrgQR3EEFBlIiICIiAiIgIiICIiCNcRuIeC4U6LymqtSXPIcPjow+aUNL3Elwa1rWjqXOc5rQPWe5R/CYjM6y1thddxaoyVPR8+EaINJTURBzyzem6WxzelzBvIAzYFpDuuxcHSrW785Ho3Ov0zFXn1E2jM7GxWxvDJZDD4pr/Sb6JdsD1HamiJs3Z0bg5dS14qmonUYTkoIHB0bLPIPGhpBILefm22PYg29evFTrxQQRMggiaGRxRtDWsaBsAAOgAHcvYiICIvRevVsXSsXLliKpUrxulmnmeGMjY0buc5x6AAAkkoIN4QliKrwF4jyTSMij83MiOZ7g0bmtIANz6yQP61heDH08HPhj8G8f8Am7FUFWG94bOq2XbUc9HgRhrW9au8GN+qrUbv9Y8dCKrHDoD9ER167iPqiCCKrBHDDGyGGNoYyONoa1rQNgAB2ADuQexERAREQFVOZ4bX+F+lcxJwZwOAx2byOUZk7lHImWOta7BK1nKdonOa0bbDlB3O253VrIgj2J15g8xqrKaXgyVeXUmJghnyGOicXOrtlG7CSQAQR19exBIG4UhVbacytGfjrrChHot+OvQY6m+XVRi2bkWuB5YQ7lG/i/VzHt7ArJQEREBERAREQEReL5GRjd7g0fZHZBUnhDeEvp7wasZh8jqXC5/JUMnLJAyzhqscscMjQ1wZK6SRga54Li0DckRv/irhbRP7oTqLS2Ng0ToDTjsvZtZ2Y0Mnq+8+1LJBPM4xxGKPkLHbvZ18a8D0h6W4I+jnEjQmnOK+i8ppbUcMV3FZCIxyNLhzRu+tkYT9C9p2IPcQvmhwj8FDM8MfDi0ppjNR+V4WjafmqeWa351ZrwNdJE/t6O8Y2Nrmk7tJ7wQTbSPqyi9XlUP8tH/aCeVQ/wAtH/aCWkeckjIY3ySPayNgLnOcdgAO0krljK3L3hqarmw2MnnpcDcNZ5MlkYHGN+p7LDv4iJw6+TMI9J4+iPZ3Fvt1rqHKeFvrK/w+0lemxvC7FS+J1RqWs7lflHjtoVHd7P5SQdNjt2ECTpLTmnMZpHBUMLhqUONxVGFsFarA3lZGwDYAfL2ntKgycZjKmFx1WhQrRUqNWJsMFaBgZHExo2a1rR0AAAAAWSiICIiAiIgKtPCI4tZPgdwvv6yx2mfOtmPkjNukLpqujgceUyhwjk5uVxZuNh0LjuOXrY3lUP8AKs/tBYWbx+M1Hhr+JyTYbePvQSVbMEjgWyRvaWuafeIJCtpHziofurmeg1hlMhZ0Oy1gJ4Io6eGGVax1WRv0chnFbmk5unokDb319FtF5y1qfR2CzN7Gvw13I0ILc+NlfzvqSSRte6Iu2G5YSW77DfbsHYvl7wV8DaxH4ZN/SGaj8q0rpWcZWazO353crbh1VhO3KTIS3mb2bMlH1q+q3lUP8tH/AGglpHtRettiJ5AbIwk9wcF7FAREQEREGLlLvsbjLdvl5vEQvl5fXytJ/wCCrzF6SxWex1TJZnH1MxkrULJprN6Bszt3AEtbzD0WDsDRsNh69ypzqr+DGY/oc3+AqPaa/g5iv6JF/gC9LJ5mjDmqmbTdlqhhfM+0t7msP8Qi/RT5n2lvc1h/iEX6KgvCvwisFxJGqTJFZwzMHcuMfNdp2YYPJYHhvjnzSxMYxx35jETzsG+46EqQaI426K4i356WBzYtW4q/lZhnrTVnPg328dH41jfGR7kDnZu3qOvULbGPiT/OeKXna3XzPtLe5rD/ABCL9FPmfaW9zWH+IRforQaS48aE11n2YbCagju35WyPrtNeaKO02P6MwSvYGTBveY3O6dexRrQ/hD4t/B7SertbW6+LvZwyRx1sdVnmMsjXyDaKFgkkOzWbnt27TsnaMT654l52rDHD/TLG/OsDjqr992y1azIZGHr6TXsAc09T1BBCkWhMpYymAJtSmxYrWbFN0x23kEUrmNcdgBzFrQTsAN99uiwMJmaeo8PSyuOm8ooXYWWIJuUt543DdrtnAEbgjtC8uGf7yZD7rXvzh6wxqpxMGZqm9pj1W940pciIvLYiIiAoPqoNz+qm4O388xkFJtuWrv6Fhz3uY0SD65rRG48vYSdyDsNpwoNe+mdd+49b/OnXZkvzTV3xDKGMeH+l3Ek6bxBJ6kmjF+inzPtLe5rD/EIv0VAvCF44Hg/j8BWpRtfmc7fbTry2Mfat14GbF0kr212lzyANhE0hzt9x0a4j2QcaK+G1TnsfqHM4yOrpzTsOUzHicZehmjlJJkmj5muY+vy7bBjnvDg4HsXX2jEvbOnil52pz8z7S3uaw/xCL9FPmfaW9zWH+IRforU6U4zaO1tl7GMw2YFq7DWN3xb600Qmr78vjoXPY1s0e5A54y5vUdeoWNo3jroniHkrGN09mxcyMdd1pteepPXMsQIBkj8axvjGbkDmZuOo69U6/E+ueJedrfHh7pYjbzaxH9VGIf8AxW20NakguZjDOlfNXx74jXMri5zI3s3DC49SAQ7bfc7EDfoojwP1zf4l8JNK6pycVeDIZWiyzPHUa5sTXHtDQ5ziB9slSfRv8MdVfzan+B6lddWJhV503tETzhb3ibpoiIvJYiIiDV6q/gxmP6HN/gKj2mv4OYr+iRf4ApJqOF9jT2UijaXSPqyta0d5LCAo1pd7ZNNYlzTu11SEg+scgXoYP7M+Povc5mymk9RZjh9xq4axafy8Gby+XymWx1x9R7cfchllbNGwWfoA543jLSQQd99gthq7H5vwgdU4n2E0xnNH1sVpvM0rFzO0XUeWe5WbDFXiB6yBjhzlzQWDkbsSSunETNRzDp5mb1vLwW05X0VnNM2NGWIbWYu5Kia9au2CnJXdBBKfRmEjnjYxkjlG52Wm0/gHYngfo/F57TWusPqvSuRuVqeV05i3WLFOfd58cxo5hNXlZKGk8rmu6g7bbjrdEzREeEmR1NluGunbmsqjaOp5qjHX4GtDeWT32gkNcRsS0dhJHcpHwz/eTIfda9+cPWWsbhqwtwNx/wBbJlLzmnbtHlMg3/uP/wCLKvRgVeMeq9yWIiLzUEREBQa99M679x63+dOpyoPkGFnEqy89BJiIA33+WaXm+9zt++uzJtdXh6wyjvQHjhhshls1wqfRo2brKWsILVp1eF0gghFS20yPIHotDnNHMdhu4DvCr3j5pPOZnVvFOahhshehucMvIK0laq+Rs9nym2fEsIB5pNnNPIOuzh06rpVFtmm7FQ+t8JqaHiDw5yWAxM016hpPNQNlfC7xEVp0VQ14pnbbN5nsOwcRvyu27Cq/4eYnO5Lipwwzd3Ga/t3IaN+rnsnqWCdteC5PXaeWKI+jFHzxvHNGwR/6scxOy64RTN0ip/BZqZHDcD9OYLMYi/hcrg4jjLVe/CY+Z8Z6vjPY+NwIIeOh6+oqx9G/wx1V/Nqf4HrYrA0awnVmqZAPQ3qx77fXCMkj7zm/fWerCrjdHnDKNUpkiIvMYiIiAona4fN8fI/GZvJYOF7i81aYgfCHHqS1ssT+Xc9dmkDck7dVLEWyjEqw/llb2Q3zAyHuzzf4Cl+zp5gZD3Z5v8BS/Z1MkW7tOJu4R7F0N8wMh7s83+Apfs6eYGQ92eb/AAFL9nUyRO04m7hHsXRBnD+w/wBG1qrNWoT9FF/yaHmHeOeKFrx9trgfUQpTTpwY6pDVqwsr1oWCOOKJoa1jQNgAB2Be5Frrxa8TRVPp5F7iIi0oIiIC1We05V1BHEZXy1rUBJguVnBs0JI2PKSCCD03aQQdhuOg22qLKmqaJzqZ0iHO0DfLiRrLNNBPYIaXT/26/PMDIe7PN/gKX7Opki6O04m7hHst3POlsvqXOeERrjQU+qr7MTg8XRu15461QTvfMDzh5MJaQNumzR9sq1PMDIe7PN/gKX7Oqp4e/Vt8Wvg/iPyOXRCdpxN3CPYuhw0BeJ9PWObc3vAipDf+sV9wpHhsNVwNFtWpGWs3L3Pe4ufI89r3uPVzj3krORYV41eJFqp0eER5FxERaEEREBERAREQEREBERAREQEREBERAREQc78Pfq2+LXwfxH5HLohc78Pfq2+LXwfxH5HLohAREQEREBERAREQEREBERAREQEREBERAREQERVH4VPBWPj1wUzmmWRsdlmNF7FveduS3GCWDc9nMC6MnuEhKCJ8Pfq2+LXwfxH5HLohfALQ+gMxr7XuJ0jjazzmMjcbSZFI0gxuLtnF47QGjcu9QB9S+8Wh9K19CaK0/pqnI+aphsfXx0Mku3O9kMbY2k7d5DQg3aIiAiIgIiICIiAiIgIiICIiAiIgIiqPjFrOaS4dM0ZXRR+KEmQkYdiWu35YQe7cbud7xaOxxXXkuTV5VixhUf8AkDdag414bFzyVsbDPnbDDyudU5WwMPqMriAf+pzbd6jjuPOW6culau32eWcD/dAVXrGNjY1rWhrWjYNA2AC/V9rh9E5JRTaqnO3zM+kwX3LA+bzmPcrT/G7v2dPm85j3K0/xu79nVfotnwvIvt86vdM7chWitBUdE+EXqTizW07Tlt5WNxr4zy8tZSnkA8oma/xG7nSel02G3jH9u42vX5vOY9ytP8bu/Z1X6xsjkqeIqPtX7UFKqwta6exII2NLnBrQXEgdSQB6yQEnozIo0zh86vdb7lk/N5zHuVp/jd37OvKPj1lA4eN0rXDN+visqXH7xgH5VXqJ8LyL7fOr3M7cvLSnFfC6psx0yJ8VkX/QVLzQ0yfzHtJa77QPNt3BTRcrTQssRlkg5mn39iCOoIPcQeoPcrp4S63n1HStYzIymXJ0A0+Od0M8Lt+V5+yBa5rtvUD05gF890j0XGT09dg/L3xs/BrWAiIvnAREQEREBERAREQEREBcx52w+5q3Uk8h3kdk5mE+8wiNv/lY0Lpxc+cSsFJp/XV5/JtUyp8sgcB05w1rZWfb3Af/ANp7x2+j6Drppx6qZ1zGjj/uC90o4iwM3YyVbHvkxNKvkLoI5YLVo12Eb9SXhjyOn2P3lG/ZrX3uTwf/AIgl/Y19jVXFM2m/CWtuNcakGjtG5zOmHyj2NpTWhDvtzljC4N37tyO1VhoTUXEi5m8DZu1MpdxV/wBK+LtSjBWrMdGXNfA6Kd0hAdyjZ4cS0k7ghTyvNqjOPdj87pfDRYe1G+G06LMSWHFjmkEeLNZgcD2H0h0J+0vTo7hdX0VagdU1BnrlGtEYauNu3RJWgYdgGtAaHO5QABzudsOxc1dNeJiU1UzMRH9eeuFVxo3XGsRprhzqfJ6hGSi1Bfix1vHGjDFGGyNkDZGuaObnBYCevKdzs0LT66y2pdfcKsvqubOtqYGTKxQVsFHTjIMMWQZCHSSn0xIXM5uh2HZsrepcJsRQ0xpfBR2bpqaduRXaj3PZ4x74+flEh5NiPTO+wB7Oq0+T4BYbI+yMEeZztDFX7YvS4mrbYKomEgkLmtcwkAvbuW78vXoB025asDGnDzJm+jbOu3lr0CzUUPmzOumzPEWlcI+MOIa52flaSO4keSHb7W5Xic1r3fppPB7fCCX9jXo9bTsnhPsiZKScLZ3V+JeNDP8Ab1bEL/fbsx/5WD7/AL6ilCSzLSgfchjrW3MBlhhlMrGO26gPLWlwB7+Ub+oKxOCmBkvaju5x7SKtOF1KEkdHyvLXSEfzQ1o39bnDuK5cvrpoyXEmrvi3HUyp1rpREX5uoiIgIiICIiAiIgIiIC1GqNL0dXYl9C+x3ITzxzR7CSGQAgPYSDs4bnuIIJBBBIO3RZU1VUVRVTNpgc8Z7h5qTTcrg+g/MVAdm28a3ncR9nD9G0+83nHv9yj7hZYdn4zKMd/FfjbDSPtgs3C6nRfR4fTmLTFq6ImeBocr7z+1+S/F8/6Cbz+1+S/F8/6C6oRbfj1X2+f4LQ5X3n9r8l+L5/0E3n9r8l+L5/0F1QifHqvt8/wWhyvvP7X5L8Xz/oLyjjtzODYsVlZXHoGx42w4/wBzF1MifHavt8/wWhQumuFed1HKx1+F+Cxp6vfK5vlTx6mMG4b9tx3H8Uq78ViqmEx0FGjAytUgbyRxM7AP+JJ6knqSSSstF4uV5di5ZMZ+iI7o1KIiLz0f/9k=",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from IPython.display import display, Image\n",
    "\n",
    "display(Image(app.get_graph().draw_mermaid_png()))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "521adaef-dd2f-46d6-8f6a-5cc1d6e0aefc",
   "metadata": {},
   "source": [
    "## Stream outputs from the final node"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "84d65cbe-4cfe-44f8-b49e-b37632887c91",
   "metadata": {},
   "outputs": [],
   "source": [
    "import warnings\n",
    "from langchain_core._api import LangChainBetaWarning\n",
    "warnings.filterwarnings('ignore', category=LangChainBetaWarning)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5cfaeb64-5506-4546-96c0-4891e6288ad9",
   "metadata": {},
   "source": [
    "### Filter on event metadata"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f218a05d-1590-4d5c-b0b7-97d94c744efb",
   "metadata": {},
   "source": [
    "First option to get the LLM events from within a specific node (`final` node in our case) is to filter on the `langgraph_node` field in the event metadata. This will be sufficient in case you need to stream events from ALL LLM calls inside the node. This means that if you have multiple different LLMs invoked inside the node, this filter will include events from all of them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "a37c3a5f-5a43-46db-940e-c583df776520",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Well| folks|,| looks| like| we|'ve| got| some| cloudy| skies| in| the| Big| Apple| today|.| So| grab| your| umbrella| just| in| case|,| and| don|'t| let| those| clouds| rain| on| your| parade|!|"
     ]
    }
   ],
   "source": [
    "inputs = {\"messages\": [(\"human\", \"what's the weather in nyc?\")]}\n",
    "async for event in app.astream_events(inputs, version=\"v2\"):\n",
    "    kind = event[\"event\"]\n",
    "    tags = event.get(\"tags\", [])\n",
    "    # filter on the langgraph node name\n",
    "    if kind == \"on_chat_model_stream\" and event[\"metadata\"].get(\"langgraph_node\") == \"final\":\n",
    "        data = event[\"data\"]\n",
    "        if data[\"chunk\"].content:\n",
    "            # Empty content in the context of OpenAI or Anthropic usually means\n",
    "            # that the model is asking for a tool to be invoked.\n",
    "            # So we only print non-empty content\n",
    "            print(data[\"chunk\"].content, end=\"|\", flush=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b0bb447a-6650-4166-b124-2d5b99a1f88b",
   "metadata": {},
   "source": [
    "### Filter on custom tags"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ea4db927-44b6-46ab-8b8d-f237edaf1438",
   "metadata": {},
   "source": [
    "Alternatively, you can add configuration with custom tags to your LLM, like we did in the beginning, by adding `final_model.with_config(tags=[\"final_node\"])`. This will allow us to more precisely filter the events to keep the ones only from this model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "55d60dfa-96e3-442f-9924-0c99f46baed8",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Well| folks|,| looks| like| we|'ve| got| some| cloudy| skies| in| the| Big| Apple| today|.| So| grab| your| umbrella| just| in| case|,| and| don|'t| let| those| clouds| rain| on| your| parade|!|"
     ]
    }
   ],
   "source": [
    "inputs = {\"messages\": [(\"human\", \"what's the weather in nyc?\")]}\n",
    "async for event in app.astream_events(inputs, version=\"v2\"):\n",
    "    kind = event[\"event\"]\n",
    "    tags = event.get(\"tags\", [])\n",
    "    # filter on the custom tag\n",
    "    if kind == \"on_chat_model_stream\" and \"final_node\" in event.get(\"tags\", []):\n",
    "        data = event[\"data\"]\n",
    "        if data[\"chunk\"].content:\n",
    "            # Empty content in the context of OpenAI or Anthropic usually means\n",
    "            # that the model is asking for a tool to be invoked.\n",
    "            # So we only print non-empty content\n",
    "            print(data[\"chunk\"].content, end=\"|\", flush=True)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}