mirror of
https://github.com/langchain-ai/langgraph.git
synced 2026-09-13 13:17:52 +02:00
docs: add more streaming how-tos (#803)
This commit is contained in:
Vadym Barda
@@ -14,6 +14,9 @@ cloud_sdk_dir = docs_dir / "cloud"
|
||||
_MANUAL = {
|
||||
"how-tos": [
|
||||
"async.ipynb",
|
||||
"stream-values.ipynb",
|
||||
"stream-updates.ipynb",
|
||||
"stream-multiple.ipynb",
|
||||
"streaming-tokens.ipynb",
|
||||
"streaming-content.ipynb",
|
||||
"persistence.ipynb",
|
||||
|
||||
@@ -32,8 +32,11 @@ These guides cover common examples of that.
|
||||
LangGraph is built to be streaming first.
|
||||
These guides show how to use different streaming modes.
|
||||
|
||||
- [How to stream full state of your graph](stream-values.ipynb)
|
||||
- [How to stream state updates of your graph](stream-updates.ipynb)
|
||||
- [How to stream LLM tokens](streaming-tokens.ipynb)
|
||||
- [How to stream arbitrarily nested content](streaming-content.ipynb)
|
||||
- [How to configure multiple streaming modes at the same time](stream-multiple.ipynb)
|
||||
|
||||
## Other
|
||||
- [How to run graph asynchronously](async.ipynb)
|
||||
|
||||
+7
-4
@@ -145,8 +145,11 @@ nav:
|
||||
- View and update past graph state: how-tos/human_in_the_loop/time-travel.ipynb
|
||||
- Edit graph state: how-tos/human_in_the_loop/edit-graph-state.ipynb
|
||||
- Streaming:
|
||||
- Stream full state: how-tos/stream-values.ipynb
|
||||
- Stream state updates: how-tos/stream-updates.ipynb
|
||||
- Stream LLM tokens: how-tos/streaming-tokens.ipynb
|
||||
- Stream Arbitrarily Nested Content: how-tos/streaming-content.ipynb
|
||||
- Stream arbitrarily nested content: how-tos/streaming-content.ipynb
|
||||
- Configure multiple streaming modes: how-tos/stream-multiple.ipynb
|
||||
- Other:
|
||||
- Run graph asynchronously: how-tos/async.ipynb
|
||||
- Visualize your graph: how-tos/visualization.ipynb
|
||||
@@ -154,9 +157,9 @@ nav:
|
||||
- Use Pydantic model as state: how-tos/state-model.ipynb
|
||||
- Prebuilt ReAct Agent:
|
||||
- Create a ReAct agent: how-tos/create-react-agent.ipynb
|
||||
- Add Memory to a ReAct agent: how-tos/create-react-agent-memory.ipynb
|
||||
- Add a System Prompt to a ReAct agent: how-tos/create-react-agent-system-prompt.ipynb
|
||||
- Add Human-in-the-Loop to a ReAct agent: how-tos/create-react-agent-hitl.ipynb
|
||||
- Add memory to a ReAct agent: how-tos/create-react-agent-memory.ipynb
|
||||
- Add a system prompt to a ReAct agent: how-tos/create-react-agent-system-prompt.ipynb
|
||||
- Add human-in-the-Loop to a ReAct agent: how-tos/create-react-agent-hitl.ipynb
|
||||
- 'Conceptual Guides':
|
||||
- 'concepts/index.md'
|
||||
- LangGraph for Agentic Applications: concepts/high_level.md
|
||||
|
||||
@@ -29,7 +29,7 @@
|
||||
"id": "323db423-b644-40bd-9c2d-976a53f602f7",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"We'll be using a simple ReAct agent for this tutorial. You will also need to set up a project with `agent.py` and `langgraph.json` files. See [quick start](https://langchain-ai.github.io/langgraph/cloud/quick_start/#develop) for setting this up."
|
||||
"We'll be using a simple ReAct agent for this how-to guide. You will also need to set up a project with `agent.py` and `langgraph.json` files. See [quick start](https://langchain-ai.github.io/langgraph/cloud/quick_start/#develop) for setting this up."
|
||||
]
|
||||
},
|
||||
{
|
||||
|
||||
@@ -4,8 +4,8 @@
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# How to Stream Debug Events\n",
|
||||
"This notebook covers how to stream debug events from your graph (`stream_mode=\"debug\"`)."
|
||||
"# How to stream debug events\n",
|
||||
"This guide covers how to stream debug events from your graph (`stream_mode=\"debug\"`)."
|
||||
]
|
||||
},
|
||||
{
|
||||
|
||||
@@ -4,8 +4,8 @@
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# How to Stream Events\n",
|
||||
"This notebook covers how to stream events from your graph (`stream_mode=\"events\"`). Depending on the use case and user experience of your LangGraph application, your application may process event types differently."
|
||||
"# How to stream events\n",
|
||||
"This guide covers how to stream events from your graph (`stream_mode=\"events\"`). Depending on the use case and user experience of your LangGraph application, your application may process event types differently."
|
||||
]
|
||||
},
|
||||
{
|
||||
|
||||
@@ -7,14 +7,14 @@
|
||||
"source": [
|
||||
"# How to stream messages from your graph\n",
|
||||
"\n",
|
||||
"There are multiple different streaming modes.\n",
|
||||
"LangGraph Cloud supports multiple streaming modes. The main ones are:\n",
|
||||
"\n",
|
||||
"- `values`: This streaming mode streams back values of the graph. This is the **full state of the graph** after each node is called.\n",
|
||||
"- `updates`: This streaming mode streams back updates to the graph. This is the **update to the state of the graph** after each node is called.\n",
|
||||
"- `messages`: This streaming mode streams back messages - both complete messages (at the end of a node) as well as **tokens** for any messages generated inside a node. This mode is primarily meant for powering chat applications.\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"This notebook covers `stream_mode=\"messages\"`.\n",
|
||||
"This guide covers `stream_mode=\"messages\"`.\n",
|
||||
"\n",
|
||||
"In order to use this mode, the state of the graph you are interacting with MUST have a messages key that is a list of messages.\n",
|
||||
"Eg, the state should look something like:\n",
|
||||
|
||||
@@ -4,9 +4,9 @@
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# How to Configure Multiple Streaming Modes at the Same Time\n",
|
||||
"# How to configure multiple streaming modes at the same time\n",
|
||||
"\n",
|
||||
"This notebook covers how to configure multiple streaming modes at the same time."
|
||||
"This guide covers how to configure multiple streaming modes at the same time."
|
||||
]
|
||||
},
|
||||
{
|
||||
|
||||
@@ -5,16 +5,16 @@
|
||||
"id": "51466c8d-8ce4-4b3d-be4e-18fdbeda5f53",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# How to stream updates from your graph\n",
|
||||
"# How to stream state updates of your graph\n",
|
||||
"\n",
|
||||
"There are multiple different streaming modes.\n",
|
||||
"LangGraph Cloud supports multiple streaming modes. The main ones are:\n",
|
||||
"\n",
|
||||
"- `values`: This streaming mode streams back values of the graph. This is the **full state of the graph** after each node is called.\n",
|
||||
"- `updates`: This streaming mode streams back updates to the graph. This is the **update to the state of the graph** after each node is called.\n",
|
||||
"- `messages`: This streaming mode streams back messages - both complete messages (at the end of a node) as well as **tokens** for any messages generated inside a node. This mode is primarily meant for powering chat applications.\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"This notebook covers `stream_mode=\"updates\"`."
|
||||
"This guide covers `stream_mode=\"updates\"`."
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -170,7 +170,7 @@
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.11.9"
|
||||
"version": "3.12.3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
|
||||
@@ -5,16 +5,16 @@
|
||||
"id": "51466c8d-8ce4-4b3d-be4e-18fdbeda5f53",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# How to stream values from your graph\n",
|
||||
"# How to stream full state of your graph\n",
|
||||
"\n",
|
||||
"There are multiple different streaming modes.\n",
|
||||
"LangGraph Cloud supports multiple streaming modes. The main ones are:\n",
|
||||
"\n",
|
||||
"- `values`: This streaming mode streams back values of the graph. This is the **full state of the graph** after each node is called.\n",
|
||||
"- `updates`: This streaming mode streams back updates to the graph. This is the **update to the state of the graph** after each node is called.\n",
|
||||
"- `messages`: This streaming mode streams back messages - both complete messages (at the end of a node) as well as **tokens** for any messages generated inside a node. This mode is primarily meant for powering chat applications.\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"This notebook covers `stream_mode=\"values\"`."
|
||||
"This guide covers `stream_mode=\"values\"`."
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -204,7 +204,7 @@
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.11.9"
|
||||
"version": "3.12.3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
|
||||
@@ -0,0 +1,210 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "3631f2b9-aa79-472e-a9d6-9125a90ee704",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# How to configure multiple streaming modes at the same time"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "858c7499-0c92-40a9-bd95-e5a5a5817e92",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"This guide covers how to configure multiple streaming modes at the same time."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "7c2f84f1-0751-4779-97d4-5cbb286093b7",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Setup"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "323db423-b644-40bd-9c2d-976a53f602f7",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"We'll be using a simple ReAct agent for this guide."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"id": "6b4285e4-7434-4971-bde0-aabceef8ee7e",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"%%capture --no-stderr\n",
|
||||
"%pip install -U langgraph langchain-openai"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"id": "f7f9f24a-e3d0-422b-8924-47950b2facd6",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdin",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"OPENAI_API_KEY: ········\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"import getpass\n",
|
||||
"import os\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"def _set_env(var: str):\n",
|
||||
" if not os.environ.get(var):\n",
|
||||
" os.environ[var] = getpass.getpass(f\"{var}: \")\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"_set_env(\"OPENAI_API_KEY\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"id": "85cf2e23-29f2-40cc-b302-5377b3b49da9",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# this is all that's needed for the agent.py\n",
|
||||
"from typing import Literal\n",
|
||||
"from langchain_community.tools.tavily_search import TavilySearchResults\n",
|
||||
"from langchain_core.runnables import ConfigurableField\n",
|
||||
"from langchain_core.tools import tool\n",
|
||||
"from langchain_openai import ChatOpenAI\n",
|
||||
"from langgraph.prebuilt import create_react_agent\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"@tool\n",
|
||||
"def get_weather(city: Literal[\"nyc\", \"sf\"]):\n",
|
||||
" \"\"\"Use this to get weather information.\"\"\"\n",
|
||||
" if city == \"nyc\":\n",
|
||||
" return \"It might be cloudy in nyc\"\n",
|
||||
" elif city == \"sf\":\n",
|
||||
" return \"It's always sunny in sf\"\n",
|
||||
" else:\n",
|
||||
" raise AssertionError(\"Unknown city\")\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"tools = [get_weather]\n",
|
||||
"\n",
|
||||
"model = ChatOpenAI(model_name=\"gpt-4o\", temperature=0)\n",
|
||||
"graph = create_react_agent(model, tools)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "48a7751c-3f06-452b-89f4-70267e4dd305",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Stream multiple"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"id": "e9e9ffb0-2cd5-466f-b70b-b6ed51b852d1",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Receiving new event of type: debug...\n",
|
||||
"{'type': 'task', 'timestamp': '2024-06-25T16:12:29.144117+00:00', 'step': 1, 'payload': {'id': '8399d8fd-4b28-515a-b0e9-1679557c0953', 'name': 'agent', 'input': {'messages': [HumanMessage(content=\"what's the weather in sf\", id='44ff9154-9485-49c9-b679-791314cc19e3')], 'is_last_step': False}, 'triggers': ['start:agent']}}\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"Receiving new event of type: updates...\n",
|
||||
"{'agent': {'messages': [AIMessage(content='', additional_kwargs={'tool_calls': [{'id': 'call_gZEyPpcgwnzsnee1HH4geKmB', 'function': {'arguments': '{\"city\":\"sf\"}', 'name': 'get_weather'}, 'type': 'function'}]}, response_metadata={'token_usage': {'completion_tokens': 14, 'prompt_tokens': 57, 'total_tokens': 71}, 'model_name': 'gpt-4o-2024-05-13', 'system_fingerprint': 'fp_3e7d703517', 'finish_reason': 'tool_calls', 'logprobs': None}, id='run-37ca191f-f68f-4a70-8924-a40f90c8c0ed-0', tool_calls=[{'name': 'get_weather', 'args': {'city': 'sf'}, 'id': 'call_gZEyPpcgwnzsnee1HH4geKmB'}], usage_metadata={'input_tokens': 57, 'output_tokens': 14, 'total_tokens': 71})]}}\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"Receiving new event of type: debug...\n",
|
||||
"{'type': 'task_result', 'timestamp': '2024-06-25T16:12:29.802322+00:00', 'step': 1, 'payload': {'id': '8399d8fd-4b28-515a-b0e9-1679557c0953', 'name': 'agent', 'result': [('messages', [AIMessage(content='', additional_kwargs={'tool_calls': [{'id': 'call_gZEyPpcgwnzsnee1HH4geKmB', 'function': {'arguments': '{\"city\":\"sf\"}', 'name': 'get_weather'}, 'type': 'function'}]}, response_metadata={'token_usage': {'completion_tokens': 14, 'prompt_tokens': 57, 'total_tokens': 71}, 'model_name': 'gpt-4o-2024-05-13', 'system_fingerprint': 'fp_3e7d703517', 'finish_reason': 'tool_calls', 'logprobs': None}, id='run-37ca191f-f68f-4a70-8924-a40f90c8c0ed-0', tool_calls=[{'name': 'get_weather', 'args': {'city': 'sf'}, 'id': 'call_gZEyPpcgwnzsnee1HH4geKmB'}], usage_metadata={'input_tokens': 57, 'output_tokens': 14, 'total_tokens': 71})])]}}\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"Receiving new event of type: debug...\n",
|
||||
"{'type': 'task', 'timestamp': '2024-06-25T16:12:29.802738+00:00', 'step': 2, 'payload': {'id': 'f22971bf-6eff-55a2-84ab-fb97f629b133', 'name': 'tools', 'input': {'messages': [HumanMessage(content=\"what's the weather in sf\", id='44ff9154-9485-49c9-b679-791314cc19e3'), AIMessage(content='', additional_kwargs={'tool_calls': [{'id': 'call_gZEyPpcgwnzsnee1HH4geKmB', 'function': {'arguments': '{\"city\":\"sf\"}', 'name': 'get_weather'}, 'type': 'function'}]}, response_metadata={'token_usage': {'completion_tokens': 14, 'prompt_tokens': 57, 'total_tokens': 71}, 'model_name': 'gpt-4o-2024-05-13', 'system_fingerprint': 'fp_3e7d703517', 'finish_reason': 'tool_calls', 'logprobs': None}, id='run-37ca191f-f68f-4a70-8924-a40f90c8c0ed-0', tool_calls=[{'name': 'get_weather', 'args': {'city': 'sf'}, 'id': 'call_gZEyPpcgwnzsnee1HH4geKmB'}], usage_metadata={'input_tokens': 57, 'output_tokens': 14, 'total_tokens': 71})], 'is_last_step': False}, 'triggers': ['branch:agent:should_continue:tools']}}\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"Receiving new event of type: updates...\n",
|
||||
"{'tools': {'messages': [ToolMessage(content=\"It's always sunny in sf\", name='get_weather', tool_call_id='call_gZEyPpcgwnzsnee1HH4geKmB')]}}\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"Receiving new event of type: debug...\n",
|
||||
"{'type': 'task_result', 'timestamp': '2024-06-25T16:12:29.806676+00:00', 'step': 2, 'payload': {'id': 'f22971bf-6eff-55a2-84ab-fb97f629b133', 'name': 'tools', 'result': [('messages', [ToolMessage(content=\"It's always sunny in sf\", name='get_weather', tool_call_id='call_gZEyPpcgwnzsnee1HH4geKmB')])]}}\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"Receiving new event of type: debug...\n",
|
||||
"{'type': 'task', 'timestamp': '2024-06-25T16:12:29.807014+00:00', 'step': 3, 'payload': {'id': '3e1a91b9-b94c-56a7-ace5-6fd8ee73fe8d', 'name': 'agent', 'input': {'messages': [HumanMessage(content=\"what's the weather in sf\", id='44ff9154-9485-49c9-b679-791314cc19e3'), AIMessage(content='', additional_kwargs={'tool_calls': [{'id': 'call_gZEyPpcgwnzsnee1HH4geKmB', 'function': {'arguments': '{\"city\":\"sf\"}', 'name': 'get_weather'}, 'type': 'function'}]}, response_metadata={'token_usage': {'completion_tokens': 14, 'prompt_tokens': 57, 'total_tokens': 71}, 'model_name': 'gpt-4o-2024-05-13', 'system_fingerprint': 'fp_3e7d703517', 'finish_reason': 'tool_calls', 'logprobs': None}, id='run-37ca191f-f68f-4a70-8924-a40f90c8c0ed-0', tool_calls=[{'name': 'get_weather', 'args': {'city': 'sf'}, 'id': 'call_gZEyPpcgwnzsnee1HH4geKmB'}], usage_metadata={'input_tokens': 57, 'output_tokens': 14, 'total_tokens': 71}), ToolMessage(content=\"It's always sunny in sf\", name='get_weather', id='afc3ceaa-6663-4f7a-b874-e77e5515b175', tool_call_id='call_gZEyPpcgwnzsnee1HH4geKmB')], 'is_last_step': False}, 'triggers': ['tools']}}\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"Receiving new event of type: updates...\n",
|
||||
"{'agent': {'messages': [AIMessage(content='The weather in San Francisco is currently sunny.', response_metadata={'token_usage': {'completion_tokens': 10, 'prompt_tokens': 84, 'total_tokens': 94}, 'model_name': 'gpt-4o-2024-05-13', 'system_fingerprint': 'fp_3e7d703517', 'finish_reason': 'stop', 'logprobs': None}, id='run-575efeca-fdeb-4b4f-80f8-08ff177c34a5-0', usage_metadata={'input_tokens': 84, 'output_tokens': 10, 'total_tokens': 94})]}}\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"Receiving new event of type: debug...\n",
|
||||
"{'type': 'task_result', 'timestamp': '2024-06-25T16:12:30.355658+00:00', 'step': 3, 'payload': {'id': '3e1a91b9-b94c-56a7-ace5-6fd8ee73fe8d', 'name': 'agent', 'result': [('messages', [AIMessage(content='The weather in San Francisco is currently sunny.', response_metadata={'token_usage': {'completion_tokens': 10, 'prompt_tokens': 84, 'total_tokens': 94}, 'model_name': 'gpt-4o-2024-05-13', 'system_fingerprint': 'fp_3e7d703517', 'finish_reason': 'stop', 'logprobs': None}, id='run-575efeca-fdeb-4b4f-80f8-08ff177c34a5-0', usage_metadata={'input_tokens': 84, 'output_tokens': 10, 'total_tokens': 94})])]}}\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"inputs = {\"messages\": [(\"human\", \"what's the weather in sf\")]}\n",
|
||||
"async for event, chunk in graph.astream(inputs, stream_mode=[\"updates\", \"debug\"]):\n",
|
||||
" print(f\"Receiving new event of type: {event}...\")\n",
|
||||
" print(chunk)\n",
|
||||
" print(\"\\n\\n\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "8cc57240-243c-4d9a-a845-cb55ef973a59",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "langgraph",
|
||||
"language": "python",
|
||||
"name": "langgraph"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.12.3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
@@ -0,0 +1,186 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "3631f2b9-aa79-472e-a9d6-9125a90ee704",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# How to stream state updates of your graph"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "858c7499-0c92-40a9-bd95-e5a5a5817e92",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"LangGraph supports multiple streaming modes. The main ones are:\n",
|
||||
"\n",
|
||||
"- `values`: This streaming mode streams back values of the graph. This is the **full state of the graph** after each node is called.\n",
|
||||
"- `updates`: This streaming mode streams back updates to the graph. This is the **update to the state of the graph** after each node is called.\n",
|
||||
"\n",
|
||||
"This guide covers `stream_mode=\"updates\"`."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "7c2f84f1-0751-4779-97d4-5cbb286093b7",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Setup"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "323db423-b644-40bd-9c2d-976a53f602f7",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"We'll be using a simple ReAct agent for this guide."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"id": "6b4285e4-7434-4971-bde0-aabceef8ee7e",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"%%capture --no-stderr\n",
|
||||
"%pip install -U langgraph langchain-openai"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"id": "f7f9f24a-e3d0-422b-8924-47950b2facd6",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdin",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"OPENAI_API_KEY: ········\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"import getpass\n",
|
||||
"import os\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"def _set_env(var: str):\n",
|
||||
" if not os.environ.get(var):\n",
|
||||
" os.environ[var] = getpass.getpass(f\"{var}: \")\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"_set_env(\"OPENAI_API_KEY\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"id": "85cf2e23-29f2-40cc-b302-5377b3b49da9",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# this is all that's needed for the agent.py\n",
|
||||
"from typing import Literal\n",
|
||||
"from langchain_community.tools.tavily_search import TavilySearchResults\n",
|
||||
"from langchain_core.runnables import ConfigurableField\n",
|
||||
"from langchain_core.tools import tool\n",
|
||||
"from langchain_openai import ChatOpenAI\n",
|
||||
"from langgraph.prebuilt import create_react_agent\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"@tool\n",
|
||||
"def get_weather(city: Literal[\"nyc\", \"sf\"]):\n",
|
||||
" \"\"\"Use this to get weather information.\"\"\"\n",
|
||||
" if city == \"nyc\":\n",
|
||||
" return \"It might be cloudy in nyc\"\n",
|
||||
" elif city == \"sf\":\n",
|
||||
" return \"It's always sunny in sf\"\n",
|
||||
" else:\n",
|
||||
" raise AssertionError(\"Unknown city\")\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"tools = [get_weather]\n",
|
||||
"\n",
|
||||
"model = ChatOpenAI(model_name=\"gpt-4o\", temperature=0)\n",
|
||||
"graph = create_react_agent(model, tools)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "956db549-5207-4be1-a823-78311738e3f8",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Stream updates"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"id": "e9e9ffb0-2cd5-466f-b70b-b6ed51b852d1",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Receiving update from node: 'agent'\n",
|
||||
"{'messages': [AIMessage(content='', additional_kwargs={'tool_calls': [{'id': 'call_kc6cvcEkTAUGRlSHrP4PK9fn', 'function': {'arguments': '{\"city\":\"sf\"}', 'name': 'get_weather'}, 'type': 'function'}]}, response_metadata={'token_usage': {'completion_tokens': 14, 'prompt_tokens': 57, 'total_tokens': 71}, 'model_name': 'gpt-4o-2024-05-13', 'system_fingerprint': 'fp_3e7d703517', 'finish_reason': 'tool_calls', 'logprobs': None}, id='run-cd68b3a0-86c3-4afa-9649-1b962a0dd062-0', tool_calls=[{'name': 'get_weather', 'args': {'city': 'sf'}, 'id': 'call_kc6cvcEkTAUGRlSHrP4PK9fn'}], usage_metadata={'input_tokens': 57, 'output_tokens': 14, 'total_tokens': 71})]}\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"Receiving update from node: 'tools'\n",
|
||||
"{'messages': [ToolMessage(content=\"It's always sunny in sf\", name='get_weather', tool_call_id='call_kc6cvcEkTAUGRlSHrP4PK9fn')]}\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"Receiving update from node: 'agent'\n",
|
||||
"{'messages': [AIMessage(content='The weather in San Francisco is currently sunny.', response_metadata={'token_usage': {'completion_tokens': 10, 'prompt_tokens': 84, 'total_tokens': 94}, 'model_name': 'gpt-4o-2024-05-13', 'system_fingerprint': 'fp_3e7d703517', 'finish_reason': 'stop', 'logprobs': None}, id='run-009d83c4-b874-4acc-9494-20aba43132b9-0', usage_metadata={'input_tokens': 84, 'output_tokens': 10, 'total_tokens': 94})]}\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"inputs = {\"messages\": [(\"human\", \"what's the weather in sf\")]}\n",
|
||||
"async for chunk in graph.astream(inputs, stream_mode=\"updates\"):\n",
|
||||
" for node, values in chunk.items():\n",
|
||||
" print(f\"Receiving update from node: '{node}'\")\n",
|
||||
" print(values)\n",
|
||||
" print(\"\\n\\n\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "8cc57240-243c-4d9a-a845-cb55ef973a59",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "langgraph",
|
||||
"language": "python",
|
||||
"name": "langgraph"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.12.3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
@@ -0,0 +1,240 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "3631f2b9-aa79-472e-a9d6-9125a90ee704",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# How to stream full state of your graph"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "858c7499-0c92-40a9-bd95-e5a5a5817e92",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"LangGraph supports multiple streaming modes. The main ones are:\n",
|
||||
"\n",
|
||||
"- `values`: This streaming mode streams back values of the graph. This is the **full state of the graph** after each node is called.\n",
|
||||
"- `updates`: This streaming mode streams back updates to the graph. This is the **update to the state of the graph** after each node is called.\n",
|
||||
"\n",
|
||||
"This guide covers `stream_mode=\"values\"`."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "7c2f84f1-0751-4779-97d4-5cbb286093b7",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Setup"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "323db423-b644-40bd-9c2d-976a53f602f7",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"We'll be using a simple ReAct agent for this guide."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"id": "6b4285e4-7434-4971-bde0-aabceef8ee7e",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"%%capture --no-stderr\n",
|
||||
"%pip install -U langgraph langchain-openai"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"id": "f7f9f24a-e3d0-422b-8924-47950b2facd6",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdin",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"OPENAI_API_KEY: ········\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"import getpass\n",
|
||||
"import os\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"def _set_env(var: str):\n",
|
||||
" if not os.environ.get(var):\n",
|
||||
" os.environ[var] = getpass.getpass(f\"{var}: \")\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"_set_env(\"OPENAI_API_KEY\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"id": "ef5a3ec6-0cd0-4541-ab1b-d63ede22720e",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# this is all that's needed for the agent.py\n",
|
||||
"from typing import Literal\n",
|
||||
"from langchain_community.tools.tavily_search import TavilySearchResults\n",
|
||||
"from langchain_core.runnables import ConfigurableField\n",
|
||||
"from langchain_core.tools import tool\n",
|
||||
"from langchain_openai import ChatOpenAI\n",
|
||||
"from langgraph.prebuilt import create_react_agent\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"@tool\n",
|
||||
"def get_weather(city: Literal[\"nyc\", \"sf\"]):\n",
|
||||
" \"\"\"Use this to get weather information.\"\"\"\n",
|
||||
" if city == \"nyc\":\n",
|
||||
" return \"It might be cloudy in nyc\"\n",
|
||||
" elif city == \"sf\":\n",
|
||||
" return \"It's always sunny in sf\"\n",
|
||||
" else:\n",
|
||||
" raise AssertionError(\"Unknown city\")\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"tools = [get_weather]\n",
|
||||
"\n",
|
||||
"model = ChatOpenAI(model_name=\"gpt-4o\", temperature=0)\n",
|
||||
"graph = create_react_agent(model, tools)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "002a715b-e0be-4e89-8d42-f0098882586b",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Stream values"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"id": "e9e9ffb0-2cd5-466f-b70b-b6ed51b852d1",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"================================\u001b[1m Human Message \u001b[0m=================================\n",
|
||||
"\n",
|
||||
"what's the weather in sf\n",
|
||||
"==================================\u001b[1m Ai Message \u001b[0m==================================\n",
|
||||
"Tool Calls:\n",
|
||||
" get_weather (call_61VvIzqVGtyxcXi0z6knZkjZ)\n",
|
||||
" Call ID: call_61VvIzqVGtyxcXi0z6knZkjZ\n",
|
||||
" Args:\n",
|
||||
" city: sf\n",
|
||||
"=================================\u001b[1m Tool Message \u001b[0m=================================\n",
|
||||
"Name: get_weather\n",
|
||||
"\n",
|
||||
"It's always sunny in sf\n",
|
||||
"==================================\u001b[1m Ai Message \u001b[0m==================================\n",
|
||||
"\n",
|
||||
"The weather in San Francisco is currently sunny.\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"inputs = {\"messages\": [(\"human\", \"what's the weather in sf\")]}\n",
|
||||
"async for chunk in graph.astream(inputs, stream_mode=\"values\"):\n",
|
||||
" chunk[\"messages\"][-1].pretty_print()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "d73de237-bf45-4fa7-93ef-6dae7eacffc0",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"If we want to just get the final result, we can use the same method and just keep track of the last value we received"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"id": "c122bf15-a489-47bf-b482-a744a54e2cc4",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"inputs = {\"messages\": [(\"human\", \"what's the weather in sf\")]}\n",
|
||||
"async for chunk in graph.astream(inputs, stream_mode=\"values\"):\n",
|
||||
" final_result = chunk"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"id": "316022e5-4c65-48e4-9878-8d94a2425ed4",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"{'messages': [HumanMessage(content=\"what's the weather in sf\", id='54b39b6f-054b-4306-980b-86905e48a6bc'),\n",
|
||||
" AIMessage(content='', additional_kwargs={'tool_calls': [{'id': 'call_avoKnK8reERzTUSxrN9cgFxY', 'function': {'arguments': '{\"city\":\"sf\"}', 'name': 'get_weather'}, 'type': 'function'}]}, response_metadata={'token_usage': {'completion_tokens': 14, 'prompt_tokens': 57, 'total_tokens': 71}, 'model_name': 'gpt-4o-2024-05-13', 'system_fingerprint': 'fp_5e6c71d4a8', 'finish_reason': 'tool_calls', 'logprobs': None}, id='run-f2f43c89-2c96-45f4-975c-2d0f22d0d2d1-0', tool_calls=[{'name': 'get_weather', 'args': {'city': 'sf'}, 'id': 'call_avoKnK8reERzTUSxrN9cgFxY'}], usage_metadata={'input_tokens': 57, 'output_tokens': 14, 'total_tokens': 71}),\n",
|
||||
" ToolMessage(content=\"It's always sunny in sf\", name='get_weather', id='fc18a798-c7b2-4f73-84fa-8ffdffb6ddcb', tool_call_id='call_avoKnK8reERzTUSxrN9cgFxY'),\n",
|
||||
" AIMessage(content='The weather in San Francisco is currently sunny. Enjoy the sunshine!', response_metadata={'token_usage': {'completion_tokens': 14, 'prompt_tokens': 84, 'total_tokens': 98}, 'model_name': 'gpt-4o-2024-05-13', 'system_fingerprint': 'fp_5e6c71d4a8', 'finish_reason': 'stop', 'logprobs': None}, id='run-21418147-da8e-4738-a076-239377397c40-0', usage_metadata={'input_tokens': 84, 'output_tokens': 14, 'total_tokens': 98})]}"
|
||||
]
|
||||
},
|
||||
"execution_count": 6,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"final_result"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 7,
|
||||
"id": "0f64ebbe-535c-4b35-a95f-0a7490cfed90",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"==================================\u001b[1m Ai Message \u001b[0m==================================\n",
|
||||
"\n",
|
||||
"The weather in San Francisco is currently sunny. Enjoy the sunshine!\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"final_result[\"messages\"][-1].pretty_print()"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "langgraph",
|
||||
"language": "python",
|
||||
"name": "langgraph"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.12.3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
@@ -5,7 +5,7 @@
|
||||
"id": "51466c8d-8ce4-4b3d-be4e-18fdbeda5f53",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# How to stream graph responses\n",
|
||||
"# How to stream LLM tokens from your graph\n",
|
||||
"\n",
|
||||
"In this example we will stream tokens from the language model powering an agent. We will use a ReAct agent as an example. The main thing to bear in mind here is that using [async nodes](./async.ipynb) typically offers the best behavior for this, since we will be using the `astream_events` method.\n",
|
||||
"\n",
|
||||
@@ -99,7 +99,7 @@
|
||||
"id": "cd420984",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Set up the State\n",
|
||||
"## Set up the state\n",
|
||||
"\n",
|
||||
"The main type of graph in `langgraph` is the [StateGraph](https://langchain-ai.github.io/langgraph/reference/graphs/#langgraph.graph.StateGraph).\n",
|
||||
"This graph is parameterized by a `State` object that it passes around to each node.\n",
|
||||
|
||||
Reference in New Issue
Block a user