mirror of
https://github.com/langchain-ai/langgraph.git
synced 2026-09-28 04:25:08 +02:00
(wip) docs: revamp conceptual docs (#1663)
* docs: revamp conceptual docs * multi-agent draft * typo * Update Why LangGraph page * update multi-agent * Update persistence page * update * Clarification of graph re-playing * update persistence * more updates * more updates * Add multiple schemas section to glossary * more updates * Clarify I/O schema * dynamic breakpoints * Update agentic concepts * Fix comments * Update figureds * cr * cr * cr --------- Co-authored-by: Lance Martin <lance@langchain.dev> Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
This commit is contained in:
Vadym Barda
co-authored by
Lance Martin
Harrison Chase
Lance Martin
Harrison Chase
parent
182dcea526
commit
f8ba4c3b7d
@@ -0,0 +1,133 @@
|
||||
# Streaming
|
||||
|
||||
LangGraph is built with first class support for streaming. There are several different ways to stream back outputs from a graph run
|
||||
|
||||
## Streaming graph outputs (`.stream` and `.astream`)
|
||||
|
||||
`.stream` and `.astream` are sync and async methods for streaming back outputs from a graph run.
|
||||
There are several different modes you can specify when calling these methods (e.g. `graph.stream(..., mode="...")):
|
||||
|
||||
- [`"values"`](../how-tos/stream-values.ipynb): This streams the full value of the state after each step of the graph.
|
||||
- [`"updates"`](../how-tos/stream-updates.ipynb): This streams the updates to the state after each step of the graph. If multiple updates are made in the same step (e.g. multiple nodes are run) then those updates are streamed separately.
|
||||
- `"debug"`: This streams as much information as possible throughout the execution of the graph.
|
||||
|
||||
The below visualization shows the difference between the `values` and `updates` modes:
|
||||
|
||||
|
||||
|
||||
|
||||
## Streaming LLM tokens and events (`.astream_events`)
|
||||
|
||||
In addition, you can use the [`astream_events`](../how-tos/streaming-events-from-within-tools.ipynb) method to stream back events that happen _inside_ nodes. This is useful for [streaming tokens of LLM calls](../how-tos/streaming-tokens.ipynb).
|
||||
|
||||
This is a standard method on all [LangChain objects](https://python.langchain.com/v0.2/docs/concepts/#runnable-interface). This means that as the graph is executed, certain events are emitted along the way and can be seen if you run the graph using `.astream_events`.
|
||||
|
||||
All events have (among other things) `event`, `name`, and `data` fields. What do these mean?
|
||||
|
||||
- `event`: This is the type of event that is being emitted. You can find a detailed table of all callback events and triggers [here](https://python.langchain.com/v0.2/docs/concepts/#callback-events).
|
||||
- `name`: This is the name of event.
|
||||
- `data`: This is the data associated with the event.
|
||||
|
||||
What types of things cause events to be emitted?
|
||||
|
||||
* each node (runnable) emits `on_chain_start` when it starts execution, `on_chain_stream` during the node execution and `on_chain_end` when the node finishes. Node events will have the node name in the event's `name` field
|
||||
* the graph will emit `on_chain_start` in the beginning of the graph execution, `on_chain_stream` after each node execution and `on_chain_end` when the graph finishes. Graph events will have the `LangGraph` in the event's `name` field
|
||||
* Any writes to state channels (i.e. anytime you update the value of one of your state keys) will emit `on_chain_start` and `on_chain_end` events
|
||||
|
||||
Additionally, any events that are created inside your nodes (LLM events, tool events, manually emitted events, etc.) will also be visible in the output of `.astream_events`.
|
||||
|
||||
To make this more concrete and to see what this looks like, let's see what events are returned when we run a simple graph:
|
||||
|
||||
```python
|
||||
from langchain_openai import ChatOpenAI
|
||||
from langgraph.graph import StateGraph, MessagesState, START, END
|
||||
|
||||
model = ChatOpenAI(model="gpt-4o-mini")
|
||||
|
||||
|
||||
def call_model(state: MessagesState):
|
||||
response = model.invoke(state['messages'])
|
||||
return {"messages": response}
|
||||
|
||||
workflow = StateGraph(MessagesState)
|
||||
workflow.add_node(call_model)
|
||||
workflow.add_edge(START, "call_model")
|
||||
workflow.add_edge("call_model", END)
|
||||
app = workflow.compile()
|
||||
|
||||
inputs = [{"role": "user", "content": "hi!"}]
|
||||
async for event in app.astream_events({"messages": inputs}, version="v1"):
|
||||
kind = event["event"]
|
||||
print(f"{kind}: {event['name']}")
|
||||
```
|
||||
```shell
|
||||
on_chain_start: LangGraph
|
||||
on_chain_start: __start__
|
||||
on_chain_end: __start__
|
||||
on_chain_start: call_model
|
||||
on_chat_model_start: ChatOpenAI
|
||||
on_chat_model_stream: ChatOpenAI
|
||||
on_chat_model_stream: ChatOpenAI
|
||||
on_chat_model_stream: ChatOpenAI
|
||||
on_chat_model_stream: ChatOpenAI
|
||||
on_chat_model_stream: ChatOpenAI
|
||||
on_chat_model_stream: ChatOpenAI
|
||||
on_chat_model_stream: ChatOpenAI
|
||||
on_chat_model_stream: ChatOpenAI
|
||||
on_chat_model_stream: ChatOpenAI
|
||||
on_chat_model_stream: ChatOpenAI
|
||||
on_chat_model_stream: ChatOpenAI
|
||||
on_chat_model_end: ChatOpenAI
|
||||
on_chain_start: ChannelWrite<call_model,messages>
|
||||
on_chain_end: ChannelWrite<call_model,messages>
|
||||
on_chain_stream: call_model
|
||||
on_chain_end: call_model
|
||||
on_chain_stream: LangGraph
|
||||
on_chain_end: LangGraph
|
||||
```
|
||||
|
||||
We start with the overall graph start (`on_chain_start: LangGraph`). We then write to the `__start__` node (this is special node to handle input).
|
||||
We then start the `call_model` node (`on_chain_start: call_model`). We then start the chat model invocation (`on_chat_model_start: ChatOpenAI`),
|
||||
stream back token by token (`on_chat_model_stream: ChatOpenAI`) and then finish the chat model (`on_chat_model_end: ChatOpenAI`). From there,
|
||||
we write the results back to the channel (`ChannelWrite<call_model,messages>`) and then finish the `call_model` node and then the graph as a whole.
|
||||
|
||||
This should hopefully give you a good sense of what events are emitted in a simple graph. But what data do these events contain?
|
||||
Each type of event contains data in a different format. Let's look at what `on_chat_model_stream` events look like. This is an important type of event
|
||||
since it is needed for streaming tokens from an LLM response.
|
||||
|
||||
These events look like:
|
||||
|
||||
```shell
|
||||
{'event': 'on_chat_model_stream',
|
||||
'name': 'ChatOpenAI',
|
||||
'run_id': '3fdbf494-acce-402e-9b50-4eab46403859',
|
||||
'tags': ['seq:step:1'],
|
||||
'metadata': {'langgraph_step': 1,
|
||||
'langgraph_node': 'call_model',
|
||||
'langgraph_triggers': ['start:call_model'],
|
||||
'langgraph_task_idx': 0,
|
||||
'checkpoint_id': '1ef657a0-0f9d-61b8-bffe-0c39e4f9ad6c',
|
||||
'checkpoint_ns': 'call_model',
|
||||
'ls_provider': 'openai',
|
||||
'ls_model_name': 'gpt-4o-mini',
|
||||
'ls_model_type': 'chat',
|
||||
'ls_temperature': 0.7},
|
||||
'data': {'chunk': AIMessageChunk(content='Hello', id='run-3fdbf494-acce-402e-9b50-4eab46403859')},
|
||||
'parent_ids': []}
|
||||
```
|
||||
We can see that we have the event type and name (which we knew from before).
|
||||
|
||||
We also have a bunch of stuff in metadata. Noticeably, `'langgraph_node': 'call_model',` is some really helpful information
|
||||
which tells us which node this model was invoked inside of.
|
||||
|
||||
Finally, `data` is a really important field. This contains the actual data for this event! Which in this case
|
||||
is an AIMessageChunk. This contains the `content` for the message, as well as an `id`.
|
||||
This is the ID of the overall AIMessage (not just this chunk) and is super helpful - it helps
|
||||
us track which chunks are part of the same message (so we can show them together in the UI).
|
||||
|
||||
This information contains all that is needed for creating a UI for streaming LLM tokens. You can see a
|
||||
guide for that [here](../how-tos/streaming-tokens.ipynb).
|
||||
|
||||
|
||||
!!! warning "ASYNC IN PYTHON<=3.10"
|
||||
You may fail to see events being emitted from inside a node when using `.astream_events` in Python <= 3.10. If you're using a Langchain RunnableLambda, a RunnableGenerator, or Tool asynchronously inside your node, you will have to propagate callbacks to these objects manually. This is because LangChain cannot automatically propagate callbacks to child objects in this case. Please see examples [here](../how-tos/streaming-content.ipynb) and [here](../how-tos/streaming-events-from-within-tools.ipynb).
|
||||
Reference in New Issue
Block a user