WEBDIENSTE/langgraph
1
0
Fork 0
mirror of https://github.com/langchain-ai/langgraph.git synced 2026-08-29 03:09:45 +02:00
Code Issues Packages Projects Releases Wiki Activity

docs: remove astream events from streaming conceptual guide (#3438)

Browse Source
This commit is contained in:
Eugene Yurtsev
2025-02-13 21:07:30 -05:00
committed by GitHub
parent cd976e779d
commit b4f6cdf01f
1 changed files with 8 additions and 119 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/docs/concepts/streaming.md
+8 -119
View File
@@ -1,6 +1,12 @@
# Streaming
LangGraph is built with first class support for streaming. There are several different ways to stream back outputs from a graph run
Building a responsive app for end-users? Real-time updates are key to keeping users engaged as your app progresses.
There are three main types of data you’ll want to stream:
1. Workflow progress (e.g., get state updates after each graph node is executed).
2. LLM tokens as they’re generated.
3. Custom updates (e.g., "Fetched 10/100 records").
## Streaming graph outputs (`.stream` and `.astream`)
@@ -31,123 +37,6 @@ The below visualization shows the difference between the `values` and `updates`
![values vs updates](../static/values_vs_updates.png)
## Streaming LLM tokens and events (`.astream_events`)
In addition, you can use the `astream_events` method to stream back events that happen _inside_ nodes. This is useful for [streaming tokens of LLM calls](../how-tos/streaming-tokens.ipynb).
This is a standard method on all [LangChain objects](https://python.langchain.com/docs/concepts/#runnable-interface). This means that as the graph is executed, certain events are emitted along the way and can be seen if you run the graph using `.astream_events`.
All events have (among other things) `event`, `name`, and `data` fields. What do these mean?
- `event`: This is the type of event that is being emitted. You can find a detailed table of all callback events and triggers [here](https://python.langchain.com/docs/concepts/#callback-events).
- `name`: This is the name of event.
- `data`: This is the data associated with the event.
What types of things cause events to be emitted?
* each node (runnable) emits `on_chain_start` when it starts execution, `on_chain_stream` during the node execution and `on_chain_end` when the node finishes. Node events will have the node name in the event's `name` field
* the graph will emit `on_chain_start` in the beginning of the graph execution, `on_chain_stream` after each node execution and `on_chain_end` when the graph finishes. Graph events will have the `LangGraph` in the event's `name` field
* Any writes to state channels (i.e. anytime you update the value of one of your state keys) will emit `on_chain_start` and `on_chain_end` events
Additionally, any events that are created inside your nodes (LLM events, tool events, manually emitted events, etc.) will also be visible in the output of `.astream_events`.
To make this more concrete and to see what this looks like, let's see what events are returned when we run a simple graph:
```python
from langchain_openai import ChatOpenAI
from langgraph.graph import StateGraph, MessagesState, START, END
model = ChatOpenAI(model="gpt-4o-mini")
def call_model(state: MessagesState):
response = model.invoke(state['messages'])
return {"messages": response}
workflow = StateGraph(MessagesState)
workflow.add_node(call_model)
workflow.add_edge(START, "call_model")
workflow.add_edge("call_model", END)
app = workflow.compile()
inputs = [{"role": "user", "content": "hi!"}]
async for event in app.astream_events({"messages": inputs}, version="v1"):
kind = event["event"]
print(f"{kind}: {event['name']}")
```
```shell
on_chain_start: LangGraph
on_chain_start: __start__
on_chain_end: __start__
on_chain_start: call_model
on_chat_model_start: ChatOpenAI
on_chat_model_stream: ChatOpenAI
on_chat_model_stream: ChatOpenAI
on_chat_model_stream: ChatOpenAI
on_chat_model_stream: ChatOpenAI
on_chat_model_stream: ChatOpenAI
on_chat_model_stream: ChatOpenAI
on_chat_model_stream: ChatOpenAI
on_chat_model_stream: ChatOpenAI
on_chat_model_stream: ChatOpenAI
on_chat_model_stream: ChatOpenAI
on_chat_model_stream: ChatOpenAI
on_chat_model_end: ChatOpenAI
on_chain_start: ChannelWrite<call_model,messages>
on_chain_end: ChannelWrite<call_model,messages>
on_chain_stream: call_model
on_chain_end: call_model
on_chain_stream: LangGraph
on_chain_end: LangGraph
```
We start with the overall graph start (`on_chain_start: LangGraph`). We then write to the `__start__` node (this is special node to handle input).
We then start the `call_model` node (`on_chain_start: call_model`). We then start the chat model invocation (`on_chat_model_start: ChatOpenAI`),
stream back token by token (`on_chat_model_stream: ChatOpenAI`) and then finish the chat model (`on_chat_model_end: ChatOpenAI`). From there,
we write the results back to the channel (`ChannelWrite<call_model,messages>`) and then finish the `call_model` node and then the graph as a whole.
This should hopefully give you a good sense of what events are emitted in a simple graph. But what data do these events contain?
Each type of event contains data in a different format. Let's look at what `on_chat_model_stream` events look like. This is an important type of event
since it is needed for streaming tokens from an LLM response.
These events look like:
```shell
{'event': 'on_chat_model_stream',
'name': 'ChatOpenAI',
'run_id': '3fdbf494-acce-402e-9b50-4eab46403859',
'tags': ['seq:step:1'],
'metadata': {'langgraph_step': 1,
'langgraph_node': 'call_model',
'langgraph_triggers': ['start:call_model'],
'langgraph_task_idx': 0,
'checkpoint_id': '1ef657a0-0f9d-61b8-bffe-0c39e4f9ad6c',
'checkpoint_ns': 'call_model',
'ls_provider': 'openai',
'ls_model_name': 'gpt-4o-mini',
'ls_model_type': 'chat',
'ls_temperature': 0.7},
'data': {'chunk': AIMessageChunk(content='Hello', id='run-3fdbf494-acce-402e-9b50-4eab46403859')},
'parent_ids': []}
```
We can see that we have the event type and name (which we knew from before).
We also have a bunch of stuff in metadata. Noticeably, `'langgraph_node': 'call_model',` is some really helpful information
which tells us which node this model was invoked inside of.
Finally, `data` is a really important field. This contains the actual data for this event! Which in this case
is an AIMessageChunk. This contains the `content` for the message, as well as an `id`.
This is the ID of the overall AIMessage (not just this chunk) and is super helpful - it helps
us track which chunks are part of the same message (so we can show them together in the UI).
This information contains all that is needed for creating a UI for streaming LLM tokens. You can see a
guide for that [here](../how-tos/streaming-tokens.ipynb).
!!! warning "ASYNC IN PYTHON<=3.10"
You may fail to see events being emitted from inside a node when using `.astream_events` in Python <= 3.10. If you're using a Langchain RunnableLambda, a RunnableGenerator, or Tool asynchronously inside your node, you will have to propagate callbacks to these objects manually. This is because LangChain cannot automatically propagate callbacks to child objects in this case.
## LangGraph Platform
Streaming is critical for making LLM applications feel responsive to end users. When creating a streaming run, the streaming mode determines what data is streamed back to the API client. LangGraph Platform supports five streaming modes:
@@ -155,8 +44,8 @@ Streaming is critical for making LLM applications feel responsive to end users.
- `values`: Stream the full state of the graph after each [super-step](https://langchain-ai.github.io/langgraph/concepts/low_level/#graphs) is executed. See the [how-to guide](../cloud/how-tos/stream_values.md) for streaming values.
- `messages-tuple`: Stream LLM tokens for any messages generated inside a node. This mode is primarily meant for powering chat applications. See the [how-to guide](../cloud/how-tos/stream_messages.md) for streaming messages.
- `updates`: Streams updates to the state of the graph after each node is executed. See the [how-to guide](../cloud/how-tos/stream_updates.md) for streaming updates.
- `events`: Stream all events (including the state of the graph) that occur during graph execution. See the [how-to guide](../cloud/how-tos/stream_events.md) for streaming events. This can be used to do token-by-token streaming for LLMs.
- `debug`: Stream debug events throughout graph execution. See the [how-to guide](../cloud/how-tos/stream_debug.md) for streaming debug events.
- `events`: Stream all events (including the state of the graph) that occur during graph execution. See the [how-to guide](../cloud/how-tos/stream_events.md) for streaming events. This mode is only useful for users migrating large LCEL applications to LangGraph. Generally, this mode is not necessary for most applications.
You can also specify multiple streaming modes at the same time. See the [how-to guide](../cloud/how-tos/stream_multiple.md) for configuring multiple streaming modes at the same time.