From 52945478ef9c36c19df15411854dd126ffb466b1 Mon Sep 17 00:00:00 2001
From: William FH <13333726+hinthornw@users.noreply.github.com>
Date: Tue, 4 Jun 2024 10:02:14 -0700
Subject: [PATCH] [Docs] Update stream_tokens to clarify how in older versions
of python (#587)
---
examples/streaming-tokens.ipynb | 30 ++++++++++++++++++++++++++----
1 file changed, 26 insertions(+), 4 deletions(-)
diff --git a/examples/streaming-tokens.ipynb b/examples/streaming-tokens.ipynb
index 40ef406a9..abefd51eb 100644
--- a/examples/streaming-tokens.ipynb
+++ b/examples/streaming-tokens.ipynb
@@ -16,6 +16,14 @@
" \n",
" In this how-to, we will create our agent from scratch to be transparent (but verbose). You can accomplish similar functionality using the create_react_agent(model, tools=tool) (API doc) constructor. This may be more appropriate if you are used to LangChain’s AgentExecutor class.\n",
"
\n",
+ " \n",
+ "\n",
+ "\n",
+ "
Note on Python < 3.11
\n",
+ "
\n",
+ " When using python 3.8, 3.9, or 3.10, please ensure you manually pass the RunnableConfig through to the llm when invoking it like so: llm.ainvoke(..., config).\n",
+ " The astream_events method collects all events from your nested code using a streaming tracer passed as a callback. In 3.11 and above, this is automatically handled via contextvar's; prior to 3.11, asyncio's tasks lacked proper contextvar support, meaning that the callbacks will only propagate if you manually pass the config through. We do this in the call_model method below.\n",
+ "
\n",
"
\n",
+ "
Manual Callback Propagation
\n",
+ "
\n",
+ " Note that in call_model(state: State, config: RunnableConfig): below, we a) accept the RunnableConfig in the node and b) pass this in as the second arg for llm.ainvoke(..., config). This is optional for python 3.11 and later. If you ever have a problem where the LLM tokens are not streamed when using `astream_events` and you are using an older version of python, it's worth checking to ensure that the callbacks are manually propagated.
\n",
+ "