Update README.md

Sync readmes
2026-10-13 19:55:23 +02:00 · 2024-07-11 15:42:32 -07:00 · 2024-07-11 15:42:16 -07:00 · 2024-07-11 15:16:58 -07:00 · 2024-07-11 15:14:12 -07:00 · 2024-07-11 14:33:25 -07:00
47 changed files with 3652 additions and 274 deletions
@@ -57,8 +57,6 @@ export LANGSMITH_API_KEY=lsv2_sk_...
 ```

 ```python
-from typing import Annotated, Literal, TypedDict
-
 from langchain_core.messages import HumanMessage
 from langchain_anthropic import ChatAnthropic
 from langchain_core.tools import tool
@@ -84,7 +82,7 @@ tool_node = ToolNode(tools)
 model = ChatAnthropic(model="claude-3-5-sonnet-20240620", temperature=0).bind_tools(tools)

 # Define the function that determines whether to continue or not
-def should_continue(state: MessagesState) -> Literal["tools", END]:
+def should_continue(state: MessagesState) -> Union[Literal["tools"], type(END)]:
    messages = state['messages']
    last_message = messages[-1]
    # If the LLM makes a tool call, then we route to the "tools" node
@@ -21,8 +21,10 @@ _MANUAL = {
        "stream-updates.ipynb",
        "stream-multiple.ipynb",
        "streaming-tokens.ipynb",
+        "streaming-tokens-without-langchain.ipynb",
        "streaming-content.ipynb",
        "streaming-events-from-within-tools.ipynb",
+        "streaming-events-from-within-tools-without-langchain.ipynb",
        "streaming-from-final-node.ipynb",
        "persistence.ipynb",
        "memory/manage-conversation-history.ipynb",
@@ -30,6 +32,7 @@ _MANUAL = {
        "memory/add-summary-conversation-history.ipynb",
        "persistence_postgres.ipynb",
        "persistence_mongodb.ipynb",
+        "persistence_redis.ipynb",
        "visualization.ipynb",
        "state-model.ipynb",
        "subgraph.ipynb",
@@ -63,7 +63,7 @@ There are many occasions where the graph cannot run completely autonomously. For

 ### Double Texting

-Many times users might interact with your graph in unintended ways. For instance, a user may send one message and before the graph has finished running send a second message. To solve this issue of "double-texting" (i.e. prompting the graph a second time before the first run has finished), Langgraph has provided four different solutions, all of which are covered in the [Double Texting how-tos](../how-tos/index.md#double-texting). These options are:
+Many times users might interact with your graph in unintended ways. For instance, a user may send one message and before the graph has finished running send a second message. To solve this issue of "double-texting" (i.e. prompting the graph a second time before the first run has finished), LangGraph has provided four different solutions, all of which are covered in the [Double Texting how-tos](../how-tos/index.md#double-texting). These options are:

 - `reject`: This is the simplest option, this just rejects any follow up runs and does not allow double texting. See the [how-to guide](../how-tos/reject_concurrent.md) for configuring the reject double text option.
 - `enqueue`: This is a relatively simple option which continues the first run until it completes the whole run, then sends the new input as a separate run. See the [how-to guide](../how-tos/enqueue_concurrent.md) for configuring the enqueue double text option.
@@ -101,3 +101,7 @@ The LangGraph Cloud offers several features to support secure and robost deploym
 ### Authentication

 LangGraph applications deployed to LangGraph Cloud are automatically configured with LangSmith authentication. In order to call the API, a valid <a href="https://docs.smith.langchain.com/how_to_guides/setup/create_account_api_key#api-keys" target="_blank">LangSmith API key</a> is required.
+
+### Local Testing
+
+Before deploying your app in production to LangGraph Cloud, you may wish to test out your graph locally in order to ensure that everything is running as expected. Luckily, LangGraph makes this easy for you through use of the LangGraph CLI. Read more in this [how-to guide](../deployment/test_locally.md) or look at the [CLI reference](../reference/cli.md) to learn more.
@@ -0,0 +1,28 @@
+# Cloud Concepts
+
+This page describes the high-level concepts of the LangGraph Cloud deployment.
+
+## Deployment
+
+A deployment is an instance of a LangGraph API. A single deployment can have many [revisions](#revision). When a deployment is created, all of the necessary infrastructure (e.g. database, containers, secrets store) are automatically provisioned. See the [architecture diagram](#architecture) below for more details.
+
+See the [how-to guide](../deployment/cloud.md#create-new-deployment) for creating a new deployment.
+
+## Revision
+
+A revision is an iteration of a [deployment](#deployment). When a new deployment is created, an initial revision is automatically created. To deploy new code changes or update environment variable configurations for a deployment, a new revision must be created. When a revision is created, a new container image is built automatically.
+
+See the [how-to guide](../deployment/cloud.md#create-new-revision) for creating a new revision.
+
+## Asynchronous Deployment
+
+Infrastructure for [deployments](#deployment) and [revisions](#revision) are provisioned and deployed asynchronously. They are not deployed immediately after submission. Currently, deployment can take up to several minutes.
+
+## Architecture
+
+!!! warning "Subject to Change"
+    The LangGraph Cloud deployment architecture may change in the future.
+
+A high-level diagram of a LangGraph Cloud deployment.
+
+![diagram](langgraph_cloud_architecture.png)
@@ -45,8 +45,33 @@ Starting from the <a href="https://smith.langchain.com/" target="_blank">LangSmi
        1. Update the value of existing secrets or environment variables.
 1. Select `Submit`. After a few seconds, the `New Revision` modal will close and the new revision will be queued for deployment.

-## Asynchronous Deployment
+## View Build and Deployment Logs

-New [deployments](#create-new-deployment) and [revisions](#create-new-revision) are provisioned and deployed asynchronously. They are not deployed immediately after submission. Currently, deployment can take up to several minutes.
+Build and deployment logs are available for each revision.

-The `Deployment` view continually updates the status of pending revisions.
+Starting from the `Deployment` view...
+
+1. Select the desired revision from the `Revisions` table. A panel slides open from the right-hand side and the `Build` tab is selected by default, which displays build logs for the revision.
+1. In the panel, select the `Deploy` tab to view deployment logs for the revision.
+1. Within the `Deploy` tab, adjust the date/time range picker as needed. By default, the date/time range picker is set to the `Last 15 minutes`.
+
+## Interrupt Revision
+
+Interrupting a revision will stop deployment of the revision.
+
+!!! warning "Undefined Behavior"
+    Interrupted revisions have undefined behavior. This is only useful if you need to deploy a new revision and you already have a revision "stuck" in progress. In the future, this feature may be removed.
+
+Starting from the `Deployment` view...
+
+1. Select the menu icon (three dots) on the right-hand side of the row for the desired revision from the `Revisions` table.
+1. Select `Interrupt` from the menu.
+1. A modal will appear. Review the confirmation message. Select `Interrupt revision`.
+
+## Delete Deployment
+
+Starting from the <a href="https://smith.langchain.com/" target="_blank">LangSmith UI</a>...
+
+1. In the left-hand navigation panel, select `Deployments`. The `Deployments` view contains a list of existing LangGraph Cloud deployments.
+1. Select the menu icon (three dots) on the right-hand side of the row for the desired deployment and select `Delete`.
+1. A `Confirmation` modal will appear. Select `Delete`.
@@ -101,6 +101,9 @@ Example `langgraph.json` file:

 Note that the variable name of the `CompiledGraph` appears at the end of the value of each subkey in the top-level `graphs` key (i.e. `:<variable_name>`).

+!!! warning "Configuration Location"
+    The LangGraph API configuration file must be placed in a directory that is at the same level or higher than the Python files that contain compiled graphs and associated dependencies.
+
 Example file directory:

 ```bash
@@ -0,0 +1,91 @@
+# How to test a LangGraph app locally
+
+This guide assumes you have a LangGraph app correctly set up with a proper configuration file and a corresponding compiled graph, and that you have a proper LangChain API key.
+
+## Setup
+
+Install the proper packages:
+
+```shell
+pip install langgraph-cli
+```
+
+## Start the API server
+
+Once you have downloaded the CLI, you can run the following command to start the API server for local testing:
+
+```shell
+langgraph up
+```
+
+This will start up the LangGraph API server locally. If this runs successfully, you should see something like:
+
+```shell
+Ready!
+- API: http://localhost:8123
+2024-06-26 19:20:41,056:INFO:uvicorn.access 127.0.0.1:44138 - "GET /ok HTTP/1.1" 200
+```
+
+### Interact with the server
+
+We can now interact with the API server using the LangGraph SDK. First, we need to start our client, select our assistant (in this case a graph we called "agent", make sure to select the proper assistant you wish to test).
+
+=== "Python"
+
+    ```python
+    from langgraph_sdk import get_client
+
+    # only pass the url argument to get_client() if you changed the default port when calling langgraph up
+    client = get_client()
+    assistant_id = "agent"
+    thread = await client.threads.create()
+    ```
+
+=== "Javascript"
+
+    ```js
+    import { Client } from "@langchain/langgraph-sdk";
+
+    // only set the apiUrl if you changed the default port when calling langgraph up
+    const client = new Client();
+    const assistantId = "agent"
+    const thread = await client.threads.create();
+    ```
+
+Now we can invoke our graph to ensure it is working. Make sure to change the input to match the proper schema for your graph. 
+
+=== "Python"
+
+    ```python
+    input = {"messages": [{"role": "human", "content": "what's the weather in sf"}]}
+    async for chunk in client.runs.stream(
+        thread["thread_id"],
+        assistant_id,
+        input=input,
+        stream_mode="updates",
+    ):
+        print(f"Receiving new event of type: {chunk.event}...")
+        print(chunk.data)
+        print("\n\n")
+    ```
+=== "Javascript"
+
+    ```js
+    const input = { "messages": [{ "role": "human", "content": "what's the weather in sf"}] }
+
+    const streamResponse = client.runs.stream(
+      thread["thread_id"],
+      assistantId,
+      {
+        input: input,
+        streamMode: "updates",
+      }
+    );
+    for await (const chunk of streamResponse) {
+      console.log(`Receiving new event of type: ${chunk.event}...`);
+      console.log(chunk.data);
+      console.log("\n\n");
+    }
+    ```
+
+If your graph works correctly, you should see your graph output displayed in the console. Of course, there are many more ways you might need to test your graph, for a full list of commands you can send with the SDK, see the [Python](https://langchain-ai.github.io/langgraph/cloud/reference/sdk/python_sdk_ref/) and [JS/TS](https://langchain-ai.github.io/langgraph/cloud/reference/sdk/js_ts_sdk_ref/) references.
@@ -1,6 +1,6 @@
 ## Enqueue

-This guide assumes knowledge of what double-texting is, which you can learn about in the [double-texting conceptual guide](../concepts/index.md#double-texting).
+This guide assumes knowledge of what double-texting is, which you can learn about in the [double-texting conceptual guide](../concepts/api.md#double-texting).

 The guide covers the `enqueue` option for double texting, which adds the interruptions to a queue and executes them in the order they are received by the client. Below is a quick example of using the `enqueue` option.

@@ -43,7 +43,7 @@ We can do this by adding `interrupt_before=["action"]`, which tells us to interr
 We can do this either when compiling the graph or when kicking off a run.
 Here we will do it when kicking of a run, if you would like to to do it at compile time you need to edit the python file where your graph is defined and add the `interrupt_before` parameter when you call `.compile`.

-First let's access our hosted Langgraph instance through the SDK:
+First let's access our hosted LangGraph instance through the SDK:

 And, now let's compile it with a breakpoint before the tool node:

@@ -1,6 +1,6 @@
 # How to Replay and Branch from Prior States

-With Langgraph Cloud you have the ability to return to any of your prior states and either re-run the graph to reproduce issues noticed during testing, or branch out in a different way from what was originally done in the prior states. In this guide we will show a quick example of how to rerun past states and how to branch off from previous states as well.
+With LangGraph Cloud you have the ability to return to any of your prior states and either re-run the graph to reproduce issues noticed during testing, or branch out in a different way from what was originally done in the prior states. In this guide we will show a quick example of how to rerun past states and how to branch off from previous states as well.

 ## Setup

@@ -1,6 +1,6 @@
 ## Interrupt

-This guide assumes knowledge of what double-texting is, which you can learn about in the [double-texting conceptual guide](../concepts/index.md#double-texting).
+This guide assumes knowledge of what double-texting is, which you can learn about in the [double-texting conceptual guide](../concepts/api.md#double-texting).

 The guide covers the `interrupt` option for double texting, which interrupts the prior run of the graph and starts a new one with the double-text. This option does not delete the first run, but rather keeps it in the database but sets its status to `interrupted`. Below is a quick example of using the `interrupt` option.

@@ -12,7 +12,7 @@

 LangGraph Cloud is a managed service for deploying and hosting LangGraph applications. Deploying applications with LangGraph Cloud shortens the time-to-market for developers. With one click, deploy a production-ready API with built-in persistence for your LangGraph application. LangGraph Cloud APIs are horizontally scalable and deployed with durable storage.

-The LangGraph Cloud API exposes functionality of your LangGraph application through [Assistants](./concepts/index.md#assistants). An assistant abstracts the cognitive architecture of your graph. Invoke an assistant by calling the pre-built [API endpoints](./reference/api/api_ref.md).
+The LangGraph Cloud API exposes functionality of your LangGraph application through [Assistants](./concepts/api.md#assistants). An assistant abstracts the cognitive architecture of your graph. Invoke an assistant by calling the pre-built [API endpoints](./reference/api/api_ref.md).

 LangGraph Cloud is seamlessly integrated with [LangSmith](https://www.langchain.com/langsmith) and is accessible from within the LangSmith UI.

@@ -31,5 +31,5 @@ The LangGraph Cloud API supports key LangGraph features in addition to new funct

 - [Tutorials](./quick_start.md): Learn to build and deploy applications for LangGraph Cloud.
 - [How-to Guides](./how-tos/index.md): Learn how to set up a LangGraph application for deployment and implement features of the LangGraph Cloud API such as streaming tokens, configuring double texting, and creating cron jobs. Go here if you want to copy and run a specific code snippet.
- [Conceptual Guides](./concepts/index.md): In-depth explanations of the core data models (e.g. assistants) and key features (e.g. double texting) of the LangGraph Cloud API.
+- [Conceptual Guides](./concepts/api.md): In-depth explanations of the core data models (e.g. assistants), key features of the LangGraph Cloud API (e.g. double texting), and the architecture of a LangGraph Cloud deployment.
 - [Reference](./reference/api/api_ref.md): References for the LangGraph Cloud API, the corresponding Python and JS/TS SDKs, the LangGraph CLI, and deployment environment variables.
@@ -74,13 +74,13 @@ In order to do this we can first install the LangGraph CLI
 pip install langgraph-cli
 ```

-We can then stand up a simple test server. The server this stands up is INCREDIBLY simple - it is just a single endpoint and has no persistence. **This should not be used for hosting your application, only for testing the build and basic functionality.**
+We can then test our API server locally. This requires access to LangGraph closed beta. In order to run the server locally, you will need to add your `LANGSMITH_API_KEY` to the .env file so we can validate you have access to LangGraph closed beta.

 ```shell
-langgraph test
+langgraph up
 ```

-This will test building of the agent server. If this runs successfully, you should see something like:
+This will start up the LangGraph API server locally. If this runs successfully, you should see something like:

 ```shell
 Ready!
@@ -88,7 +88,7 @@ Ready!
 2024-06-26 19:20:41,056:INFO:uvicorn.access 127.0.0.1:44138 - "GET /ok HTTP/1.1" 200
 ```

-You can now test this out! Again, we only expose a single simple endpoint (for streaming stateless runs). This is intended to allow you to test that the agent is properly set up, but should **NOT** but used for production purposes. To test it out, you can go to another terminal window and run:
+You can now test this out! **Note: this local server is intended SOLELY for local testing purposes and is not performant enough for production applications, so please do not use it as such.** To test it out, you can go to another terminal window and run:

 ```shell
 curl --request POST \
@@ -186,14 +186,12 @@ Once you have tested that your hosted graph works as expected using LangGraph St

 First, make sure you have the SDK installed by calling `pip install langgraph_sdk`.

-Before using, you need to get the URL of your LangGraph deployment. You can find this on the auto generated documentation page here:
-
-![Base URL of LangGraph deployment](./deployment/img/base_url.png)
+Before using, you need to get the URL of your LangGraph deployment. You can find this in the `Deployment` view. Click the URL to copy it to the clipboard.

 You also need to make sure you have set up your API key properly so you can authenticate with LangGraph Cloud.

 ```shell
-export LANGCHAIN_API_KEY=...
+export LANGSMITH_API_KEY=...
 ```

 The first thing to do when using the SDK is to setup our client, access our assistant, and create a thread to execute a run on:
@@ -83,6 +83,31 @@ langgraph build [OPTIONS]
 | `-c, --config FILE` | `langgraph.json` | Path to configuration file declaring dependencies, graphs and environment variables. |
 | `--help` | | Display command documentation. |

+### `up`
+Start langgraph API server. For local testing, requires a LangSmith API key with access to LangGraph Cloud closed beta. Requires a license key for production use.
+
+**Usage**
+```
+langgraph up [OPTIONS]
+```
+
+**Options**
+
+| Option | Default | Description |
+| ------ | ------- | ----------- |
+| `--wait` | | Wait for services to start before returning. Implies --detach |
+| `--postgres-uri TEXT` | Local database | Postgres URI to use for the database. |
+| `--watch` | | Restart on file changes |
+| `--debugger-base-url TEXT` | `http://127.0.0.1:[PORT]` | URL used by the debugger to access LangGraph API. |
+| `--debugger-port INTEGER` | | Pull the debugger image locally and serve the UI on specified port |
+| `--verbose` | | Show more output from the server logs. |
+| `-c, --config FILE` | `langgraph.json` | Path to configuration file declaring dependencies, graphs and environment variables. |
+| `-d, --docker-compose FILE` | | Path to docker-compose.yml file with additional services to launch. |
+| `-p, --port INTEGER` | `8123` | Port to expose. Example: `langgraph test --port 8000` |
+| `--pull / --no-pull` | `pull` | Pull latest images. Use --no-pull for running the server with locally-built images. Example: `langgraph up --no-pull` |
+| `--recreate / --no-recreate` | `no-recreate` | Recreate containers even if their configuration and image haven't changed |
+| `--help` | | Display command documentation. | 
+
 ### `test`
 Test your LangGraph in the cloud. The only function you can call from the SDK after testing your graph is `client.runs.stream(thread_id=None, ...)`

@@ -14,7 +14,9 @@ By composing `Nodes` and `Edges`, you can create complex, looping workflows that

 In short: _nodes do the work. edges tell what to do next_.

-LangGraph's underlying graph algorithm uses [message passing](https://en.wikipedia.org/wiki/Message_passing) to define a general program. When a `Node` completes, it sends a message along one or more edges to other node(s). These nodes run their functions, pass the resulting messages to the next set of nodes, and on and on it goes. Inspired by [Pregel](https://research.google/pubs/pregel-a-system-for-large-scale-graph-processing/), the program proceeds in discrete "super-steps" that are all executed conceptually in parallel. Whenever the graph is run, all the nodes start in an `inactive` state. Whenever an incoming edge (or "channel") receives a new message (state), the node becomes `active`, runs the function, and responds with updates. At the end of each superstep, each node votes to `halt` by marking itself as `inactive` if it has no more incoming messages. The graph terminates when all nodes are `inactive` and when no messages are in transit.
+LangGraph's underlying graph algorithm uses [message passing](https://en.wikipedia.org/wiki/Message_passing) to define a general program. When a Node completes its operation, it sends messages along one or more edges to other node(s). These recipient nodes then execute their functions, pass the resulting messages to the next set of nodes, and the process continues. Inspired by Google's [Pregel](https://research.google/pubs/pregel-a-system-for-large-scale-graph-processing/) system, the program proceeds in discrete "super-steps."
+
+A super-step can be considered a single iteration over the graph nodes. Nodes that run in parallel are part of the same super-step, while nodes that run sequentially belong to separate super-steps. At the start of graph execution, all nodes begin in an `inactive` state. A node becomes `active` when it receives a new message (state) on any of its incoming edges (or "channels"). The active node then runs its function and responds with updates. At the end of each super-step, nodes with no incoming messages vote to `halt` by marking themselves as `inactive`. The graph execution terminates when all nodes are `inactive` and no messages are in transit.

 ### StateGraph

@@ -232,11 +234,9 @@ graph.add_conditional_edges("node_a", continue_to_jokes)

 ## Checkpointer

-One of the main benefits of LangGraph is that it comes backed by a persistence layer. This is accomplished via [checkpointers][basecheckpointsaver].
+LangGraph has a built-in persistence layer, implemented through [checkpointers][basecheckpointsaver]. When you use a checkpointer with a graph, you can interact with the state of that graph. When you use a checkpointer with a graph, you can interact with and manage the graph's state. The checkpointer saves a _checkpoint_ of the graph state at every super-step, enabling several powerful capabilities:

-Checkpointers can be used to save a _checkpoint_ of the state of a graph after all steps of the graph. This allows for several things.
-
-First, it allows for [human-in-the-loop workflows](agentic_concepts.md#human-in-the-loop), as it allows humans to inspect, interrupt, and approve steps. Checkpointers are needed for these workflows as the human has to be able to view the state of a graph at any point in time, and the graph has to be to resume execution after the human has made any updates to the state.
+First, checkpointers facilitate [human-in-the-loop workflows](agentic_concepts.md#human-in-the-loop) workflows by allowing humans to inspect, interrupt, and approve steps.Checkpointers are needed for these workflows as the human has to be able to view the state of a graph at any point in time, and the graph has to be to resume execution after the human has made any updates to the state.

 Second, it allows for ["memory"](agentic_concepts.md#memory) between interactions. You can use checkpointers to create threads and save the state of a thread after a graph executes. In the case of repeated human interactions (like conversations) any follow up messages can be sent to that checkpoint, which will retain its memory of previous ones.

@@ -244,8 +244,7 @@ See [this guide](../how-tos/persistence.ipynb) for how to add a checkpointer to

 ## Threads

-When using a checkpointer, you must specify a `thread_id` or `thread_ts` when running the graph.
-Threads are used to checkpoint multiple different runs. This can be used to enable a multi-tenant chat applications.
+Threads enable the checkpointing of multiple different runs, making them essential for multi-tenant chat applications and other scenarios where maintaining separate states is necessary. A thread is a unique ID assigned to a series of checkpoints saved by a checkpointer. When using a checkpointer, you must specify a `thread_id` or `thread_ts` when running the graph.

 `thread_id` is simply the ID of a thread. This is always required

@@ -262,14 +261,7 @@ See [this guide](../how-tos/persistence.ipynb) for how to use threads.

 ## Checkpointer state

-When you use a checkpointer with a graph, you can interact with the state of that graph.
-This usually done when enabling different human-in-the-loop interaction patterns.
-Each time you run the graph, the checkpointer creates several checkpoints every time a
-node or set of nodes finishes running.
-The most recent checkpoint is the current state of the thread.
-When interacting with the checkpointer state, you must specify a [thread identifier](#threads).
-
-Each checkpoint has two properties:
+ When interacting with the checkpointer state, you must specify a [thread identifier](#threads).Each checkpoint saved by the checkpointer has two properties:

 - **values**: This is the value of the state at this point in time.
 - **next**: This is a tuple of the nodes to execute next in the graph.
@@ -27,6 +27,7 @@ LangGraph makes it easy to persist state across graph runs. The guide below show
 - [How to add summary conversation memory](memory/add-summary-conversation-history.ipynb)
 - [How to create a custom checkpointer using Postgres](persistence_postgres.ipynb)
 - [How to create a custom checkpointer using MongoDB](persistence_mongodb.ipynb)
+- [How to create a custom checkpointer using Redis](persistence_redis.ipynb)

 ## Human in the Loop

@@ -46,9 +47,11 @@ These guides show how to use different streaming modes.
 - [How to stream full state of your graph](stream-values.ipynb)
 - [How to stream state updates of your graph](stream-updates.ipynb)
 - [How to stream LLM tokens](streaming-tokens.ipynb)
+- [How to stream LLM tokens without LangChain models](streaming-tokens-without-langchain.ipynb)
 - [How to stream arbitrarily nested content](streaming-content.ipynb)
 - [How to configure multiple streaming modes at the same time](stream-multiple.ipynb)
 - [How to stream events from within a tool](streaming-events-from-within-tools.ipynb)
+- [How to stream events from within a tool without LangChain models](streaming-events-from-within-tools-without-langchain.ipynb)
 - [How to stream events from the final node](streaming-from-final-node.ipynb)

 ## Other
@@ -138,6 +138,7 @@ nav:
          - Add summary of the conversation history: how-tos/memory/add-summary-conversation-history.ipynb
          - Create custom checkpointer using Postgres: how-tos/persistence_postgres.ipynb
          - Create custom checkpointer using MongoDB: how-tos/persistence_mongodb.ipynb
+          - Create custom checkpointer using Redis: how-tos/persistence_redis.ipynb
      - Human-in-the-loop:
          - Add breakpoints: how-tos/human_in_the_loop/breakpoints.ipynb
          - Wait for user input: how-tos/human_in_the_loop/wait-user-input.ipynb
@@ -147,9 +148,11 @@ nav:
          - Stream full state: how-tos/stream-values.ipynb
          - Stream state updates: how-tos/stream-updates.ipynb
          - Stream LLM tokens: how-tos/streaming-tokens.ipynb
+          - Stream LLM tokens without LangChain models: how-tos/streaming-tokens-without-langchain.ipynb
          - Stream arbitrarily nested content: how-tos/streaming-content.ipynb
          - Configure multiple streaming modes: how-tos/stream-multiple.ipynb
          - Stream events from within tools: how-tos/streaming-events-from-within-tools.ipynb
+          - Stream events from within tools without LangChain models: how-tos/streaming-events-from-within-tools-without-langchain.ipynb
          - Stream events from the final node: how-tos/streaming-from-final-node.ipynb
      - Other:
          - Run graph asynchronously: how-tos/async.ipynb
@@ -182,6 +185,7 @@ nav:
          - Deployment:
              - Setup App: "cloud/deployment/setup.md"
              - Setup App (pyproject.toml): "cloud/deployment/setup_pyproject.md"
+              - Test App Locally: "cloud/deployment/test_locally.md"
              - Deploy to Cloud: "cloud/deployment/cloud.md"
              - Self-Host: "cloud/deployment/self_hosted.md"
          - Streaming:
@@ -214,7 +218,9 @@ nav:
              - Configure Agents: "cloud/how-tos/cloud_examples/configuration_cloud.ipynb"
              - Convert LangGraph calls to LangGraph Cloud calls: "cloud/how-tos/cloud_examples/langgraph_to_langgraph_cloud.ipynb"
              - Integrate Webhooks: 'cloud/how-tos/cloud_examples/webhooks.ipynb'
-      - Conceptual Guides: "cloud/concepts/index.md"
+      - Conceptual Guides:
+          - API Concepts: "cloud/concepts/api.md"
+          - Cloud Concepts: "cloud/concepts/cloud.md"
      - Reference:
          - API: "cloud/reference/api/api_ref.md"
          - SDK:
@@ -2,10 +2,739 @@
 "cells": [
  {
   "cell_type": "markdown",
-   "id": "4c37bb65-6e2c-42e4-bfa7-9df10e2652a0",
+   "id": "d9d1a28b-c2a1-4246-b1c2-c58d6938f798",
   "metadata": {},
   "source": [
-    "This example has moved! Check out the [Customer Support Tutorial](../customer-support/customer-support.ipynb) for more information."
+    "# Customer Support\n",
+    "\n",
+    "Here, we show an example of building a customer support chatbot.\n",
+    "\n",
+    "This customer support chatbot interacts with SQL database to answer questions.\n",
+    "We will use a mock SQL database to get started: the [Chinook](https://www.sqlitetutorial.net/sqlite-sample-database/) database.\n",
+    "This database is about sales from a music store: what songs and album exists, customer orders, things like that.\n",
+    "\n",
+    "This chatbot has two different states: \n",
+    "1. Music: the user can inquire about different songs and albums present in the store\n",
+    "2. Account: the user can ask questions about their account\n",
+    "\n",
+    "Under the hood, this is handled by two separate agents. \n",
+    "Each has a specific prompt and tools related to their objective. \n",
+    "There is also a generic agent who is responsible for routing between these two agents as needed.\n",
+    "\n",
+    "Note: This is a very simple example! For a more complete tutorial on building a customer support bot, check out the [Customer Support Tutorial](../customer-support/customer-support.ipynb) for more information."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "35abc013-2613-4a49-a806-939dcf13ccf3",
+   "metadata": {
+    "scrolled": true
+   },
+   "outputs": [],
+   "source": [
+    "%%capture --no-stderr\n",
+    "%pip install -U langgraph langchain-community langchain-openai scikit-learn"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "9431e7f1-07fa-49d9-ac45-29613703dcc1",
+   "metadata": {},
+   "source": [
+    "## Load the data"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "3d1ef253-6b0c-4481-868c-e1fe84f2c8ff",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import requests\n",
+    "\n",
+    "url = \"https://storage.googleapis.com/benchmarks-artifacts/chinook/Chinook.db\"\n",
+    "response = requests.get(url)\n",
+    "\n",
+    "with open(\"Chinook.db\", \"wb\") as file:\n",
+    "    file.write(response.content)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "61f7ef9c",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "['Album',\n",
+       " 'Artist',\n",
+       " 'Customer',\n",
+       " 'Employee',\n",
+       " 'Genre',\n",
+       " 'Invoice',\n",
+       " 'InvoiceLine',\n",
+       " 'MediaType',\n",
+       " 'Playlist',\n",
+       " 'PlaylistTrack',\n",
+       " 'Track']"
+      ]
+     },
+     "execution_count": 2,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "from langchain_community.utilities import SQLDatabase\n",
+    "\n",
+    "db = SQLDatabase.from_uri(\"sqlite:///Chinook.db\")\n",
+    "db.get_usable_table_names()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "1cf668e4-8cb4-4de1-bc5e-c90284bf74bc",
+   "metadata": {},
+   "source": [
+    "## Load an LLM\n",
+    "\n",
+    "We will load a language model to use.\n",
+    "For this demo we will use OpenAI."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "d9ea4e80-30e6-4d46-b480-35f0be2fb055",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain_openai import ChatOpenAI\n",
+    "\n",
+    "model = ChatOpenAI(temperature=0, streaming=True, model=\"gpt-4o\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "73907422-7e05-431e-b06d-256c9ec1f6f6",
+   "metadata": {},
+   "source": [
+    "## Load Other Modules\n",
+    "\n",
+    "Load other modules we will use.\n",
+    "\n",
+    "All of the tools our agents will use will be custom tools. As such, we will use the `@tool` decorator to create custom tools.\n",
+    "\n",
+    "We will pass in messages to the agent, so we load `HumanMessage` and `SystemMessage`"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "ea958e9f-ab1f-49b5-bd85-16332055297c",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain_core.messages import HumanMessage, SystemMessage"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "35271d4d-2a1c-41be-9359-a3a7c3fed3d9",
+   "metadata": {},
+   "source": [
+    "## Define the Customer Agent\n",
+    "\n",
+    "This agent is responsible for looking up customer information.\n",
+    "It will have a specific prompt as well a specific tool to look up information about that customer (after asking for their user id)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "id": "975b039a",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# This tool is given to the agent to look up information about a customer\n",
+    "def get_customer_info(customer_id: int):\n",
+    "    \"\"\"Look up customer info given their ID. ALWAYS make sure you have the customer ID before invoking this.\"\"\"\n",
+    "    return db.run(f\"SELECT * FROM Customer WHERE CustomerID = {customer_id};\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "id": "1d5fa446",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "customer_prompt = \"\"\"Your job is to help a user update their profile.\n",
+    "\n",
+    "You only have certain tools you can use. These tools require specific input. If you don't know the required input, then ask the user for it.\n",
+    "\n",
+    "If you are unable to help the user, you can \"\"\"\n",
+    "\n",
+    "\n",
+    "def get_customer_messages(messages):\n",
+    "    return [SystemMessage(content=customer_prompt)] + messages\n",
+    "\n",
+    "\n",
+    "customer_chain = get_customer_messages | model.bind_tools([get_customer_info])"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "904a9485-3857-458e-8b9d-33bc33842bc9",
+   "metadata": {},
+   "source": [
+    "## Define the Music Agent\n",
+    "\n",
+    "This agent is responsible for figuring out information about music. To do that, we will create a prompt and various tools for looking up information about music\n",
+    "\n",
+    "First, we will create indexes for looking up artists and track names.\n",
+    "This will allow us to look up artists and tracks without having to spell their names exactly right."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "id": "a8604a3b-b484-4b2b-a914-4236cb98c524",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain_community.vectorstores import SKLearnVectorStore\n",
+    "from langchain_openai import OpenAIEmbeddings\n",
+    "\n",
+    "artists = db._execute(\"select * from Artist\")\n",
+    "songs = db._execute(\"select * from Track\")\n",
+    "artist_retriever = SKLearnVectorStore.from_texts(\n",
+    "    [a[\"Name\"] for a in artists], OpenAIEmbeddings(), metadatas=artists\n",
+    ").as_retriever()\n",
+    "song_retriever = SKLearnVectorStore.from_texts(\n",
+    "    [a[\"Name\"] for a in songs], OpenAIEmbeddings(), metadatas=songs\n",
+    ").as_retriever()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "ac7eb264-c572-4925-ad55-a1d52a18b1c0",
+   "metadata": {},
+   "source": [
+    "First, let's create a tool for getting albums by artist."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "id": "0a2a2b74",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def get_albums_by_artist(artist):\n",
+    "    \"\"\"Get albums by an artist (or similar artists).\"\"\"\n",
+    "    docs = artist_retriever.get_relevant_documents(artist)\n",
+    "    artist_ids = \", \".join([str(d.metadata[\"ArtistId\"]) for d in docs])\n",
+    "    return db.run(\n",
+    "        f\"SELECT Title, Name FROM Album LEFT JOIN Artist ON Album.ArtistId = Artist.ArtistId WHERE Album.ArtistId in ({artist_ids});\",\n",
+    "        include_columns=True,\n",
+    "    )"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "45e85066-f2fc-490e-992d-cd66c9cd6486",
+   "metadata": {},
+   "source": [
+    "Next, lets create a tool for getting tracks by an artist"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "id": "da533f50",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def get_tracks_by_artist(artist):\n",
+    "    \"\"\"Get songs by an artist (or similar artists).\"\"\"\n",
+    "    docs = artist_retriever.invoke(artist)\n",
+    "    artist_ids = \", \".join([str(d.metadata[\"ArtistId\"]) for d in docs])\n",
+    "    return db.run(\n",
+    "        f\"SELECT Track.Name as SongName, Artist.Name as ArtistName FROM Album LEFT JOIN Artist ON Album.ArtistId = Artist.ArtistId LEFT JOIN Track ON Track.AlbumId = Album.AlbumId WHERE Album.ArtistId in ({artist_ids});\",\n",
+    "        include_columns=True,\n",
+    "    )"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "bb0e50ab-b059-427c-924b-f8072d8db23c",
+   "metadata": {},
+   "source": [
+    "Finally, let's create a tool for looking up songs by their name."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "id": "b3c07010",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def check_for_songs(song_title):\n",
+    "    \"\"\"Check if a song exists by its name.\"\"\"\n",
+    "    return song_retriever.invoke(song_title)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "88388ff8-38b5-4e4e-a24d-de8c3670bd2b",
+   "metadata": {},
+   "source": [
+    "Create the chain to call the relevant tools"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 11,
+   "id": "72a14d5c",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "song_system_message = \"\"\"Your job is to help a customer find any songs they are looking for. \n",
+    "\n",
+    "You only have certain tools you can use. If a customer asks you to look something up that you don't know how, politely tell them what you can help with.\n",
+    "\n",
+    "When looking up artists and songs, sometimes the artist/song will not be found. In that case, the tools will return information \\\n",
+    "on similar songs and artists. This is intentional, it is not the tool messing up.\"\"\"\n",
+    "\n",
+    "\n",
+    "def get_song_messages(messages):\n",
+    "    return [SystemMessage(content=song_system_message)] + messages\n",
+    "\n",
+    "\n",
+    "song_recc_chain = get_song_messages | model.bind_tools(\n",
+    "    [get_albums_by_artist, get_tracks_by_artist, check_for_songs]\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 12,
+   "id": "cff15eb0-62c7-451d-a5f9-4576b24c879e",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "AIMessage(content='', additional_kwargs={'tool_calls': [{'index': 0, 'id': 'call_aXa9rSRXvTCJabrMY6AqkSV8', 'function': {'arguments': '{\"artist\":\"Amy Winehouse\"}', 'name': 'get_tracks_by_artist'}, 'type': 'function'}]}, response_metadata={'finish_reason': 'tool_calls', 'model_name': 'gpt-4o-2024-05-13', 'system_fingerprint': 'fp_ce0793330f'}, id='run-60633269-fd02-43e8-b434-28e3b8b69fb1-0', tool_calls=[{'name': 'get_tracks_by_artist', 'args': {'artist': 'Amy Winehouse'}, 'id': 'call_aXa9rSRXvTCJabrMY6AqkSV8'}])"
+      ]
+     },
+     "execution_count": 12,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "msgs = [HumanMessage(content=\"hi! can you help me find songs by amy whinehouse?\")]\n",
+    "song_recc_chain.invoke(msgs)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "0a42c293-0816-4f3c-b4a3-5b9f3a0665d1",
+   "metadata": {},
+   "source": [
+    "## Define the Generic Agent\n",
+    "\n",
+    "We now define a generic agent that is responsible for handling initial inquiries and routing to the right sub agent."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 13,
+   "id": "73e74268",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain_core.messages import AIMessage, HumanMessage, SystemMessage\n",
+    "from langchain_core.pydantic_v1 import BaseModel, Field\n",
+    "\n",
+    "\n",
+    "class Router(BaseModel):\n",
+    "    \"\"\"Call this if you are able to route the user to the appropriate representative.\"\"\"\n",
+    "\n",
+    "    choice: str = Field(description=\"should be one of: music, customer\")\n",
+    "\n",
+    "\n",
+    "system_message = \"\"\"Your job is to help as a customer service representative for a music store.\n",
+    "\n",
+    "You should interact politely with customers to try to figure out how you can help. You can help in a few ways:\n",
+    "\n",
+    "- Updating user information: if a customer wants to update the information in the user database. Call the router with `customer`\n",
+    "- Recommending music: if a customer wants to find some music or information about music. Call the router with `music`\n",
+    "\n",
+    "If the user is asking or wants to ask about updating or accessing their information, send them to that route.\n",
+    "If the user is asking or wants to ask about music, send them to that route.\n",
+    "Otherwise, respond.\"\"\"\n",
+    "\n",
+    "\n",
+    "def get_messages(messages):\n",
+    "    return [SystemMessage(content=system_message)] + messages"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 14,
+   "id": "ddf27314",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "chain = get_messages | model.bind_tools([Router])"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 15,
+   "id": "3c896f34",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "AIMessage(content='', additional_kwargs={'tool_calls': [{'index': 0, 'id': 'call_0aaFPPCWDiAoPyXQX2PS8TcJ', 'function': {'arguments': '{\"choice\":\"music\"}', 'name': 'Router'}, 'type': 'function'}]}, response_metadata={'finish_reason': 'tool_calls', 'model_name': 'gpt-4o-2024-05-13', 'system_fingerprint': 'fp_ce0793330f'}, id='run-73d51d75-b7c5-49fe-b558-105ede7c75d1-0', tool_calls=[{'name': 'Router', 'args': {'choice': 'music'}, 'id': 'call_0aaFPPCWDiAoPyXQX2PS8TcJ'}])"
+      ]
+     },
+     "execution_count": 15,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "msgs = [HumanMessage(content=\"hi! can you help me find a good song?\")]\n",
+    "chain.invoke(msgs)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 16,
+   "id": "40d86f59",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "AIMessage(content='', additional_kwargs={'tool_calls': [{'index': 0, 'id': 'call_Okla9DfMHIPs5TslS6KPaoBA', 'function': {'arguments': '{\"choice\":\"customer\"}', 'name': 'Router'}, 'type': 'function'}]}, response_metadata={'finish_reason': 'tool_calls', 'model_name': 'gpt-4o-2024-05-13', 'system_fingerprint': 'fp_4008e3b719'}, id='run-0d82d4f8-f4eb-4b16-add8-3e7fdffd6332-0', tool_calls=[{'name': 'Router', 'args': {'choice': 'customer'}, 'id': 'call_Okla9DfMHIPs5TslS6KPaoBA'}])"
+      ]
+     },
+     "execution_count": 16,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "msgs = [HumanMessage(content=\"hi! what's the email you have for me?\")]\n",
+    "chain.invoke(msgs)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 17,
+   "id": "bd6ddd8b-7500-46a7-811d-3bcb937bda51",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain_core.messages import AIMessage\n",
+    "\n",
+    "\n",
+    "def add_name(message, name):\n",
+    "    _dict = message.dict()\n",
+    "    _dict[\"name\"] = name\n",
+    "    return AIMessage(**_dict)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 18,
+   "id": "27494de5-8345-4c23-bc0e-81e0dd5d47d8",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import json\n",
+    "\n",
+    "from langgraph.graph import END\n",
+    "\n",
+    "\n",
+    "def _get_last_ai_message(messages):\n",
+    "    for m in messages[::-1]:\n",
+    "        if isinstance(m, AIMessage):\n",
+    "            return m\n",
+    "    return None\n",
+    "\n",
+    "\n",
+    "def _is_tool_call(msg):\n",
+    "    return hasattr(msg, \"additional_kwargs\") and \"tool_calls\" in msg.additional_kwargs\n",
+    "\n",
+    "\n",
+    "def _route(messages):\n",
+    "    last_message = messages[-1]\n",
+    "    if isinstance(last_message, AIMessage):\n",
+    "        if not last_message.tool_calls:\n",
+    "            return END\n",
+    "        else:\n",
+    "            if last_message.name == \"general\":\n",
+    "                if len(last_message.tool_calls) > 1:\n",
+    "                    raise ValueError(\"Too many tools\")\n",
+    "                return last_message.tool_calls[0][\"args\"][\"choice\"]\n",
+    "            else:\n",
+    "                return \"tools\"\n",
+    "    last_m = _get_last_ai_message(messages)\n",
+    "    if last_m is None:\n",
+    "        return \"general\"\n",
+    "    if last_m.name == \"music\":\n",
+    "        return \"music\"\n",
+    "    elif last_m.name == \"customer\":\n",
+    "        return \"customer\"\n",
+    "    else:\n",
+    "        return \"general\""
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 19,
+   "id": "8aec704a-46fe-4fb3-bdee-11c3bbffc370",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langgraph.prebuilt import ToolNode\n",
+    "\n",
+    "tools = [get_albums_by_artist, get_tracks_by_artist, check_for_songs, get_customer_info]\n",
+    "tool_node = ToolNode(tools)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 20,
+   "id": "4d5b75c6-73e0-4922-a765-a15be63f869e",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def _filter_out_routes(messages):\n",
+    "    ms = []\n",
+    "    for m in messages:\n",
+    "        if _is_tool_call(m):\n",
+    "            if m.name == \"general\":\n",
+    "                continue\n",
+    "        ms.append(m)\n",
+    "    return ms"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 21,
+   "id": "fd4dbf98-dbb3-411a-bad6-2bb334072aaf",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from functools import partial\n",
+    "\n",
+    "general_node = _filter_out_routes | chain | partial(add_name, name=\"general\")\n",
+    "music_node = _filter_out_routes | song_recc_chain | partial(add_name, name=\"music\")\n",
+    "customer_node = _filter_out_routes | customer_chain | partial(add_name, name=\"customer\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 22,
+   "id": "dcade924",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langgraph.checkpoint.sqlite import SqliteSaver\n",
+    "\n",
+    "from langgraph.graph import MessageGraph\n",
+    "\n",
+    "memory = SqliteSaver.from_conn_string(\":memory:\")\n",
+    "graph = MessageGraph()\n",
+    "nodes = {\n",
+    "    \"general\": \"general\",\n",
+    "    \"music\": \"music\",\n",
+    "    END: END,\n",
+    "    \"tools\": \"tools\",\n",
+    "    \"customer\": \"customer\",\n",
+    "}\n",
+    "# Define a new graph\n",
+    "workflow = MessageGraph()\n",
+    "workflow.add_node(\"general\", general_node)\n",
+    "workflow.add_node(\"music\", music_node)\n",
+    "workflow.add_node(\"customer\", customer_node)\n",
+    "workflow.add_node(\"tools\", tool_node)\n",
+    "workflow.add_conditional_edges(\"general\", _route, nodes)\n",
+    "workflow.add_conditional_edges(\"tools\", _route, nodes)\n",
+    "workflow.add_conditional_edges(\"music\", _route, nodes)\n",
+    "workflow.add_conditional_edges(\"customer\", _route, nodes)\n",
+    "workflow.set_conditional_entry_point(_route, nodes)\n",
+    "graph = workflow.compile()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 23,
+   "id": "ac65d6d2",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdin",
+     "output_type": "stream",
+     "text": [
+      "User (q/Q to quit):  what music do you have?\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Output from node 'general':\n",
+      "---\n",
+      "content='' additional_kwargs={'tool_calls': [{'index': 0, 'id': 'call_iste6NuKvZou8O9QudOectOU', 'function': {'arguments': '{\"choice\":\"music\"}', 'name': 'Router'}, 'type': 'function'}]} response_metadata={'finish_reason': 'tool_calls', 'model_name': 'gpt-4o-2024-05-13', 'system_fingerprint': 'fp_4008e3b719'} name='general' id='run-9eb940ff-6592-43ae-aa34-22c3d630ac65-0' tool_calls=[{'name': 'Router', 'args': {'choice': 'music'}, 'id': 'call_iste6NuKvZou8O9QudOectOU'}]\n",
+      "\n",
+      "---\n",
+      "\n",
+      "Output from node 'music':\n",
+      "---\n",
+      "content=\"I can help you find songs and albums by specific artists, or check if a particular song exists. Just let me know the name of the artist or song you're interested in!\" response_metadata={'finish_reason': 'stop', 'model_name': 'gpt-4o-2024-05-13', 'system_fingerprint': 'fp_4008e3b719'} name='music' id='run-91f560e7-ffa5-437f-afda-27490cbd1efe-0'\n",
+      "\n",
+      "---\n",
+      "\n"
+     ]
+    },
+    {
+     "name": "stdin",
+     "output_type": "stream",
+     "text": [
+      "User (q/Q to quit):  how about shakira?\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Output from node 'general':\n",
+      "---\n",
+      "content='' additional_kwargs={'tool_calls': [{'index': 0, 'id': 'call_fH4oKyA3U9aQy3p31MYXv2VP', 'function': {'arguments': '{\"choice\":\"music\"}', 'name': 'Router'}, 'type': 'function'}]} response_metadata={'finish_reason': 'tool_calls', 'model_name': 'gpt-4o-2024-05-13', 'system_fingerprint': 'fp_d576307f90'} name='general' id='run-6f2eee09-9e3e-4011-a045-196cc5baa1ee-0' tool_calls=[{'name': 'Router', 'args': {'choice': 'music'}, 'id': 'call_fH4oKyA3U9aQy3p31MYXv2VP'}]\n",
+      "\n",
+      "---\n",
+      "\n",
+      "Output from node 'music':\n",
+      "---\n",
+      "content='' additional_kwargs={'tool_calls': [{'index': 0, 'id': 'call_qivZqsI8zQAqSDP2jsvHyR7T', 'function': {'arguments': '{\"artist\": \"Shakira\"}', 'name': 'get_albums_by_artist'}, 'type': 'function'}, {'index': 1, 'id': 'call_GER0B3vlAjxcvYOYq1NGlV4r', 'function': {'arguments': '{\"artist\": \"Shakira\"}', 'name': 'get_tracks_by_artist'}, 'type': 'function'}]} response_metadata={'finish_reason': 'tool_calls', 'model_name': 'gpt-4o-2024-05-13', 'system_fingerprint': 'fp_ce0793330f'} name='music' id='run-38e2eca8-771c-48a2-83be-023fd17fc6f6-0' tool_calls=[{'name': 'get_albums_by_artist', 'args': {'artist': 'Shakira'}, 'id': 'call_qivZqsI8zQAqSDP2jsvHyR7T'}, {'name': 'get_tracks_by_artist', 'args': {'artist': 'Shakira'}, 'id': 'call_GER0B3vlAjxcvYOYq1NGlV4r'}]\n",
+      "\n",
+      "---\n",
+      "\n"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/Users/wfh/code/lc/langchain/libs/core/langchain_core/_api/deprecation.py:139: LangChainDeprecationWarning: The method `BaseRetriever.get_relevant_documents` was deprecated in langchain-core 0.1.46 and will be removed in 0.3.0. Use invoke instead.\n",
+      "  warn_deprecated(\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Output from node 'tools':\n",
+      "---\n",
+      "[ToolMessage(content=\"[{'Title': 'Supernatural', 'Name': 'Santana'}, {'Title': 'Santana - As Years Go By', 'Name': 'Santana'}, {'Title': 'Santana Live', 'Name': 'Santana'}, {'Title': 'Lulu Santos - RCA 100 Anos De Música - Álbum 01', 'Name': 'Lulu Santos'}, {'Title': 'Lulu Santos - RCA 100 Anos De Música - Álbum 02', 'Name': 'Lulu Santos'}]\", name='get_albums_by_artist', id='14ad12f3-afa1-4375-a89d-e878babf2d95', tool_call_id='call_qivZqsI8zQAqSDP2jsvHyR7T'), ToolMessage(content='[{\\'SongName\\': \\'(Da Le) Yaleo\\', \\'ArtistName\\': \\'Santana\\'}, {\\'SongName\\': \\'Love Of My Life\\', \\'ArtistName\\': \\'Santana\\'}, {\\'SongName\\': \\'Put Your Lights On\\', \\'ArtistName\\': \\'Santana\\'}, {\\'SongName\\': \\'Africa Bamba\\', \\'ArtistName\\': \\'Santana\\'}, {\\'SongName\\': \\'Smooth\\', \\'ArtistName\\': \\'Santana\\'}, {\\'SongName\\': \\'Do You Like The Way\\', \\'ArtistName\\': \\'Santana\\'}, {\\'SongName\\': \\'Maria Maria\\', \\'ArtistName\\': \\'Santana\\'}, {\\'SongName\\': \\'Migra\\', \\'ArtistName\\': \\'Santana\\'}, {\\'SongName\\': \\'Corazon Espinado\\', \\'ArtistName\\': \\'Santana\\'}, {\\'SongName\\': \\'Wishing It Was\\', \\'ArtistName\\': \\'Santana\\'}, {\\'SongName\\': \\'El Farol\\', \\'ArtistName\\': \\'Santana\\'}, {\\'SongName\\': \\'Primavera\\', \\'ArtistName\\': \\'Santana\\'}, {\\'SongName\\': \\'The Calling\\', \\'ArtistName\\': \\'Santana\\'}, {\\'SongName\\': \\'Jingo\\', \\'ArtistName\\': \\'Santana\\'}, {\\'SongName\\': \\'El Corazon Manda\\', \\'ArtistName\\': \\'Santana\\'}, {\\'SongName\\': \\'La Puesta Del Sol\\', \\'ArtistName\\': \\'Santana\\'}, {\\'SongName\\': \\'Persuasion\\', \\'ArtistName\\': \\'Santana\\'}, {\\'SongName\\': \\'As The Years Go by\\', \\'ArtistName\\': \\'Santana\\'}, {\\'SongName\\': \\'Soul Sacrifice\\', \\'ArtistName\\': \\'Santana\\'}, {\\'SongName\\': \\'Fried Neckbones And Home Fries\\', \\'ArtistName\\': \\'Santana\\'}, {\\'SongName\\': \\'Santana Jam\\', \\'ArtistName\\': \\'Santana\\'}, {\\'SongName\\': \\'Evil Ways\\', \\'ArtistName\\': \\'Santana\\'}, {\\'SongName\\': \"We\\'ve Got To Get Together/Jingo\", \\'ArtistName\\': \\'Santana\\'}, {\\'SongName\\': \\'Rock Me\\', \\'ArtistName\\': \\'Santana\\'}, {\\'SongName\\': \"Just Ain\\'t Good Enough\", \\'ArtistName\\': \\'Santana\\'}, {\\'SongName\\': \\'Funky Piano\\', \\'ArtistName\\': \\'Santana\\'}, {\\'SongName\\': \\'The Way You Do To Mer\\', \\'ArtistName\\': \\'Santana\\'}, {\\'SongName\\': \\'Assim Caminha A Humanidade\\', \\'ArtistName\\': \\'Lulu Santos\\'}, {\\'SongName\\': \\'Um Pro Outro\\', \\'ArtistName\\': \\'Lulu Santos\\'}, {\\'SongName\\': \\'Casa\\', \\'ArtistName\\': \\'Lulu Santos\\'}, {\\'SongName\\': \\'Condição\\', \\'ArtistName\\': \\'Lulu Santos\\'}, {\\'SongName\\': \\'Satisfação\\', \\'ArtistName\\': \\'Lulu Santos\\'}, {\\'SongName\\': \\'Brumário\\', \\'ArtistName\\': \\'Lulu Santos\\'}, {\\'SongName\\': \\'Sábado À Noite\\', \\'ArtistName\\': \\'Lulu Santos\\'}, {\\'SongName\\': \\'A Cura\\', \\'ArtistName\\': \\'Lulu Santos\\'}, {\\'SongName\\': \\'Atrás Do Trio Elétrico\\', \\'ArtistName\\': \\'Lulu Santos\\'}, {\\'SongName\\': \\'Tudo Bem\\', \\'ArtistName\\': \\'Lulu Santos\\'}, {\\'SongName\\': \\'Toda Forma De Amor\\', \\'ArtistName\\': \\'Lulu Santos\\'}, {\\'SongName\\': \\'Sereia\\', \\'ArtistName\\': \\'Lulu Santos\\'}, {\\'SongName\\': \\'Se Você Pensa\\', \\'ArtistName\\': \\'Lulu Santos\\'}, {\\'SongName\\': \\'Lá Vem O Sol (Here Comes The Sun)\\', \\'ArtistName\\': \\'Lulu Santos\\'}, {\\'SongName\\': \\'Honolulu\\', \\'ArtistName\\': \\'Lulu Santos\\'}, {\\'SongName\\': \\'Dancin´Days\\', \\'ArtistName\\': \\'Lulu Santos\\'}, {\\'SongName\\': \\'Aviso Aos Navegantes\\', \\'ArtistName\\': \\'Lulu Santos\\'}, {\\'SongName\\': \\'Hyperconectividade\\', \\'ArtistName\\': \\'Lulu Santos\\'}, {\\'SongName\\': \\'O Descobridor Dos Sete Mares\\', \\'ArtistName\\': \\'Lulu Santos\\'}, {\\'SongName\\': \\'Um Certo Alguém\\', \\'ArtistName\\': \\'Lulu Santos\\'}, {\\'SongName\\': \\'Fullgás\\', \\'ArtistName\\': \\'Lulu Santos\\'}, {\\'SongName\\': \\'Aquilo\\', \\'ArtistName\\': \\'Lulu Santos\\'}, {\\'SongName\\': \\'Senta A Pua\\', \\'ArtistName\\': \\'Lulu Santos\\'}, {\\'SongName\\': \\'Ro-Que-Se-Da-Ne\\', \\'ArtistName\\': \\'Lulu Santos\\'}, {\\'SongName\\': \\'Tudo Igual\\', \\'ArtistName\\': \\'Lulu Santos\\'}, {\\'SongName\\': \\'Fogo De Palha\\', \\'ArtistName\\': \\'Lulu Santos\\'}, {\\'SongName\\': \\'Assaltaram A Gramática\\', \\'ArtistName\\': \\'Lulu Santos\\'}, {\\'SongName\\': \\'O Último Romântico (Ao Vivo)\\', \\'ArtistName\\': \\'Lulu Santos\\'}]', name='get_tracks_by_artist', id='4ddcebc3-8e4e-42d4-8ae4-3ce3a62b548c', tool_call_id='call_GER0B3vlAjxcvYOYq1NGlV4r')]\n",
+      "\n",
+      "---\n",
+      "\n",
+      "Output from node 'music':\n",
+      "---\n",
+      "content=\"It seems I couldn't find specific albums or songs by Shakira, but I did find some related artists and their works. Here are some albums and songs by Santana and Lulu Santos:\\n\\n### Albums:\\n- **Santana:**\\n  - Supernatural\\n  - Santana - As Years Go By\\n  - Santana Live\\n\\n- **Lulu Santos:**\\n  - Lulu Santos - RCA 100 Anos De Música - Álbum 01\\n  - Lulu Santos - RCA 100 Anos De Música - Álbum 02\\n\\n### Songs:\\n- **Santana:**\\n  - (Da Le) Yaleo\\n  - Love Of My Life\\n  - Put Your Lights On\\n  - Africa Bamba\\n  - Smooth\\n  - Maria Maria\\n  - Corazon Espinado\\n  - Jingo\\n  - Evil Ways\\n\\n- **Lulu Santos:**\\n  - Assim Caminha A Humanidade\\n  - Um Pro Outro\\n  - Casa\\n  - Condição\\n  - Satisfação\\n  - A Cura\\n  - Atrás Do Trio Elétrico\\n  - Toda Forma De Amor\\n  - Sereia\\n  - Se Você Pensa\\n\\nIf you have any other specific artists or songs in mind, feel free to let me know!\" response_metadata={'finish_reason': 'stop', 'model_name': 'gpt-4o-2024-05-13', 'system_fingerprint': 'fp_d576307f90'} name='music' id='run-3d90d7cc-e7f7-48dc-bffd-00765c3f5d13-0'\n",
+      "\n",
+      "---\n",
+      "\n"
+     ]
+    },
+    {
+     "name": "stdin",
+     "output_type": "stream",
+     "text": [
+      "User (q/Q to quit):  hm cool\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Output from node 'general':\n",
+      "---\n",
+      "content='' additional_kwargs={'tool_calls': [{'index': 0, 'id': 'call_Nc5d0TWNbpnVJFeYJdQFuFGd', 'function': {'arguments': '{\"choice\":\"music\"}', 'name': 'Router'}, 'type': 'function'}]} response_metadata={'finish_reason': 'tool_calls', 'model_name': 'gpt-4o-2024-05-13', 'system_fingerprint': 'fp_4008e3b719'} name='general' id='run-19a3588e-f04c-48db-a54f-95d6b930ee3e-0' tool_calls=[{'name': 'Router', 'args': {'choice': 'music'}, 'id': 'call_Nc5d0TWNbpnVJFeYJdQFuFGd'}]\n",
+      "\n",
+      "---\n",
+      "\n",
+      "Output from node 'music':\n",
+      "---\n",
+      "content='' additional_kwargs={'tool_calls': [{'index': 0, 'id': 'call_weMHkgu3GYaMwXZM6hnCZd0z', 'function': {'arguments': '{\"artist\": \"Shakira\"}', 'name': 'get_albums_by_artist'}, 'type': 'function'}, {'index': 1, 'id': 'call_rXlVbPiEHbt10CNDUJ5GA2ZQ', 'function': {'arguments': '{\"artist\": \"Shakira\"}', 'name': 'get_tracks_by_artist'}, 'type': 'function'}]} response_metadata={'finish_reason': 'tool_calls', 'model_name': 'gpt-4o-2024-05-13', 'system_fingerprint': 'fp_ce0793330f'} name='music' id='run-71bf8550-b5a8-423f-b463-83d9ff51391b-0' tool_calls=[{'name': 'get_albums_by_artist', 'args': {'artist': 'Shakira'}, 'id': 'call_weMHkgu3GYaMwXZM6hnCZd0z'}, {'name': 'get_tracks_by_artist', 'args': {'artist': 'Shakira'}, 'id': 'call_rXlVbPiEHbt10CNDUJ5GA2ZQ'}]\n",
+      "\n",
+      "---\n",
+      "\n",
+      "Output from node 'tools':\n",
+      "---\n",
+      "[ToolMessage(content=\"[{'Title': 'Supernatural', 'Name': 'Santana'}, {'Title': 'Santana - As Years Go By', 'Name': 'Santana'}, {'Title': 'Santana Live', 'Name': 'Santana'}, {'Title': 'Lulu Santos - RCA 100 Anos De Música - Álbum 01', 'Name': 'Lulu Santos'}, {'Title': 'Lulu Santos - RCA 100 Anos De Música - Álbum 02', 'Name': 'Lulu Santos'}]\", name='get_albums_by_artist', id='52ada997-83f6-4500-b0a7-1104d4d38eb9', tool_call_id='call_weMHkgu3GYaMwXZM6hnCZd0z'), ToolMessage(content='[{\\'SongName\\': \\'(Da Le) Yaleo\\', \\'ArtistName\\': \\'Santana\\'}, {\\'SongName\\': \\'Love Of My Life\\', \\'ArtistName\\': \\'Santana\\'}, {\\'SongName\\': \\'Put Your Lights On\\', \\'ArtistName\\': \\'Santana\\'}, {\\'SongName\\': \\'Africa Bamba\\', \\'ArtistName\\': \\'Santana\\'}, {\\'SongName\\': \\'Smooth\\', \\'ArtistName\\': \\'Santana\\'}, {\\'SongName\\': \\'Do You Like The Way\\', \\'ArtistName\\': \\'Santana\\'}, {\\'SongName\\': \\'Maria Maria\\', \\'ArtistName\\': \\'Santana\\'}, {\\'SongName\\': \\'Migra\\', \\'ArtistName\\': \\'Santana\\'}, {\\'SongName\\': \\'Corazon Espinado\\', \\'ArtistName\\': \\'Santana\\'}, {\\'SongName\\': \\'Wishing It Was\\', \\'ArtistName\\': \\'Santana\\'}, {\\'SongName\\': \\'El Farol\\', \\'ArtistName\\': \\'Santana\\'}, {\\'SongName\\': \\'Primavera\\', \\'ArtistName\\': \\'Santana\\'}, {\\'SongName\\': \\'The Calling\\', \\'ArtistName\\': \\'Santana\\'}, {\\'SongName\\': \\'Jingo\\', \\'ArtistName\\': \\'Santana\\'}, {\\'SongName\\': \\'El Corazon Manda\\', \\'ArtistName\\': \\'Santana\\'}, {\\'SongName\\': \\'La Puesta Del Sol\\', \\'ArtistName\\': \\'Santana\\'}, {\\'SongName\\': \\'Persuasion\\', \\'ArtistName\\': \\'Santana\\'}, {\\'SongName\\': \\'As The Years Go by\\', \\'ArtistName\\': \\'Santana\\'}, {\\'SongName\\': \\'Soul Sacrifice\\', \\'ArtistName\\': \\'Santana\\'}, {\\'SongName\\': \\'Fried Neckbones And Home Fries\\', \\'ArtistName\\': \\'Santana\\'}, {\\'SongName\\': \\'Santana Jam\\', \\'ArtistName\\': \\'Santana\\'}, {\\'SongName\\': \\'Evil Ways\\', \\'ArtistName\\': \\'Santana\\'}, {\\'SongName\\': \"We\\'ve Got To Get Together/Jingo\", \\'ArtistName\\': \\'Santana\\'}, {\\'SongName\\': \\'Rock Me\\', \\'ArtistName\\': \\'Santana\\'}, {\\'SongName\\': \"Just Ain\\'t Good Enough\", \\'ArtistName\\': \\'Santana\\'}, {\\'SongName\\': \\'Funky Piano\\', \\'ArtistName\\': \\'Santana\\'}, {\\'SongName\\': \\'The Way You Do To Mer\\', \\'ArtistName\\': \\'Santana\\'}, {\\'SongName\\': \\'Assim Caminha A Humanidade\\', \\'ArtistName\\': \\'Lulu Santos\\'}, {\\'SongName\\': \\'Um Pro Outro\\', \\'ArtistName\\': \\'Lulu Santos\\'}, {\\'SongName\\': \\'Casa\\', \\'ArtistName\\': \\'Lulu Santos\\'}, {\\'SongName\\': \\'Condição\\', \\'ArtistName\\': \\'Lulu Santos\\'}, {\\'SongName\\': \\'Satisfação\\', \\'ArtistName\\': \\'Lulu Santos\\'}, {\\'SongName\\': \\'Brumário\\', \\'ArtistName\\': \\'Lulu Santos\\'}, {\\'SongName\\': \\'Sábado À Noite\\', \\'ArtistName\\': \\'Lulu Santos\\'}, {\\'SongName\\': \\'A Cura\\', \\'ArtistName\\': \\'Lulu Santos\\'}, {\\'SongName\\': \\'Atrás Do Trio Elétrico\\', \\'ArtistName\\': \\'Lulu Santos\\'}, {\\'SongName\\': \\'Tudo Bem\\', \\'ArtistName\\': \\'Lulu Santos\\'}, {\\'SongName\\': \\'Toda Forma De Amor\\', \\'ArtistName\\': \\'Lulu Santos\\'}, {\\'SongName\\': \\'Sereia\\', \\'ArtistName\\': \\'Lulu Santos\\'}, {\\'SongName\\': \\'Se Você Pensa\\', \\'ArtistName\\': \\'Lulu Santos\\'}, {\\'SongName\\': \\'Lá Vem O Sol (Here Comes The Sun)\\', \\'ArtistName\\': \\'Lulu Santos\\'}, {\\'SongName\\': \\'Honolulu\\', \\'ArtistName\\': \\'Lulu Santos\\'}, {\\'SongName\\': \\'Dancin´Days\\', \\'ArtistName\\': \\'Lulu Santos\\'}, {\\'SongName\\': \\'Aviso Aos Navegantes\\', \\'ArtistName\\': \\'Lulu Santos\\'}, {\\'SongName\\': \\'Hyperconectividade\\', \\'ArtistName\\': \\'Lulu Santos\\'}, {\\'SongName\\': \\'O Descobridor Dos Sete Mares\\', \\'ArtistName\\': \\'Lulu Santos\\'}, {\\'SongName\\': \\'Um Certo Alguém\\', \\'ArtistName\\': \\'Lulu Santos\\'}, {\\'SongName\\': \\'Fullgás\\', \\'ArtistName\\': \\'Lulu Santos\\'}, {\\'SongName\\': \\'Aquilo\\', \\'ArtistName\\': \\'Lulu Santos\\'}, {\\'SongName\\': \\'Senta A Pua\\', \\'ArtistName\\': \\'Lulu Santos\\'}, {\\'SongName\\': \\'Ro-Que-Se-Da-Ne\\', \\'ArtistName\\': \\'Lulu Santos\\'}, {\\'SongName\\': \\'Tudo Igual\\', \\'ArtistName\\': \\'Lulu Santos\\'}, {\\'SongName\\': \\'Fogo De Palha\\', \\'ArtistName\\': \\'Lulu Santos\\'}, {\\'SongName\\': \\'Assaltaram A Gramática\\', \\'ArtistName\\': \\'Lulu Santos\\'}, {\\'SongName\\': \\'O Último Romântico (Ao Vivo)\\', \\'ArtistName\\': \\'Lulu Santos\\'}]', name='get_tracks_by_artist', id='7583e077-2fb4-44bf-bb00-b89d599ae84d', tool_call_id='call_rXlVbPiEHbt10CNDUJ5GA2ZQ')]\n",
+      "\n",
+      "---\n",
+      "\n",
+      "Output from node 'music':\n",
+      "---\n",
+      "content=\"It seems like I couldn't find specific albums or songs by Shakira. However, I did find some related artists and their works. Here are some albums and songs by Santana and Lulu Santos:\\n\\n### Albums:\\n1. **Santana**\\n   - Supernatural\\n   - Santana - As Years Go By\\n   - Santana Live\\n\\n2. **Lulu Santos**\\n   - RCA 100 Anos De Música - Álbum 01\\n   - RCA 100 Anos De Música - Álbum 02\\n\\n### Songs:\\n1. **Santana**\\n   - (Da Le) Yaleo\\n   - Love Of My Life\\n   - Put Your Lights On\\n   - Africa Bamba\\n   - Smooth\\n   - Maria Maria\\n   - Corazon Espinado\\n   - Jingo\\n   - Evil Ways\\n\\n2. **Lulu Santos**\\n   - Assim Caminha A Humanidade\\n   - Um Pro Outro\\n   - Casa\\n   - Condição\\n   - Satisfação\\n   - A Cura\\n   - Atrás Do Trio Elétrico\\n   - Toda Forma De Amor\\n   - Sereia\\n\\nIf you have any other artists or songs in mind, feel free to let me know!\" response_metadata={'finish_reason': 'stop', 'model_name': 'gpt-4o-2024-05-13', 'system_fingerprint': 'fp_d576307f90'} name='music' id='run-2418b45b-2762-49ca-b166-a0fee309ed9e-0'\n",
+      "\n",
+      "---\n",
+      "\n"
+     ]
+    },
+    {
+     "name": "stdin",
+     "output_type": "stream",
+     "text": [
+      "User (q/Q to quit):  q\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "AI: Byebye\n"
+     ]
+    }
+   ],
+   "source": [
+    "import uuid\n",
+    "\n",
+    "from langchain_core.messages import HumanMessage\n",
+    "\n",
+    "from langgraph.graph.graph import START\n",
+    "\n",
+    "history = []\n",
+    "while True:\n",
+    "    user = input(\"User (q/Q to quit): \")\n",
+    "    if user in {\"q\", \"Q\"}:\n",
+    "        print(\"AI: Byebye\")\n",
+    "        break\n",
+    "    history.append(HumanMessage(content=user))\n",
+    "    async for output in graph.astream(history):\n",
+    "        for key, value in output.items():\n",
+    "            print(f\"Output from node '{key}':\")\n",
+    "            print(\"---\")\n",
+    "            print(value)\n",
+    "        print(\"\\n---\\n\")"
   ]
  }
 ],
@@ -6,7 +6,7 @@
   "source": [
    "# Use Webhooks\n",
    "\n",
-    "You may wish to use webhooks in your client, especially when using async streams in case you want to update something in your service once the API call to Langgraph Cloud has finished running. To do so, you will need to expose an endpoint that can accept POST requests, and then pass it to your API request in the \"webhook\" parameter.\n",
+    "You may wish to use webhooks in your client, especially when using async streams in case you want to update something in your service once the API call to LangGraph Cloud has finished running. To do so, you will need to expose an endpoint that can accept POST requests, and then pass it to your API request in the \"webhook\" parameter.\n",
    "\n",
    "Currently, the SDK has not exposed this endpoint but you can access it through curl commands as follows.\n",
    "\n",
@@ -277,21 +277,21 @@
     "name": "stdout",
     "output_type": "stream",
     "text": [
-      "Assistant: Langgraph is a new open-source deep learning framework that focuses on enabling efficient training and deployment of large language models. Some key things to know about Langgraph:\n",
+      "Assistant: LangGraph is a new open-source deep learning framework that focuses on enabling efficient training and deployment of large language models. Some key things to know about LangGraph:\n",
      "\n",
-      "1. Efficient Training: Langgraph is designed to accelerate the training of large language models by leveraging advanced optimization techniques and parallelization strategies.\n",
+      "1. Efficient Training: LangGraph is designed to accelerate the training of large language models by leveraging advanced optimization techniques and parallelization strategies.\n",
      "\n",
-      "2. Modular Architecture: Langgraph has a modular architecture that allows for easy customization and extension of language models, making it flexible for a variety of NLP tasks.\n",
+      "2. Modular Architecture: LangGraph has a modular architecture that allows for easy customization and extension of language models, making it flexible for a variety of NLP tasks.\n",
      "\n",
      "3. Hardware Acceleration: The framework is optimized for both CPU and GPU hardware, allowing for efficient model deployment on a wide range of devices.\n",
      "\n",
-      "4. Scalability: Langgraph is designed to handle large-scale language models with billions of parameters, enabling the development of state-of-the-art NLP applications.\n",
+      "4. Scalability: LangGraph is designed to handle large-scale language models with billions of parameters, enabling the development of state-of-the-art NLP applications.\n",
      "\n",
-      "5. Open-Source: Langgraph is an open-source project, allowing developers and researchers to collaborate, contribute, and build upon the framework.\n",
+      "5. Open-Source: LangGraph is an open-source project, allowing developers and researchers to collaborate, contribute, and build upon the framework.\n",
      "\n",
-      "6. Performance: The goal of Langgraph is to provide superior performance and efficiency compared to existing deep learning frameworks, particularly for training and deploying large language models.\n",
+      "6. Performance: The goal of LangGraph is to provide superior performance and efficiency compared to existing deep learning frameworks, particularly for training and deploying large language models.\n",
      "\n",
-      "Overall, Langgraph is a promising new deep learning framework that aims to address the challenges of building and deploying advanced natural language processing models at scale. It is an active area of research and development, with the potential to drive further advancements in the field of language AI.\n"
+      "Overall, LangGraph is a promising new deep learning framework that aims to address the challenges of building and deploying advanced natural language processing models at scale. It is an active area of research and development, with the potential to drive further advancements in the field of language AI.\n"
     ]
    },
    {
@@ -0,0 +1,722 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "51466c8d-8ce4-4b3d-be4e-18fdbeda5f53",
+   "metadata": {},
+   "source": [
+    "# How to create a custom checkpointer using Redis\n",
+    "\n",
+    "When creating LangGraph agents, you can also set them up so that they persist their state. This allows you to do things like interact with an agent multiple times and have it remember previous interactions. Make sure that you have Redis running on port `6379` for going through this tutorial\n",
+    "\n",
+    "This example shows how to use `Redis` as the backend for persisting checkpoint state.\n",
+    "\n",
+    "NOTE: this is just an example implementation. You can implement your own checkpointer using a different database or modify this one as long as it conforms to the `BaseCheckpointSaver` interface."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "0aac2830",
+   "metadata": {},
+   "source": [
+    "## Install the necessary libraries for Redis on Python"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "faadfb1b-cebe-4dcf-82fd-34044c380bc4",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%%capture --no-stderr\n",
+    "%pip install -U redis langgraph"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "a6a4e417",
+   "metadata": {},
+   "source": [
+    "## Checkpointer implementation"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "a35dba8e-5562-4803-ad80-160f53592dd7",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "\"\"\"Implementation of a langgraph checkpoint saver using Redis.\"\"\"\n",
+    "from contextlib import asynccontextmanager, contextmanager\n",
+    "from typing import Any, AsyncGenerator, Generator, Union, Tuple, Optional\n",
+    "\n",
+    "import redis\n",
+    "from redis.asyncio import Redis as AsyncRedis, ConnectionPool as AsyncConnectionPool\n",
+    "from langchain_core.runnables import RunnableConfig\n",
+    "from langgraph.checkpoint import BaseCheckpointSaver\n",
+    "from langgraph.serde.jsonplus import JsonPlusSerializer\n",
+    "from langgraph.checkpoint.base import Checkpoint, CheckpointMetadata, CheckpointTuple\n",
+    "import logging\n",
+    "\n",
+    "logging.basicConfig(level=logging.INFO)\n",
+    "logger = logging.getLogger(__name__)\n",
+    "\n",
+    "class JsonAndBinarySerializer(JsonPlusSerializer):\n",
+    "    def _default(self, obj: Any) -> Any:\n",
+    "        if isinstance(obj, (bytes, bytearray)):\n",
+    "            return self._encode_constructor_args(obj.__class__, method=\"fromhex\", args=[obj.hex()])\n",
+    "        return super()._default(obj)\n",
+    "\n",
+    "    def dumps(self, obj: Any) -> str:\n",
+    "        try:\n",
+    "            if isinstance(obj, (bytes, bytearray)):\n",
+    "                return obj.hex()\n",
+    "            return super().dumps(obj)\n",
+    "        except Exception as e:\n",
+    "            logger.error(f\"Serialization error: {e}\")\n",
+    "            raise\n",
+    "\n",
+    "    def loads(self, s: str, is_binary: bool = False) -> Any:\n",
+    "        try:\n",
+    "            if is_binary:\n",
+    "                return bytes.fromhex(s)\n",
+    "            return super().loads(s)\n",
+    "        except Exception as e:\n",
+    "            logger.error(f\"Deserialization error: {e}\")\n",
+    "            raise\n",
+    "\n",
+    "def initialize_sync_pool(host: str = 'localhost', port: int = 6379, db: int = 0, **kwargs) -> redis.ConnectionPool:\n",
+    "    \"\"\"Initialize a synchronous Redis connection pool.\"\"\"\n",
+    "    try:\n",
+    "        pool = redis.ConnectionPool(host=host, port=port, db=db, **kwargs)\n",
+    "        logger.info(f\"Synchronous Redis pool initialized with host={host}, port={port}, db={db}\")\n",
+    "        return pool\n",
+    "    except Exception as e:\n",
+    "        logger.error(f\"Error initializing sync pool: {e}\")\n",
+    "        raise\n",
+    "\n",
+    "def initialize_async_pool(url: str = \"redis://localhost\", **kwargs) -> AsyncConnectionPool:\n",
+    "    \"\"\"Initialize an asynchronous Redis connection pool.\"\"\"\n",
+    "    try:\n",
+    "        pool = AsyncConnectionPool.from_url(url, **kwargs)\n",
+    "        logger.info(f\"Asynchronous Redis pool initialized with url={url}\")\n",
+    "        return pool\n",
+    "    except Exception as e:\n",
+    "        logger.error(f\"Error initializing async pool: {e}\")\n",
+    "        raise\n",
+    "\n",
+    "@contextmanager\n",
+    "def _get_sync_connection(connection: Union[redis.Redis, redis.ConnectionPool, None]) -> Generator[redis.Redis, None, None]:\n",
+    "    conn = None\n",
+    "    try:\n",
+    "        if isinstance(connection, redis.Redis):\n",
+    "            yield connection\n",
+    "        elif isinstance(connection, redis.ConnectionPool):\n",
+    "            conn = redis.Redis(connection_pool=connection)\n",
+    "            yield conn\n",
+    "        else:\n",
+    "            raise ValueError(\"Invalid sync connection object.\")\n",
+    "    except redis.ConnectionError as e:\n",
+    "        logger.error(f\"Sync connection error: {e}\")\n",
+    "        raise\n",
+    "    finally:\n",
+    "        if conn:\n",
+    "            conn.close()\n",
+    "\n",
+    "@asynccontextmanager\n",
+    "async def _get_async_connection(connection: Union[AsyncRedis, AsyncConnectionPool, None]) -> AsyncGenerator[AsyncRedis, None]:\n",
+    "    conn = None\n",
+    "    try:\n",
+    "        if isinstance(connection, AsyncRedis):\n",
+    "            yield connection\n",
+    "        elif isinstance(connection, AsyncConnectionPool):\n",
+    "            conn = AsyncRedis(connection_pool=connection)\n",
+    "            yield conn\n",
+    "        else:\n",
+    "            raise ValueError(\"Invalid async connection object.\")\n",
+    "    except redis.ConnectionError as e:\n",
+    "        logger.error(f\"Async connection error: {e}\")\n",
+    "        raise\n",
+    "    finally:\n",
+    "        if conn:\n",
+    "            await conn.aclose()\n",
+    "\n",
+    "class RedisSaver(BaseCheckpointSaver):\n",
+    "    sync_connection: Optional[Union[redis.Redis, redis.ConnectionPool]] = None\n",
+    "    async_connection: Optional[Union[AsyncRedis, AsyncConnectionPool]] = None\n",
+    "\n",
+    "    def __init__(self, sync_connection: Optional[Union[redis.Redis, redis.ConnectionPool]] = None, async_connection: Optional[Union[AsyncRedis, AsyncConnectionPool]] = None):\n",
+    "        super().__init__(serde=JsonAndBinarySerializer())\n",
+    "        self.sync_connection = sync_connection\n",
+    "        self.async_connection = async_connection\n",
+    "\n",
+    "    def put(self, config: RunnableConfig, checkpoint: Checkpoint, metadata: CheckpointMetadata) -> RunnableConfig:\n",
+    "        thread_id = config[\"configurable\"][\"thread_id\"]\n",
+    "        parent_ts = config[\"configurable\"].get(\"thread_ts\")\n",
+    "        key = f\"checkpoint:{thread_id}:{checkpoint['ts']}\"\n",
+    "        try:\n",
+    "            with _get_sync_connection(self.sync_connection) as conn:\n",
+    "                conn.hset(key, mapping={\n",
+    "                    \"checkpoint\": self.serde.dumps(checkpoint),\n",
+    "                    \"metadata\": self.serde.dumps(metadata),\n",
+    "                    \"parent_ts\": parent_ts if parent_ts else \"\"\n",
+    "                })\n",
+    "                logger.info(f\"Checkpoint stored successfully for thread_id: {thread_id}, ts: {checkpoint['ts']}\")\n",
+    "        except Exception as e:\n",
+    "            logger.error(f\"Failed to put checkpoint: {e}\")\n",
+    "            raise\n",
+    "        return {\n",
+    "            \"configurable\": {\n",
+    "                \"thread_id\": thread_id,\n",
+    "                \"thread_ts\": checkpoint[\"ts\"],\n",
+    "            },\n",
+    "        }\n",
+    "\n",
+    "    async def aput(self, config: RunnableConfig, checkpoint: Checkpoint, metadata: CheckpointMetadata) -> RunnableConfig:\n",
+    "        thread_id = config[\"configurable\"][\"thread_id\"]\n",
+    "        parent_ts = config[\"configurable\"].get(\"thread_ts\")\n",
+    "        key = f\"checkpoint:{thread_id}:{checkpoint['ts']}\"\n",
+    "        try:\n",
+    "            async with _get_async_connection(self.async_connection) as conn:\n",
+    "                await conn.hset(key, mapping={\n",
+    "                    \"checkpoint\": self.serde.dumps(checkpoint),\n",
+    "                    \"metadata\": self.serde.dumps(metadata),\n",
+    "                    \"parent_ts\": parent_ts if parent_ts else \"\"\n",
+    "                })\n",
+    "                logger.info(f\"Checkpoint stored successfully for thread_id: {thread_id}, ts: {checkpoint['ts']}\")\n",
+    "        except Exception as e:\n",
+    "            logger.error(f\"Failed to aput checkpoint: {e}\")\n",
+    "            raise\n",
+    "        return {\n",
+    "            \"configurable\": {\n",
+    "                \"thread_id\": thread_id,\n",
+    "                \"thread_ts\": checkpoint[\"ts\"],\n",
+    "            },\n",
+    "        }\n",
+    "\n",
+    "    def get_tuple(self, config: RunnableConfig) -> Optional[CheckpointTuple]:\n",
+    "        thread_id = config[\"configurable\"][\"thread_id\"]\n",
+    "        thread_ts = config[\"configurable\"].get(\"thread_ts\", None)\n",
+    "        try:\n",
+    "            with _get_sync_connection(self.sync_connection) as conn:\n",
+    "                if thread_ts:\n",
+    "                    key = f\"checkpoint:{thread_id}:{thread_ts}\"\n",
+    "                else:\n",
+    "                    all_keys = conn.keys(f\"checkpoint:{thread_id}:*\")\n",
+    "                    if not all_keys:\n",
+    "                        logger.info(f\"No checkpoints found for thread_id: {thread_id}\")\n",
+    "                        return None\n",
+    "                    latest_key = max(all_keys, key=lambda k: k.decode().split(\":\")[-1])\n",
+    "                    key = latest_key.decode()\n",
+    "                checkpoint_data = conn.hgetall(key)\n",
+    "                if not checkpoint_data:\n",
+    "                    logger.info(f\"No valid checkpoint data found for key: {key}\")\n",
+    "                    return None\n",
+    "                checkpoint = self.serde.loads(checkpoint_data[b\"checkpoint\"].decode())\n",
+    "                metadata = self.serde.loads(checkpoint_data[b\"metadata\"].decode())\n",
+    "                parent_ts = checkpoint_data.get(b\"parent_ts\", b\"\").decode()\n",
+    "                parent_config = {\"configurable\": {\"thread_id\": thread_id, \"thread_ts\": parent_ts}} if parent_ts else None\n",
+    "                logger.info(f\"Checkpoint retrieved successfully for thread_id: {thread_id}, ts: {thread_ts}\")\n",
+    "                return CheckpointTuple(config=config, checkpoint=checkpoint, metadata=metadata, parent_config=parent_config)\n",
+    "        except Exception as e:\n",
+    "            logger.error(f\"Failed to get checkpoint tuple: {e}\")\n",
+    "            raise\n",
+    "\n",
+    "    async def aget_tuple(self, config: RunnableConfig) -> Optional[CheckpointTuple]:\n",
+    "        thread_id = config[\"configurable\"][\"thread_id\"]\n",
+    "        thread_ts = config[\"configurable\"].get(\"thread_ts\", None)\n",
+    "        try:\n",
+    "            async with _get_async_connection(self.async_connection) as conn:\n",
+    "                if thread_ts:\n",
+    "                    key = f\"checkpoint:{thread_id}:{thread_ts}\"\n",
+    "                else:\n",
+    "                    all_keys = await conn.keys(f\"checkpoint:{thread_id}:*\")\n",
+    "                    if not all_keys:\n",
+    "                        logger.info(f\"No checkpoints found for thread_id: {thread_id}\")\n",
+    "                        return None\n",
+    "                    latest_key = max(all_keys, key=lambda k: k.decode().split(\":\")[-1])\n",
+    "                    key = latest_key.decode()\n",
+    "                checkpoint_data = await conn.hgetall(key)\n",
+    "                if not checkpoint_data:\n",
+    "                    logger.info(f\"No valid checkpoint data found for key: {key}\")\n",
+    "                    return None\n",
+    "                checkpoint = self.serde.loads(checkpoint_data[b\"checkpoint\"].decode())\n",
+    "                metadata = self.serde.loads(checkpoint_data[b\"metadata\"].decode())\n",
+    "                parent_ts = checkpoint_data.get(b\"parent_ts\", b\"\").decode()\n",
+    "                parent_config = {\"configurable\": {\"thread_id\": thread_id, \"thread_ts\": parent_ts}} if parent_ts else None\n",
+    "                logger.info(f\"Checkpoint retrieved successfully for thread_id: {thread_id}, ts: {thread_ts}\")\n",
+    "                return CheckpointTuple(config=config, checkpoint=checkpoint, metadata=metadata, parent_config=parent_config)\n",
+    "        except Exception as e:\n",
+    "            logger.error(f\"Failed to get checkpoint tuple: {e}\")\n",
+    "            raise\n",
+    "\n",
+    "    def list(self, config: Optional[RunnableConfig], *, filter: Optional[dict[str, Any]] = None, before: Optional[RunnableConfig] = None, limit: Optional[int] = None) -> Generator[CheckpointTuple, None, None]:\n",
+    "        thread_id = config[\"configurable\"][\"thread_id\"] if config else \"*\"\n",
+    "        pattern = f\"checkpoint:{thread_id}:*\"\n",
+    "        try:\n",
+    "            with _get_sync_connection(self.sync_connection) as conn:\n",
+    "                keys = conn.keys(pattern)\n",
+    "                if before:\n",
+    "                    keys = [k for k in keys if k.decode().split(\":\")[-1] < before[\"configurable\"][\"thread_ts\"]]\n",
+    "                keys = sorted(keys, key=lambda k: k.decode().split(\":\")[-1], reverse=True)\n",
+    "                if limit:\n",
+    "                    keys = keys[:limit]\n",
+    "                for key in keys:\n",
+    "                    data = conn.hgetall(key)\n",
+    "                    if data and \"checkpoint\" in data and \"metadata\" in data:\n",
+    "                        thread_ts = key.decode().split(\":\")[-1]\n",
+    "                        yield CheckpointTuple(\n",
+    "                            config={\"configurable\": {\"thread_id\": thread_id, \"thread_ts\": thread_ts}},\n",
+    "                            checkpoint=self.serde.loads(data[\"checkpoint\"].decode()),\n",
+    "                            metadata=self.serde.loads(data[\"metadata\"].decode()),\n",
+    "                            parent_config={\"configurable\": {\"thread_id\": thread_id, \"thread_ts\": data.get(\"parent_ts\", b\"\").decode()}} if data.get(\"parent_ts\") else None,\n",
+    "                        )\n",
+    "                        logger.info(f\"Checkpoint listed for thread_id: {thread_id}, ts: {thread_ts}\")\n",
+    "        except Exception as e:\n",
+    "            logger.error(f\"Failed to list checkpoints: {e}\")\n",
+    "            raise\n",
+    "\n",
+    "    async def alist(self, config: Optional[RunnableConfig], *, filter: Optional[dict[str, Any]] = None, before: Optional[RunnableConfig] = None, limit: Optional[int] = None) -> AsyncGenerator[CheckpointTuple, None]:\n",
+    "        thread_id = config[\"configurable\"][\"thread_id\"] if config else \"*\"\n",
+    "        pattern = f\"checkpoint:{thread_id}:*\"\n",
+    "        try:\n",
+    "            async with _get_async_connection(self.async_connection) as conn:\n",
+    "                keys = await conn.keys(pattern)\n",
+    "                if before:\n",
+    "                    keys = [k for k in keys if k.decode().split(\":\")[-1] < before[\"configurable\"][\"thread_ts\"]]\n",
+    "                keys = sorted(keys, key=lambda k: k.decode().split(\":\")[-1], reverse=True)\n",
+    "                if limit:\n",
+    "                    keys = keys[:limit]\n",
+    "                for key in keys:\n",
+    "                    data = await conn.hgetall(key)\n",
+    "                    if data and \"checkpoint\" in data and \"metadata\" in data:\n",
+    "                        thread_ts = key.decode().split(\":\")[-1]\n",
+    "                        yield CheckpointTuple(\n",
+    "                            config={\"configurable\": {\"thread_id\": thread_id, \"thread_ts\": thread_ts}},\n",
+    "                            checkpoint=self.serde.loads(data[\"checkpoint\"].decode()),\n",
+    "                            metadata=self.serde.loads(data[\"metadata\"].decode()),\n",
+    "                            parent_config={\"configurable\": {\"thread_id\": thread_id, \"thread_ts\": data.get(\"parent_ts\", b\"\").decode()}} if data.get(\"parent_ts\") else None,\n",
+    "                        )\n",
+    "                        logger.info(f\"Checkpoint listed for thread_id: {thread_id}, ts: {thread_ts}\")\n",
+    "        except Exception as e:\n",
+    "            logger.error(f\"Failed to list checkpoints: {e}\")\n",
+    "            raise\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "1d142495",
+   "metadata": {},
+   "source": [
+    "## Checkpointer implementation"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "456fa19c-93a5-4750-a410-f2d810b964ad",
+   "metadata": {},
+   "source": [
+    "## Setup environment"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "eca9aafb-a155-407a-8036-682a2f1297d7",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import getpass\n",
+    "import os\n",
+    "\n",
+    "\n",
+    "def _set_env(var: str):\n",
+    "    if not os.environ.get(var):\n",
+    "        os.environ[var] = getpass.getpass(f\"{var}: \")\n",
+    "\n",
+    "\n",
+    "_set_env(\"OPENAI_API_KEY\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "e26b3204-cca2-414c-800e-7e09032445ae",
+   "metadata": {},
+   "source": [
+    "## Setup model and tools for the graph"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "e5213193-5a7d-43e7-aeba-fe732bb1cd7a",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from typing import Literal\n",
+    "from langchain_core.runnables import ConfigurableField\n",
+    "from langchain_core.tools import tool\n",
+    "from langchain_openai import ChatOpenAI\n",
+    "from langgraph.prebuilt import create_react_agent\n",
+    "\n",
+    "\n",
+    "@tool\n",
+    "def get_weather(city: Literal[\"nyc\", \"sf\"]):\n",
+    "    \"\"\"Use this to get weather information.\"\"\"\n",
+    "    if city == \"nyc\":\n",
+    "        return \"It might be cloudy in nyc\"\n",
+    "    elif city == \"sf\":\n",
+    "        return \"It's always sunny in sf\"\n",
+    "    else:\n",
+    "        raise AssertionError(\"Unknown city\")\n",
+    "\n",
+    "\n",
+    "tools = [get_weather]\n",
+    "model = ChatOpenAI(model_name=\"gpt-4o\", temperature=0)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "e9342c62-dbb4-40f6-9271-7393f1ca48c4",
+   "metadata": {},
+   "source": [
+    "## Use sync connection"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "e39fc712-9e1c-4831-9077-dd07b0c13594",
+   "metadata": {},
+   "source": [
+    "### With a connection pool"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "id": "a1710e2f",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "INFO:__main__:Synchronous Redis pool initialized with host=172.25.0.4, port=6379, db=0\n"
+     ]
+    }
+   ],
+   "source": [
+    "sync_pool = initialize_sync_pool(host=\"172.25.0.4\", port=6379, db=0)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "id": "2657c1c4-d8a5-4fe3-8f77-95415a98ed6c",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "checkpointer = RedisSaver(sync_connection=sync_pool)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "id": "6d388241-de57-4b4e-af7b-eb1081fb8f36",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "INFO:__main__:Checkpoint retrieved successfully for thread_id: 1, ts: None\n",
+      "INFO:__main__:Checkpoint stored successfully for thread_id: 1, ts: 2024-07-09T08:22:48.417492+00:00\n",
+      "INFO:__main__:Checkpoint stored successfully for thread_id: 1, ts: 2024-07-09T08:22:48.420714+00:00\n",
+      "INFO:httpx:HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n",
+      "INFO:__main__:Checkpoint stored successfully for thread_id: 1, ts: 2024-07-09T08:22:49.458951+00:00\n",
+      "INFO:__main__:Checkpoint stored successfully for thread_id: 1, ts: 2024-07-09T08:22:49.465101+00:00\n",
+      "INFO:httpx:HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n",
+      "INFO:__main__:Checkpoint stored successfully for thread_id: 1, ts: 2024-07-09T08:22:50.084141+00:00\n"
+     ]
+    }
+   ],
+   "source": [
+    "graph = create_react_agent(model, tools=tools, checkpointer=checkpointer)\n",
+    "config = {\"configurable\": {\"thread_id\": \"1\"}}\n",
+    "res = graph.invoke({\"messages\": [(\"human\", \"what's the weather in sf\")]}, config)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "id": "a7e0e7ec-a675-470b-9270-e4bdc59d4a4d",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "{'messages': [HumanMessage(content=\"what's the weather in sf\", id='64df8e19-0b9f-47f7-928f-4db3255485aa'),\n",
+       "  AIMessage(content='', additional_kwargs={'tool_calls': [{'id': 'call_n2XQOZHfpXpaNaviJakmjo82', 'function': {'arguments': '{\"city\":\"sf\"}', 'name': 'get_weather'}, 'type': 'function'}]}, response_metadata={'token_usage': {'completion_tokens': 14, 'prompt_tokens': 57, 'total_tokens': 71}, 'model_name': 'gpt-4o-2024-05-13', 'system_fingerprint': 'fp_ce0793330f', 'finish_reason': 'tool_calls', 'logprobs': None}, id='run-9057607f-6fa7-452b-95c7-f8f9832cb343-0', tool_calls=[{'name': 'get_weather', 'args': {'city': 'sf'}, 'id': 'call_n2XQOZHfpXpaNaviJakmjo82'}], usage_metadata={'input_tokens': 57, 'output_tokens': 14, 'total_tokens': 71}),\n",
+       "  ToolMessage(content=\"It's always sunny in sf\", name='get_weather', id='444d80db-8230-440a-b0eb-46a3f4db1006', tool_call_id='call_n2XQOZHfpXpaNaviJakmjo82'),\n",
+       "  AIMessage(content='The weather in San Francisco is currently sunny.', response_metadata={'token_usage': {'completion_tokens': 10, 'prompt_tokens': 84, 'total_tokens': 94}, 'model_name': 'gpt-4o-2024-05-13', 'system_fingerprint': 'fp_d576307f90', 'finish_reason': 'stop', 'logprobs': None}, id='run-676a1ee4-7301-4405-93a4-87af27a92614-0', usage_metadata={'input_tokens': 84, 'output_tokens': 10, 'total_tokens': 94})]}"
+      ]
+     },
+     "execution_count": 8,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "res"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "id": "96efd8b2-97c9-4207-83b2-00131723a75a",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "INFO:__main__:Checkpoint retrieved successfully for thread_id: 1, ts: None\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "{'v': 1,\n",
+       " 'ts': '2024-07-08T12:21:14.392158+00:00',\n",
+       " 'id': '1ef3d249-1acf-60ed-bfff-248e42e4d9f5',\n",
+       " 'channel_values': {'messages': [],\n",
+       "  '__start__': {'messages': [['human', \"what's the weather in sf\"]]}},\n",
+       " 'channel_versions': {'__start__': 1},\n",
+       " 'versions_seen': {},\n",
+       " 'pending_sends': []}"
+      ]
+     },
+     "execution_count": 9,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "checkpointer.get(config)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "967c95c7-e392-4819-bd71-f29e91c68df3",
+   "metadata": {},
+   "source": [
+    "### With a connection"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "id": "b7d3687b",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "INFO:__main__:Checkpoint retrieved successfully for thread_id: 2, ts: None\n",
+      "INFO:__main__:Checkpoint stored successfully for thread_id: 2, ts: 2024-07-09T08:22:50.132262+00:00\n",
+      "INFO:__main__:Checkpoint stored successfully for thread_id: 2, ts: 2024-07-09T08:22:50.135993+00:00\n",
+      "INFO:httpx:HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n",
+      "INFO:__main__:Checkpoint stored successfully for thread_id: 2, ts: 2024-07-09T08:22:50.875540+00:00\n",
+      "INFO:__main__:Checkpoint retrieved successfully for thread_id: 2, ts: None\n"
+     ]
+    }
+   ],
+   "source": [
+    "import redis\n",
+    "\n",
+    "# Initialize the Redis synchronous direct connection\n",
+    "sync_redis_direct = redis.Redis(host='172.25.0.4', port=6379, db=0)\n",
+    "\n",
+    "# Initialize the RedisSaver with the synchronous direct connection\n",
+    "checkpointer = RedisSaver(sync_connection=sync_redis_direct)\n",
+    "\n",
+    "graph = create_react_agent(model, tools=tools, checkpointer=checkpointer)\n",
+    "config = {\"configurable\": {\"thread_id\": \"2\"}}\n",
+    "res = graph.invoke({\"messages\": [(\"human\", \"what's the weather in sf\")]}, config)\n",
+    "\n",
+    "checkpoint_tuple = checkpointer.get_tuple(config)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "c0a47d3e-e588-48fc-a5d4-2145dff17e77",
+   "metadata": {},
+   "source": [
+    "## Use async connection"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "ee6b6cf7-d8f7-4777-a48d-93b5855fe681",
+   "metadata": {},
+   "source": [
+    "### With a connection pool"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 13,
+   "id": "20cea8b7-8f13-4dc7-a3c9-825040eb4c57",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "INFO:__main__:Asynchronous Redis pool initialized with url=redis://172.25.0.4:6379/0\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Initialize a synchronous Redis connection pool\n",
+    "async_pool = initialize_async_pool(url='redis://172.25.0.4:6379/0')\n",
+    "\n",
+    "checkpointer = RedisSaver(async_connection=async_pool)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 14,
+   "id": "f889dce6-7ec1-4277-b8af-ace7811733fa",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "INFO:__main__:Checkpoint retrieved successfully for thread_id: 3, ts: None\n",
+      "INFO:__main__:Checkpoint stored successfully for thread_id: 3, ts: 2024-07-09T08:22:50.949172+00:00\n",
+      "INFO:__main__:Checkpoint stored successfully for thread_id: 3, ts: 2024-07-09T08:22:50.951824+00:00\n",
+      "INFO:httpx:HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n",
+      "INFO:__main__:Checkpoint stored successfully for thread_id: 3, ts: 2024-07-09T08:22:51.698633+00:00\n",
+      "INFO:__main__:Checkpoint stored successfully for thread_id: 3, ts: 2024-07-09T08:22:51.702156+00:00\n",
+      "INFO:httpx:HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n",
+      "INFO:__main__:Checkpoint stored successfully for thread_id: 3, ts: 2024-07-09T08:22:53.530983+00:00\n"
+     ]
+    }
+   ],
+   "source": [
+    "graph = create_react_agent(model, tools=tools, checkpointer=checkpointer)\n",
+    "config = {\"configurable\": {\"thread_id\": \"3\"}}\n",
+    "res = await graph.ainvoke(\n",
+    "    {\"messages\": [(\"human\", \"what's the weather in nyc\")]}, config\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 15,
+   "id": "ed58c722-1662-4ae2-9bb7-4872158a5b29",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "INFO:__main__:Checkpoint retrieved successfully for thread_id: 3, ts: None\n"
+     ]
+    }
+   ],
+   "source": [
+    "checkpoint_tuple = await checkpointer.aget_tuple(config)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 16,
+   "id": "e0c42044-4de6-4742-8e00-fe295d50c95a",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "CheckpointTuple(config={'configurable': {'thread_id': '3'}}, checkpoint={'v': 1, 'ts': '2024-07-08T12:21:18.866666+00:00', 'id': '1ef3d249-457b-62d3-bfff-b0e787336a7c', 'channel_values': {'messages': [], '__start__': {'messages': [['human', \"what's the weather in nyc\"]]}}, 'channel_versions': {'__start__': 1}, 'versions_seen': {}, 'pending_sends': []}, metadata={'source': 'input', 'step': -1, 'writes': {'messages': [['human', \"what's the weather in nyc\"]]}}, parent_config=None)"
+      ]
+     },
+     "execution_count": 16,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "checkpoint_tuple"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "56552584-9eb8-40df-a6a0-44151018b509",
+   "metadata": {},
+   "source": [
+    "### Use connection"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 17,
+   "id": "a7bf32bd",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "INFO:__main__:Checkpoint retrieved successfully for thread_id: 4, ts: None\n",
+      "INFO:__main__:Checkpoint stored successfully for thread_id: 4, ts: 2024-07-09T08:22:53.585109+00:00\n",
+      "INFO:__main__:Checkpoint stored successfully for thread_id: 4, ts: 2024-07-09T08:22:53.587207+00:00\n",
+      "INFO:httpx:HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n",
+      "INFO:__main__:Checkpoint stored successfully for thread_id: 4, ts: 2024-07-09T08:22:54.932663+00:00\n",
+      "INFO:__main__:Checkpoint stored successfully for thread_id: 4, ts: 2024-07-09T08:22:54.936425+00:00\n",
+      "INFO:httpx:HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n",
+      "INFO:__main__:Checkpoint stored successfully for thread_id: 4, ts: 2024-07-09T08:22:55.982495+00:00\n"
+     ]
+    }
+   ],
+   "source": [
+    "from redis.asyncio import Redis as AsyncRedis\n",
+    "\n",
+    "async with await AsyncRedis(host='172.25.0.4', port=6379, db=0) as conn:\n",
+    "    checkpointer = RedisSaver(async_connection=conn)\n",
+    "    graph = create_react_agent(model, tools=tools, checkpointer=checkpointer)\n",
+    "    config = {\"configurable\": {\"thread_id\": \"4\"}}\n",
+    "    res = await graph.ainvoke(\n",
+    "        {\"messages\": [(\"human\", \"what's the weather in nyc\")]}, config\n",
+    "    )\n",
+    "    checkpoint_tuples = [c async for c in checkpointer.alist(config)]"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.11.9"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
@@ -0,0 +1,330 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "b23ced4e-dc29-43be-9f94-0c36bb181b8a",
+   "metadata": {},
+   "source": [
+    "# How to stream events from within a tool (without LangChain LLMs / tools)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "7044eeb8-4074-4f9c-8a62-962488744557",
+   "metadata": {},
+   "source": [
+    "In this example we will stream tokens from within tools that an agent is using. We'll also be using OpenAI client library directly, without using LangChain chat models. We will use a ReAct agent as an example."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "a37f60af-43ea-4aa6-847a-df8cc47065f5",
+   "metadata": {},
+   "source": [
+    "## Setup"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "47f79af8-58d8-4a48-8d9a-88823d88701f",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%%capture --no-stderr\n",
+    "%pip install -U langgraph openai"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "0cf6b41d-7fcb-40b6-9a72-229cdd00a094",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "OPENAI_API_KEY:  ········\n"
+     ]
+    }
+   ],
+   "source": [
+    "import getpass\n",
+    "import os\n",
+    "\n",
+    "\n",
+    "def _set_env(var: str):\n",
+    "    if not os.environ.get(var):\n",
+    "        os.environ[var] = getpass.getpass(f\"{var}: \")\n",
+    "\n",
+    "\n",
+    "_set_env(\"OPENAI_API_KEY\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "e3d02ebb-c2e1-4ef7-b187-810d55139317",
+   "metadata": {},
+   "source": [
+    "## Define model, tools and graph"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "3ba684f1-d46b-42e4-95cf-9685209a5992",
+   "metadata": {},
+   "source": [
+    "### Define a node that will call OpenAI API"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "d59234f9-173e-469d-a725-c13e0979663e",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from openai import AsyncOpenAI\n",
+    "from langchain_core.language_models.chat_models import ChatGenerationChunk\n",
+    "from langchain_core.messages import AIMessageChunk\n",
+    "from langchain_core.runnables.config import ensure_config, get_callback_manager_for_config\n",
+    "\n",
+    "openai_client = AsyncOpenAI()\n",
+    "# define tool schema for openai tool calling\n",
+    "\n",
+    "tool = {\n",
+    "  \"type\": \"function\",\n",
+    "  \"function\": {\n",
+    "    \"name\": \"get_items\",\n",
+    "    \"description\": \"Use this tool to look up which items are in the given place.\",\n",
+    "    \"parameters\": {\n",
+    "      \"type\": \"object\",\n",
+    "      \"properties\": {\n",
+    "        \"place\": {\n",
+    "          \"type\": \"string\"\n",
+    "        }\n",
+    "      },\n",
+    "      \"required\": [\n",
+    "        \"place\"\n",
+    "      ]\n",
+    "    }\n",
+    "  }\n",
+    "}\n",
+    "async def call_model(state, config=None):\n",
+    "    config = ensure_config(config | {\"tags\": [\"agent_llm\"]})\n",
+    "    callback_manager = get_callback_manager_for_config(config)\n",
+    "    messages = state[\"messages\"]\n",
+    "    \n",
+    "    llm_run_manager = callback_manager.on_chat_model_start({}, [messages])[0]\n",
+    "    response = await openai_client.chat.completions.create(\n",
+    "        messages=messages,\n",
+    "        model=\"gpt-3.5-turbo\",\n",
+    "        tools=[tool],\n",
+    "        stream=True\n",
+    "    )\n",
+    "\n",
+    "    response_content = \"\"\n",
+    "    role = None\n",
+    "\n",
+    "    tool_call_id = None\n",
+    "    tool_call_function_name = None\n",
+    "    tool_call_function_arguments = \"\"\n",
+    "    async for chunk in response:\n",
+    "        delta = chunk.choices[0].delta\n",
+    "        if delta.role is not None:\n",
+    "            role = delta.role\n",
+    "\n",
+    "        if delta.content:\n",
+    "            response_content += delta.content\n",
+    "            llm_run_manager.on_llm_new_token(delta.content)\n",
+    "\n",
+    "        if delta.tool_calls:\n",
+    "            # note: for simplicity we're only handling a single tool call here\n",
+    "            if delta.tool_calls[0].function.name is not None:\n",
+    "                tool_call_function_name = delta.tool_calls[0].function.name\n",
+    "                tool_call_id = delta.tool_calls[0].id\n",
+    "\n",
+    "            # note: we're wrapping the tools calls in ChatGenerationChunk so that the events from .astream_events in the graph can render tool calls correctly\n",
+    "            tool_call_chunk = ChatGenerationChunk(\n",
+    "                message=AIMessageChunk(content=\"\", additional_kwargs={\"tool_calls\": [delta.tool_calls[0].dict()]})\n",
+    "            )\n",
+    "            llm_run_manager.on_llm_new_token(\"\", chunk=tool_call_chunk)\n",
+    "            tool_call_function_arguments += delta.tool_calls[0].function.arguments\n",
+    "\n",
+    "    if tool_call_function_name is not None:\n",
+    "        tool_calls = [\n",
+    "            {\n",
+    "                \"id\": tool_call_id,\n",
+    "                \"function\": {\"name\": tool_call_function_name, \"arguments\": tool_call_function_arguments},\n",
+    "                \"type\": \"function\"\n",
+    "            }\n",
+    "        ]\n",
+    "    else:\n",
+    "        tool_calls = None\n",
+    "\n",
+    "    response_message = {\n",
+    "        \"role\": role,\n",
+    "        \"content\": response_content,\n",
+    "        \"tool_calls\": tool_calls\n",
+    "    }\n",
+    "    return {\"messages\": [response_message]}"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "3a3877e8-8ace-40d5-ad04-cbf21c6f3250",
+   "metadata": {},
+   "source": [
+    "### Define our tools and a tool-calling node"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "b90941d8-afe4-42ec-9262-9c3b87c3b1ec",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import json\n",
+    "from langchain_core.runnables import RunnableLambda\n",
+    "\n",
+    "async def get_items(place: str) -> str:\n",
+    "    \"\"\"Use this tool to look up which items are in the given place.\"\"\"\n",
+    "    # NOTE: we need to define a special langchain runnable that we'll be using for logging the streaming outputs from within a tool\n",
+    "    tool_logger = RunnableLambda(lambda inputs: inputs).with_config({\"tags\": [\"tool_call\"]})\n",
+    "\n",
+    "    # this can be replaced with any actual streaming logic that you might have\n",
+    "    def stream(place: str):\n",
+    "        if \"bed\" in place:  # For under the bed\n",
+    "            yield from [\"socks\", \"shoes\", \"dust bunnies\"]\n",
+    "        elif \"shelf\" in place:  # For 'shelf'\n",
+    "            yield from [\"books\", \"penciles\", \"pictures\"]\n",
+    "        else:  # if the agent decides to ask about a different place\n",
+    "            yield \"cat snacks\"\n",
+    "\n",
+    "    tokens = []\n",
+    "    for token in stream(place):\n",
+    "        tool_logger.invoke(token)\n",
+    "        tokens.append(token)\n",
+    "\n",
+    "    return \", \".join(tokens)\n",
+    "\n",
+    "# define mapping to look up functions when running tools\n",
+    "function_name_to_function = {\n",
+    "    \"get_items\": get_items\n",
+    "}\n",
+    "\n",
+    "async def call_tools(state):\n",
+    "    messages = state[\"messages\"]\n",
+    "\n",
+    "    tool_call = messages[-1][\"tool_calls\"][0]\n",
+    "    function_name = tool_call[\"function\"][\"name\"]\n",
+    "    function_arguments = tool_call[\"function\"][\"arguments\"]\n",
+    "    arguments = json.loads(function_arguments)\n",
+    "    \n",
+    "    function_response = await function_name_to_function[function_name](**arguments)    \n",
+    "    tool_message = {\n",
+    "        \"tool_call_id\": tool_call[\"id\"],\n",
+    "        \"role\": \"tool\",\n",
+    "        \"name\": function_name,\n",
+    "        \"content\": function_response,\n",
+    "    }\n",
+    "    return {\n",
+    "        \"messages\": [tool_message]\n",
+    "    }"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "6685898c-9a1c-4803-a492-bd70574ebe38",
+   "metadata": {},
+   "source": [
+    "### Define our graph"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "228260be-1f9a-4195-80e0-9604f8a5dba6",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import operator\n",
+    "from typing import Annotated, TypedDict, Literal\n",
+    "\n",
+    "from langgraph.graph import StateGraph, END\n",
+    "\n",
+    "class State(TypedDict):\n",
+    "    messages: Annotated[list, operator.add]\n",
+    "\n",
+    "def should_continue(state) -> Literal[\"tools\", END]:\n",
+    "    messages = state['messages']\n",
+    "    last_message = messages[-1]\n",
+    "    if last_message[\"tool_calls\"]:\n",
+    "        return \"tools\"\n",
+    "    return END\n",
+    "\n",
+    "workflow = StateGraph(State)\n",
+    "workflow.set_entry_point(\"model\")\n",
+    "workflow.add_node(\"model\", call_model)  # i.e. our \"agent\"\n",
+    "workflow.add_node(\"tools\", call_tools)\n",
+    "workflow.add_conditional_edges(\"model\", should_continue)\n",
+    "workflow.add_edge(\"tools\", \"model\")\n",
+    "graph = workflow.compile()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "d046e2ef-f208-4831-ab31-203b2e75a49a",
+   "metadata": {},
+   "source": [
+    "## Stream tokens from within the tool"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 16,
+   "id": "45c96a79-4147-42e3-89fd-d942b2b49f6c",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Tool token socks\n",
+      "Tool token shoes\n",
+      "Tool token dust bunnies\n"
+     ]
+    }
+   ],
+   "source": [
+    "async for event in graph.astream_events({\"messages\": [{\"role\": \"user\", \"content\": \"what's in the bedroom\"}]}, version=\"v2\"):\n",
+    "    tags = event.get(\"tags\", [])\n",
+    "    if event[\"event\"] == \"on_chain_end\" and \"tool_call\" in tags:\n",
+    "        print(\"Tool token\", event[\"data\"][\"output\"])"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "langgraph",
+   "language": "python",
+   "name": "langgraph"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.11.9"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
@@ -0,0 +1,364 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "b23ced4e-dc29-43be-9f94-0c36bb181b8a",
+   "metadata": {},
+   "source": [
+    "# How to stream LLM tokens (without LangChain LLMs)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "7044eeb8-4074-4f9c-8a62-962488744557",
+   "metadata": {},
+   "source": [
+    "In this example we will stream tokens from the language model powering an agent. We'll be using OpenAI client library directly, without using LangChain chat models. We will also use a ReAct agent as an example."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "a37f60af-43ea-4aa6-847a-df8cc47065f5",
+   "metadata": {},
+   "source": [
+    "## Setup"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "47f79af8-58d8-4a48-8d9a-88823d88701f",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%%capture --no-stderr\n",
+    "%pip install -U langgraph openai"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "0cf6b41d-7fcb-40b6-9a72-229cdd00a094",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "OPENAI_API_KEY:  ········\n"
+     ]
+    }
+   ],
+   "source": [
+    "import getpass\n",
+    "import os\n",
+    "\n",
+    "\n",
+    "def _set_env(var: str):\n",
+    "    if not os.environ.get(var):\n",
+    "        os.environ[var] = getpass.getpass(f\"{var}: \")\n",
+    "\n",
+    "\n",
+    "_set_env(\"OPENAI_API_KEY\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "e3d02ebb-c2e1-4ef7-b187-810d55139317",
+   "metadata": {},
+   "source": [
+    "## Define model, tools and graph"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "3ba684f1-d46b-42e4-95cf-9685209a5992",
+   "metadata": {},
+   "source": [
+    "### Define a node that will call OpenAI API"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "d59234f9-173e-469d-a725-c13e0979663e",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from openai import AsyncOpenAI\n",
+    "from langchain_core.language_models.chat_models import ChatGenerationChunk\n",
+    "from langchain_core.messages import AIMessageChunk\n",
+    "from langchain_core.runnables.config import ensure_config, get_callback_manager_for_config\n",
+    "\n",
+    "openai_client = AsyncOpenAI()\n",
+    "# define tool schema for openai tool calling\n",
+    "\n",
+    "tool = {\n",
+    "  \"type\": \"function\",\n",
+    "  \"function\": {\n",
+    "    \"name\": \"get_items\",\n",
+    "    \"description\": \"Use this tool to look up which items are in the given place.\",\n",
+    "    \"parameters\": {\n",
+    "      \"type\": \"object\",\n",
+    "      \"properties\": {\n",
+    "        \"place\": {\n",
+    "          \"type\": \"string\"\n",
+    "        }\n",
+    "      },\n",
+    "      \"required\": [\n",
+    "        \"place\"\n",
+    "      ]\n",
+    "    }\n",
+    "  }\n",
+    "}\n",
+    "async def call_model(state, config=None):\n",
+    "    config = ensure_config(config | {\"tags\": [\"agent_llm\"]})\n",
+    "    callback_manager = get_callback_manager_for_config(config)\n",
+    "    messages = state[\"messages\"]\n",
+    "    \n",
+    "    llm_run_manager = callback_manager.on_chat_model_start({}, [messages])[0]\n",
+    "    response = await openai_client.chat.completions.create(\n",
+    "        messages=messages,\n",
+    "        model=\"gpt-3.5-turbo\",\n",
+    "        tools=[tool],\n",
+    "        stream=True\n",
+    "    )\n",
+    "\n",
+    "    response_content = \"\"\n",
+    "    role = None\n",
+    "\n",
+    "    tool_call_id = None\n",
+    "    tool_call_function_name = None\n",
+    "    tool_call_function_arguments = \"\"\n",
+    "    async for chunk in response:\n",
+    "        delta = chunk.choices[0].delta\n",
+    "        if delta.role is not None:\n",
+    "            role = delta.role\n",
+    "\n",
+    "        if delta.content:\n",
+    "            response_content += delta.content\n",
+    "            llm_run_manager.on_llm_new_token(delta.content)\n",
+    "\n",
+    "        if delta.tool_calls:\n",
+    "            # note: for simplicity we're only handling a single tool call here\n",
+    "            if delta.tool_calls[0].function.name is not None:\n",
+    "                tool_call_function_name = delta.tool_calls[0].function.name\n",
+    "                tool_call_id = delta.tool_calls[0].id\n",
+    "\n",
+    "            # note: we're wrapping the tools calls in ChatGenerationChunk so that the events from .astream_events in the graph can render tool calls correctly\n",
+    "            tool_call_chunk = ChatGenerationChunk(\n",
+    "                message=AIMessageChunk(content=\"\", additional_kwargs={\"tool_calls\": [delta.tool_calls[0].dict()]})\n",
+    "            )\n",
+    "            llm_run_manager.on_llm_new_token(\"\", chunk=tool_call_chunk)\n",
+    "            tool_call_function_arguments += delta.tool_calls[0].function.arguments\n",
+    "\n",
+    "    if tool_call_function_name is not None:\n",
+    "        tool_calls = [\n",
+    "            {\n",
+    "                \"id\": tool_call_id,\n",
+    "                \"function\": {\"name\": tool_call_function_name, \"arguments\": tool_call_function_arguments},\n",
+    "                \"type\": \"function\"\n",
+    "            }\n",
+    "        ]\n",
+    "    else:\n",
+    "        tool_calls = None\n",
+    "\n",
+    "    response_message = {\n",
+    "        \"role\": role,\n",
+    "        \"content\": response_content,\n",
+    "        \"tool_calls\": tool_calls\n",
+    "    }\n",
+    "    return {\"messages\": [response_message]}"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "3a3877e8-8ace-40d5-ad04-cbf21c6f3250",
+   "metadata": {},
+   "source": [
+    "### Define our tools and a tool-calling node"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "b756ea32",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import json\n",
+    "\n",
+    "async def get_items(place: str) -> str:\n",
+    "    \"\"\"Use this tool to look up which items are in the given place.\"\"\"\n",
+    "    if \"bed\" in place:  # For under the bed\n",
+    "        return \"socks, shoes and dust bunnies\"\n",
+    "    if \"shelf\" in place:  # For 'shelf'\n",
+    "        return \"books, penciles and pictures\"\n",
+    "    else:  # if the agent decides to ask about a different place\n",
+    "        return \"cat snacks\"\n",
+    "\n",
+    "# define mapping to look up functions when running tools\n",
+    "function_name_to_function = {\n",
+    "    \"get_items\": get_items\n",
+    "}\n",
+    "\n",
+    "async def call_tools(state):\n",
+    "    messages = state[\"messages\"]\n",
+    "\n",
+    "    tool_call = messages[-1][\"tool_calls\"][0]\n",
+    "    function_name = tool_call[\"function\"][\"name\"]\n",
+    "    function_arguments = tool_call[\"function\"][\"arguments\"]\n",
+    "    arguments = json.loads(function_arguments)\n",
+    "    \n",
+    "    function_response = await function_name_to_function[function_name](**arguments)    \n",
+    "    tool_message = {\n",
+    "        \"tool_call_id\": tool_call[\"id\"],\n",
+    "        \"role\": \"tool\",\n",
+    "        \"name\": function_name,\n",
+    "        \"content\": function_response,\n",
+    "    }\n",
+    "    return {\n",
+    "        \"messages\": [tool_message]\n",
+    "    }"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "6685898c-9a1c-4803-a492-bd70574ebe38",
+   "metadata": {},
+   "source": [
+    "### Define our graph"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "228260be-1f9a-4195-80e0-9604f8a5dba6",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import operator\n",
+    "from typing import Annotated, TypedDict, Literal\n",
+    "\n",
+    "from langgraph.graph import StateGraph, END\n",
+    "\n",
+    "class State(TypedDict):\n",
+    "    messages: Annotated[list, operator.add]\n",
+    "\n",
+    "def should_continue(state) -> Literal[\"tools\", END]:\n",
+    "    messages = state['messages']\n",
+    "    last_message = messages[-1]\n",
+    "    if last_message[\"tool_calls\"]:\n",
+    "        return \"tools\"\n",
+    "    return END\n",
+    "\n",
+    "workflow = StateGraph(State)\n",
+    "workflow.set_entry_point(\"model\")\n",
+    "workflow.add_node(\"model\", call_model)  # i.e. our \"agent\"\n",
+    "workflow.add_node(\"tools\", call_tools)\n",
+    "workflow.add_conditional_edges(\"model\", should_continue)\n",
+    "workflow.add_edge(\"tools\", \"model\")\n",
+    "graph = workflow.compile()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "d046e2ef-f208-4831-ab31-203b2e75a49a",
+   "metadata": {},
+   "source": [
+    "## Stream tokens"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 14,
+   "id": "45c96a79-4147-42e3-89fd-d942b2b49f6c",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/Users/vadymbarda/.virtualenvs/langgraph/lib/python3.11/site-packages/langchain_core/_api/beta_decorator.py:87: LangChainBetaWarning: This API is in beta and may change in the future.\n",
+      "  warn_beta(\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "LLM token {'content': '', 'additional_kwargs': {'tool_calls': [{'index': 0, 'id': 'call_xUcx3IPa8GREPOpjHVj5k9Wx', 'function': {'arguments': '', 'name': 'get_items'}, 'type': 'function'}]}, 'response_metadata': {}, 'type': 'AIMessageChunk', 'name': None, 'id': None, 'example': False, 'tool_calls': [], 'invalid_tool_calls': [{'name': 'get_items', 'args': '', 'id': 'call_xUcx3IPa8GREPOpjHVj5k9Wx', 'error': None}], 'usage_metadata': None, 'tool_call_chunks': [{'name': 'get_items', 'args': '', 'id': 'call_xUcx3IPa8GREPOpjHVj5k9Wx', 'index': 0}]}\n",
+      "LLM token {'content': '', 'additional_kwargs': {'tool_calls': [{'index': 0, 'id': None, 'function': {'arguments': '{\"', 'name': None}, 'type': None}]}, 'response_metadata': {}, 'type': 'AIMessageChunk', 'name': None, 'id': None, 'example': False, 'tool_calls': [{'name': '', 'args': {}, 'id': None}], 'invalid_tool_calls': [], 'usage_metadata': None, 'tool_call_chunks': [{'name': None, 'args': '{\"', 'id': None, 'index': 0}]}\n",
+      "LLM token {'content': '', 'additional_kwargs': {'tool_calls': [{'index': 0, 'id': None, 'function': {'arguments': 'place', 'name': None}, 'type': None}]}, 'response_metadata': {}, 'type': 'AIMessageChunk', 'name': None, 'id': None, 'example': False, 'tool_calls': [], 'invalid_tool_calls': [{'name': None, 'args': 'place', 'id': None, 'error': None}], 'usage_metadata': None, 'tool_call_chunks': [{'name': None, 'args': 'place', 'id': None, 'index': 0}]}\n",
+      "LLM token {'content': '', 'additional_kwargs': {'tool_calls': [{'index': 0, 'id': None, 'function': {'arguments': '\":\"', 'name': None}, 'type': None}]}, 'response_metadata': {}, 'type': 'AIMessageChunk', 'name': None, 'id': None, 'example': False, 'tool_calls': [], 'invalid_tool_calls': [{'name': None, 'args': '\":\"', 'id': None, 'error': None}], 'usage_metadata': None, 'tool_call_chunks': [{'name': None, 'args': '\":\"', 'id': None, 'index': 0}]}\n",
+      "LLM token {'content': '', 'additional_kwargs': {'tool_calls': [{'index': 0, 'id': None, 'function': {'arguments': 'bed', 'name': None}, 'type': None}]}, 'response_metadata': {}, 'type': 'AIMessageChunk', 'name': None, 'id': None, 'example': False, 'tool_calls': [], 'invalid_tool_calls': [{'name': None, 'args': 'bed', 'id': None, 'error': None}], 'usage_metadata': None, 'tool_call_chunks': [{'name': None, 'args': 'bed', 'id': None, 'index': 0}]}\n",
+      "LLM token {'content': '', 'additional_kwargs': {'tool_calls': [{'index': 0, 'id': None, 'function': {'arguments': 'room', 'name': None}, 'type': None}]}, 'response_metadata': {}, 'type': 'AIMessageChunk', 'name': None, 'id': None, 'example': False, 'tool_calls': [], 'invalid_tool_calls': [{'name': None, 'args': 'room', 'id': None, 'error': None}], 'usage_metadata': None, 'tool_call_chunks': [{'name': None, 'args': 'room', 'id': None, 'index': 0}]}\n",
+      "LLM token {'content': '', 'additional_kwargs': {'tool_calls': [{'index': 0, 'id': None, 'function': {'arguments': '\"}', 'name': None}, 'type': None}]}, 'response_metadata': {}, 'type': 'AIMessageChunk', 'name': None, 'id': None, 'example': False, 'tool_calls': [], 'invalid_tool_calls': [{'name': None, 'args': '\"}', 'id': None, 'error': None}], 'usage_metadata': None, 'tool_call_chunks': [{'name': None, 'args': '\"}', 'id': None, 'index': 0}]}\n",
+      "LLM token {'content': 'In', 'additional_kwargs': {}, 'response_metadata': {}, 'type': 'AIMessageChunk', 'name': None, 'id': None, 'example': False, 'tool_calls': [], 'invalid_tool_calls': [], 'usage_metadata': None, 'tool_call_chunks': []}\n",
+      "LLM token {'content': ' the', 'additional_kwargs': {}, 'response_metadata': {}, 'type': 'AIMessageChunk', 'name': None, 'id': None, 'example': False, 'tool_calls': [], 'invalid_tool_calls': [], 'usage_metadata': None, 'tool_call_chunks': []}\n",
+      "LLM token {'content': ' bedroom', 'additional_kwargs': {}, 'response_metadata': {}, 'type': 'AIMessageChunk', 'name': None, 'id': None, 'example': False, 'tool_calls': [], 'invalid_tool_calls': [], 'usage_metadata': None, 'tool_call_chunks': []}\n",
+      "LLM token {'content': ',', 'additional_kwargs': {}, 'response_metadata': {}, 'type': 'AIMessageChunk', 'name': None, 'id': None, 'example': False, 'tool_calls': [], 'invalid_tool_calls': [], 'usage_metadata': None, 'tool_call_chunks': []}\n",
+      "LLM token {'content': ' you', 'additional_kwargs': {}, 'response_metadata': {}, 'type': 'AIMessageChunk', 'name': None, 'id': None, 'example': False, 'tool_calls': [], 'invalid_tool_calls': [], 'usage_metadata': None, 'tool_call_chunks': []}\n",
+      "LLM token {'content': ' have', 'additional_kwargs': {}, 'response_metadata': {}, 'type': 'AIMessageChunk', 'name': None, 'id': None, 'example': False, 'tool_calls': [], 'invalid_tool_calls': [], 'usage_metadata': None, 'tool_call_chunks': []}\n",
+      "LLM token {'content': ' socks', 'additional_kwargs': {}, 'response_metadata': {}, 'type': 'AIMessageChunk', 'name': None, 'id': None, 'example': False, 'tool_calls': [], 'invalid_tool_calls': [], 'usage_metadata': None, 'tool_call_chunks': []}\n",
+      "LLM token {'content': ',', 'additional_kwargs': {}, 'response_metadata': {}, 'type': 'AIMessageChunk', 'name': None, 'id': None, 'example': False, 'tool_calls': [], 'invalid_tool_calls': [], 'usage_metadata': None, 'tool_call_chunks': []}\n",
+      "LLM token {'content': ' shoes', 'additional_kwargs': {}, 'response_metadata': {}, 'type': 'AIMessageChunk', 'name': None, 'id': None, 'example': False, 'tool_calls': [], 'invalid_tool_calls': [], 'usage_metadata': None, 'tool_call_chunks': []}\n",
+      "LLM token {'content': ',', 'additional_kwargs': {}, 'response_metadata': {}, 'type': 'AIMessageChunk', 'name': None, 'id': None, 'example': False, 'tool_calls': [], 'invalid_tool_calls': [], 'usage_metadata': None, 'tool_call_chunks': []}\n",
+      "LLM token {'content': ' and', 'additional_kwargs': {}, 'response_metadata': {}, 'type': 'AIMessageChunk', 'name': None, 'id': None, 'example': False, 'tool_calls': [], 'invalid_tool_calls': [], 'usage_metadata': None, 'tool_call_chunks': []}\n",
+      "LLM token {'content': ' some', 'additional_kwargs': {}, 'response_metadata': {}, 'type': 'AIMessageChunk', 'name': None, 'id': None, 'example': False, 'tool_calls': [], 'invalid_tool_calls': [], 'usage_metadata': None, 'tool_call_chunks': []}\n",
+      "LLM token {'content': ' dust', 'additional_kwargs': {}, 'response_metadata': {}, 'type': 'AIMessageChunk', 'name': None, 'id': None, 'example': False, 'tool_calls': [], 'invalid_tool_calls': [], 'usage_metadata': None, 'tool_call_chunks': []}\n",
+      "LLM token {'content': ' b', 'additional_kwargs': {}, 'response_metadata': {}, 'type': 'AIMessageChunk', 'name': None, 'id': None, 'example': False, 'tool_calls': [], 'invalid_tool_calls': [], 'usage_metadata': None, 'tool_call_chunks': []}\n",
+      "LLM token {'content': 'unn', 'additional_kwargs': {}, 'response_metadata': {}, 'type': 'AIMessageChunk', 'name': None, 'id': None, 'example': False, 'tool_calls': [], 'invalid_tool_calls': [], 'usage_metadata': None, 'tool_call_chunks': []}\n",
+      "LLM token {'content': 'ies', 'additional_kwargs': {}, 'response_metadata': {}, 'type': 'AIMessageChunk', 'name': None, 'id': None, 'example': False, 'tool_calls': [], 'invalid_tool_calls': [], 'usage_metadata': None, 'tool_call_chunks': []}\n",
+      "LLM token {'content': '.', 'additional_kwargs': {}, 'response_metadata': {}, 'type': 'AIMessageChunk', 'name': None, 'id': None, 'example': False, 'tool_calls': [], 'invalid_tool_calls': [], 'usage_metadata': None, 'tool_call_chunks': []}\n",
+      "LLM token {'content': ' Is', 'additional_kwargs': {}, 'response_metadata': {}, 'type': 'AIMessageChunk', 'name': None, 'id': None, 'example': False, 'tool_calls': [], 'invalid_tool_calls': [], 'usage_metadata': None, 'tool_call_chunks': []}\n",
+      "LLM token {'content': ' there', 'additional_kwargs': {}, 'response_metadata': {}, 'type': 'AIMessageChunk', 'name': None, 'id': None, 'example': False, 'tool_calls': [], 'invalid_tool_calls': [], 'usage_metadata': None, 'tool_call_chunks': []}\n",
+      "LLM token {'content': ' anything', 'additional_kwargs': {}, 'response_metadata': {}, 'type': 'AIMessageChunk', 'name': None, 'id': None, 'example': False, 'tool_calls': [], 'invalid_tool_calls': [], 'usage_metadata': None, 'tool_call_chunks': []}\n",
+      "LLM token {'content': ' else', 'additional_kwargs': {}, 'response_metadata': {}, 'type': 'AIMessageChunk', 'name': None, 'id': None, 'example': False, 'tool_calls': [], 'invalid_tool_calls': [], 'usage_metadata': None, 'tool_call_chunks': []}\n",
+      "LLM token {'content': ' you', 'additional_kwargs': {}, 'response_metadata': {}, 'type': 'AIMessageChunk', 'name': None, 'id': None, 'example': False, 'tool_calls': [], 'invalid_tool_calls': [], 'usage_metadata': None, 'tool_call_chunks': []}\n",
+      "LLM token {'content': ' would', 'additional_kwargs': {}, 'response_metadata': {}, 'type': 'AIMessageChunk', 'name': None, 'id': None, 'example': False, 'tool_calls': [], 'invalid_tool_calls': [], 'usage_metadata': None, 'tool_call_chunks': []}\n",
+      "LLM token {'content': ' like', 'additional_kwargs': {}, 'response_metadata': {}, 'type': 'AIMessageChunk', 'name': None, 'id': None, 'example': False, 'tool_calls': [], 'invalid_tool_calls': [], 'usage_metadata': None, 'tool_call_chunks': []}\n",
+      "LLM token {'content': ' to', 'additional_kwargs': {}, 'response_metadata': {}, 'type': 'AIMessageChunk', 'name': None, 'id': None, 'example': False, 'tool_calls': [], 'invalid_tool_calls': [], 'usage_metadata': None, 'tool_call_chunks': []}\n",
+      "LLM token {'content': ' know', 'additional_kwargs': {}, 'response_metadata': {}, 'type': 'AIMessageChunk', 'name': None, 'id': None, 'example': False, 'tool_calls': [], 'invalid_tool_calls': [], 'usage_metadata': None, 'tool_call_chunks': []}\n",
+      "LLM token {'content': '?', 'additional_kwargs': {}, 'response_metadata': {}, 'type': 'AIMessageChunk', 'name': None, 'id': None, 'example': False, 'tool_calls': [], 'invalid_tool_calls': [], 'usage_metadata': None, 'tool_call_chunks': []}\n"
+     ]
+    }
+   ],
+   "source": [
+    "async for event in graph.astream_events({\"messages\": [{\"role\": \"user\", \"content\": \"what's in the bedroom\"}]}, version=\"v2\"):\n",
+    "    tags = event.get(\"tags\", [])\n",
+    "    if event[\"event\"] == \"on_chat_model_stream\" and \"agent_llm\" in tags:\n",
+    "        print(\"LLM token\", event[\"data\"][\"chunk\"].dict())"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "adb0f7bc-6e51-478e-bd32-8f72df072d6c",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "langgraph",
+   "language": "python",
+   "name": "langgraph"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.11.9"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
@@ -215,13 +215,13 @@ css = ["tinycss2 (>=1.1.0,<1.3)"]

 [[package]]
 name = "certifi"
-version = "2024.6.2"
+version = "2024.7.4"
 description = "Python package for providing Mozilla's CA Bundle."
 optional = false
 python-versions = ">=3.6"
 files = [
-    {file = "certifi-2024.6.2-py3-none-any.whl", hash = "sha256:ddc6c8ce995e6987e7faf5e3f1b02b302836a0e5d98ece18392cb1a36c72ad56"},
-    {file = "certifi-2024.6.2.tar.gz", hash = "sha256:3cd43f1c6fa7dedc5899d69d3ad0398fd018ad1a17fba83ddaf78aa46c747516"},
+    {file = "certifi-2024.7.4-py3-none-any.whl", hash = "sha256:c198e21b1289c2ab85ee4e67bb4b4ef3ead0892059901a8d5b622f24a1101e90"},
+    {file = "certifi-2024.7.4.tar.gz", hash = "sha256:5a1e7645bc0ec61a09e26c36f6106dd4cf40c6db3a1fb6352b0244e7fb057c7b"},
 ]

 [[package]]
@@ -1105,7 +1105,7 @@ files = [

 [[package]]
 name = "langgraph-cli"
-version = "0.1.39"
+version = "0.1.48"
 description = "CLI for interacting with LangGraph API"
 optional = false
 python-versions = "^3.9.0,<4.0"
@@ -1117,7 +1117,7 @@ click = "^8.1.7"

 [package.source]
 type = "directory"
-url = ".."
+url = "../../cli"

 [[package]]
 name = "langgraph-sdk"
@@ -193,7 +193,9 @@ def _update_graph_paths(

 def config_to_docker(config_path: pathlib.Path, config: Config, base_image: str):
    # configure pip
-    pip_install = "PYTHONDONTWRITEBYTECODE=1 pip install --no-cache-dir -c /api/constraints.txt"
+    pip_install = (
+        "PYTHONDONTWRITEBYTECODE=1 pip install --no-cache-dir -c /api/constraints.txt"
+    )
    if config.get("pip_config_file"):
        pip_install = f"PIP_CONFIG_FILE=/pipconfig.txt {pip_install}"
    pip_config_file_str = (
@@ -92,7 +92,7 @@ services:
            dockerfile_inline: |
                FROM langchain/langgraph-api:3.11
                ADD . /deps/
-                RUN pip install -c /api/constraints.txt -e /deps/*
+                RUN PYTHONDONTWRITEBYTECODE=1 pip install --no-cache-dir -c /api/constraints.txt -e /deps/*
                ENV LANGSERVE_GRAPHS='{{"agent": "agent.py:graph"}}'
                WORKDIR /deps/
        
@@ -81,7 +81,7 @@ RUN set -ex && \\
                '"*" = ["**/*"]'; do \\
        echo "$line" >> /deps/__outer_unit_tests/pyproject.toml; \\
    done
-RUN pip install -c /api/constraints.txt -e /deps/*
+RUN PYTHONDONTWRITEBYTECODE=1 pip install --no-cache-dir -c /api/constraints.txt -e /deps/*
 ENV LANGSERVE_GRAPHS='{"agent": "/deps/__outer_unit_tests/unit_tests/agent.py:graph"}'
 WORKDIR /deps/__outer_unit_tests/unit_tests\
 """
@@ -113,7 +113,7 @@ RUN set -ex && \\
                '"*" = ["**/*"]'; do \\
        echo "$line" >> /deps/__outer_unit_tests/pyproject.toml; \\
    done
-RUN PIP_CONFIG_FILE=/pipconfig.txt pip install -c /api/constraints.txt -e /deps/*
+RUN PIP_CONFIG_FILE=/pipconfig.txt PYTHONDONTWRITEBYTECODE=1 pip install --no-cache-dir -c /api/constraints.txt -e /deps/*
 ENV LANGSERVE_GRAPHS='{"agent": "/deps/__outer_unit_tests/unit_tests/agent.py:graph"}'
 WORKDIR /deps/__outer_unit_tests/unit_tests\
 """
@@ -163,7 +163,7 @@ RUN set -ex && \\
                '"*" = ["**/*"]'; do \\
        echo "$line" >> /deps/__outer_graphs/pyproject.toml; \\
    done
-RUN pip install -c /api/constraints.txt -e /deps/*
+RUN PYTHONDONTWRITEBYTECODE=1 pip install --no-cache-dir -c /api/constraints.txt -e /deps/*
 ENV LANGSERVE_GRAPHS='{"agent": "/deps/__outer_graphs/src/agent.py:graph"}'\
 """
    assert clean_empty_lines(actual_docker_stdin) == expected_docker_stdin
@@ -192,7 +192,7 @@ dependencies = ["langchain"]"""
    os.remove(pyproject_path)
    expected_docker_stdin = """FROM langchain/langgraph-api:3.11
 ADD . /deps/unit_tests
-RUN pip install -c /api/constraints.txt -e /deps/*
+RUN PYTHONDONTWRITEBYTECODE=1 pip install --no-cache-dir -c /api/constraints.txt -e /deps/*
 ENV LANGSERVE_GRAPHS='{"agent": "/deps/unit_tests/graphs/agent.py:graph"}'
 WORKDIR /deps/unit_tests"""
    assert clean_empty_lines(actual_docker_stdin) == expected_docker_stdin
@@ -217,7 +217,7 @@ def test_config_to_docker_end_to_end():
 ARG meow
 ARG foo
 ADD pipconfig.txt /pipconfig.txt
-RUN PIP_CONFIG_FILE=/pipconfig.txt pip install -c /api/constraints.txt langchain langchain_openai
+RUN PIP_CONFIG_FILE=/pipconfig.txt PYTHONDONTWRITEBYTECODE=1 pip install --no-cache-dir -c /api/constraints.txt langchain langchain_openai
 ADD ./graphs/ /deps/__outer_graphs/src
 RUN set -ex && \\
    for line in '[project]' \\
@@ -227,7 +227,7 @@ RUN set -ex && \\
                '"*" = ["**/*"]'; do \\
        echo "$line" >> /deps/__outer_graphs/pyproject.toml; \\
    done
-RUN PIP_CONFIG_FILE=/pipconfig.txt pip install -c /api/constraints.txt -e /deps/*
+RUN PIP_CONFIG_FILE=/pipconfig.txt PYTHONDONTWRITEBYTECODE=1 pip install --no-cache-dir -c /api/constraints.txt -e /deps/*
 ENV LANGSERVE_GRAPHS='{"agent": "/deps/__outer_graphs/src/agent.py:graph"}'"""
    assert clean_empty_lines(actual_docker_stdin) == expected_docker_stdin

@@ -251,7 +251,7 @@ def test_config_to_compose_simple_config():
                                '"*" = ["**/*"]'; do \\
                        echo "$line" >> /deps/__outer_unit_tests/pyproject.toml; \\
                    done
-                RUN pip install -c /api/constraints.txt -e /deps/*
+                RUN PYTHONDONTWRITEBYTECODE=1 pip install --no-cache-dir -c /api/constraints.txt -e /deps/*
                ENV LANGSERVE_GRAPHS='{"agent": "/deps/__outer_unit_tests/unit_tests/agent.py:graph"}'
                WORKDIR /deps/__outer_unit_tests/unit_tests
        """
@@ -281,7 +281,7 @@ def test_config_to_compose_env_vars():
                                '"*" = ["**/*"]'; do \\
                        echo "$line" >> /deps/__outer_unit_tests/pyproject.toml; \\
                    done
-                RUN pip install -c /api/constraints.txt -e /deps/*
+                RUN PYTHONDONTWRITEBYTECODE=1 pip install --no-cache-dir -c /api/constraints.txt -e /deps/*
                ENV LANGSERVE_GRAPHS='{"agent": "/deps/__outer_unit_tests/unit_tests/agent.py:graph"}'
                WORKDIR /deps/__outer_unit_tests/unit_tests
        """
@@ -318,7 +318,7 @@ def test_config_to_compose_env_file():
                                '"*" = ["**/*"]'; do \\
                        echo "$line" >> /deps/__outer_unit_tests/pyproject.toml; \\
                    done
-                RUN pip install -c /api/constraints.txt -e /deps/*
+                RUN PYTHONDONTWRITEBYTECODE=1 pip install --no-cache-dir -c /api/constraints.txt -e /deps/*
                ENV LANGSERVE_GRAPHS='{"agent": "/deps/__outer_unit_tests/unit_tests/agent.py:graph"}'
                WORKDIR /deps/__outer_unit_tests/unit_tests
        """
@@ -348,7 +348,7 @@ def test_config_to_compose_watch():
                                '"*" = ["**/*"]'; do \\
                        echo "$line" >> /deps/__outer_unit_tests/pyproject.toml; \\
                    done
-                RUN pip install -c /api/constraints.txt -e /deps/*
+                RUN PYTHONDONTWRITEBYTECODE=1 pip install --no-cache-dir -c /api/constraints.txt -e /deps/*
                ENV LANGSERVE_GRAPHS='{"agent": "/deps/__outer_unit_tests/unit_tests/agent.py:graph"}'
                WORKDIR /deps/__outer_unit_tests/unit_tests
        
@@ -387,7 +387,7 @@ def test_config_to_compose_end_to_end():
                                '"*" = ["**/*"]'; do \\
                        echo "$line" >> /deps/__outer_unit_tests/pyproject.toml; \\
                    done
-                RUN pip install -c /api/constraints.txt -e /deps/*
+                RUN PYTHONDONTWRITEBYTECODE=1 pip install --no-cache-dir -c /api/constraints.txt -e /deps/*
                ENV LANGSERVE_GRAPHS='{"agent": "/deps/__outer_unit_tests/unit_tests/agent.py:graph"}'
                WORKDIR /deps/__outer_unit_tests/unit_tests
        
@@ -57,8 +57,6 @@ export LANGSMITH_API_KEY=lsv2_sk_...
 ```

 ```python
-from typing import Annotated, Literal, TypedDict
-
 from langchain_core.messages import HumanMessage
 from langchain_anthropic import ChatAnthropic
 from langchain_core.tools import tool
@@ -84,7 +82,7 @@ tool_node = ToolNode(tools)
 model = ChatAnthropic(model="claude-3-5-sonnet-20240620", temperature=0).bind_tools(tools)

 # Define the function that determines whether to continue or not
-def should_continue(state: MessagesState) -> Literal["tools", END]:
+def should_continue(state: MessagesState) -> Union[Literal["tools"], type(END)]:
    messages = state['messages']
    last_message = messages[-1]
    # If the LLM makes a tool call, then we route to the "tools" node
@@ -2,7 +2,16 @@ import asyncio
 import functools
 from contextlib import AbstractAsyncContextManager
 from types import TracebackType
-from typing import Any, AsyncIterator, Dict, Iterator, Optional, TypeVar
+from typing import (
+    Any,
+    AsyncIterator,
+    Dict,
+    Iterator,
+    Optional,
+    Sequence,
+    Tuple,
+    TypeVar,
+)

 import aiosqlite
 from langchain_core.runnables import RunnableConfig
@@ -203,6 +212,15 @@ class AsyncSqliteSaver(BaseCheckpointSaver, AbstractAsyncContextManager):
                    metadata BLOB,
                    PRIMARY KEY (thread_id, thread_ts)
                );
+                CREATE TABLE IF NOT EXISTS writes (
+                    thread_id TEXT NOT NULL,
+                    thread_ts TEXT NOT NULL,
+                    task_id TEXT NOT NULL,
+                    idx INTEGER NOT NULL,
+                    channel TEXT NOT NULL,
+                    value BLOB,
+                    PRIMARY KEY (thread_id, thread_ts, task_id, idx)
+                );
                """
            ):
                await self.conn.commit()
@@ -224,56 +242,58 @@ class AsyncSqliteSaver(BaseCheckpointSaver, AbstractAsyncContextManager):
            Optional[CheckpointTuple]: The retrieved checkpoint tuple, or None if no matching checkpoint was found.
        """
        await self.setup()
-        if config["configurable"].get("thread_ts"):
-            async with self.conn.execute(
-                "SELECT checkpoint, parent_ts, metadata FROM checkpoints WHERE thread_id = ? AND thread_ts = ?",
-                (
-                    str(config["configurable"]["thread_id"]),
-                    str(config["configurable"]["thread_ts"]),
-                ),
-            ) as cursor:
-                if value := await cursor.fetchone():
-                    return CheckpointTuple(
-                        config,
-                        self.serde.loads(value[0]),
-                        self.serde.loads(value[2]) if value[2] is not None else {},
-                        (
-                            {
-                                "configurable": {
-                                    "thread_id": config["configurable"]["thread_id"],
-                                    "thread_ts": value[1],
-                                }
-                            }
-                            if value[1]
-                            else None
-                        ),
-                    )
-        else:
-            async with self.conn.execute(
-                "SELECT thread_id, thread_ts, parent_ts, checkpoint, metadata FROM checkpoints WHERE thread_id = ? ORDER BY thread_ts DESC LIMIT 1",
-                (str(config["configurable"]["thread_id"]),),
-            ) as cursor:
-                if value := await cursor.fetchone():
-                    return CheckpointTuple(
+        async with self.conn.cursor() as cur:
+            # find the latest checkpoint for the thread_id
+            if config["configurable"].get("thread_ts"):
+                await cur.execute(
+                    "SELECT thread_id, thread_ts, parent_ts, checkpoint, metadata FROM checkpoints WHERE thread_id = ? AND thread_ts = ?",
+                    (
+                        str(config["configurable"]["thread_id"]),
+                        str(config["configurable"]["thread_ts"]),
+                    ),
+                )
+            else:
+                await cur.execute(
+                    "SELECT thread_id, thread_ts, parent_ts, checkpoint, metadata FROM checkpoints WHERE thread_id = ? ORDER BY thread_ts DESC LIMIT 1",
+                    (str(config["configurable"]["thread_id"]),),
+                )
+            # if a checkpoint is found, return it
+            if value := await cur.fetchone():
+                if not config["configurable"].get("thread_ts"):
+                    config = {
+                        "configurable": {
+                            "thread_id": value[0],
+                            "thread_ts": value[1],
+                        }
+                    }
+                # find any pending writes
+                await cur.execute(
+                    "SELECT task_id, channel, value FROM writes WHERE thread_id = ? AND thread_ts = ?",
+                    (
+                        str(config["configurable"]["thread_id"]),
+                        str(config["configurable"]["thread_ts"]),
+                    ),
+                )
+                # deserialize the checkpoint and metadata
+                return CheckpointTuple(
+                    config,
+                    self.serde.loads(value[3]),
+                    self.serde.loads(value[4]) if value[4] is not None else {},
+                    (
                        {
                            "configurable": {
                                "thread_id": value[0],
-                                "thread_ts": value[1],
+                                "thread_ts": value[2],
                            }
-                        },
-                        self.serde.loads(value[3]),
-                        self.serde.loads(value[4]) if value[4] is not None else {},
-                        (
-                            {
-                                "configurable": {
-                                    "thread_id": value[0],
-                                    "thread_ts": value[2],
-                                }
-                            }
-                            if value[2]
-                            else None
-                        ),
-                    )
+                        }
+                        if value[2]
+                        else None
+                    ),
+                    [
+                        (task_id, channel, self.serde.loads(value))
+                        async for task_id, channel, value in cur
+                    ],
+                )

    async def alist(
        self,
@@ -358,3 +378,26 @@ class AsyncSqliteSaver(BaseCheckpointSaver, AbstractAsyncContextManager):
                "thread_ts": checkpoint["id"],
            }
        }
+
+    async def aput_writes(
+        self,
+        config: RunnableConfig,
+        writes: Sequence[Tuple[str, Any]],
+        task_id: str,
+    ) -> None:
+        await self.setup()
+        async with self.conn.executemany(
+            "INSERT OR REPLACE INTO writes (thread_id, thread_ts, task_id, idx, channel, value) VALUES (?, ?, ?, ?, ?, ?)",
+            [
+                (
+                    str(config["configurable"]["thread_id"]),
+                    str(config["configurable"]["thread_ts"]),
+                    task_id,
+                    idx,
+                    channel,
+                    self.serde.dumps(value),
+                )
+                for idx, (channel, value) in enumerate(writes)
+            ],
+        ):
+            await self.conn.commit()
@@ -10,6 +10,7 @@ from typing import (
    Literal,
    NamedTuple,
    Optional,
+    Tuple,
    TypedDict,
    TypeVar,
    Union,
@@ -117,6 +118,7 @@ class CheckpointTuple(NamedTuple):
    checkpoint: Checkpoint
    metadata: CheckpointMetadata
    parent_config: Optional[RunnableConfig] = None
+    pending_writes: Optional[List[Tuple[str, str, Any]]] = None


 CheckpointThreadId = ConfigurableFieldSpec(
@@ -177,6 +179,16 @@ class BaseCheckpointSaver(ABC):
    ) -> RunnableConfig:
        raise NotImplementedError

+    def put_writes(
+        self,
+        config: RunnableConfig,
+        writes: List[Tuple[str, Any]],
+        task_id: str,
+    ) -> None:
+        raise NotImplementedError(
+            "This method was added in langgraph 0.1.7. Please update your checkpointer to implement it."
+        )
+
    async def aget(self, config: RunnableConfig) -> Optional[Checkpoint]:
        if value := await self.aget_tuple(config):
            return value.checkpoint
@@ -203,6 +215,16 @@ class BaseCheckpointSaver(ABC):
    ) -> RunnableConfig:
        raise NotImplementedError

+    async def aput_writes(
+        self,
+        config: RunnableConfig,
+        writes: List[Tuple[str, Any]],
+        task_id: str,
+    ) -> None:
+        raise NotImplementedError(
+            "This method was added in langgraph 0.1.7. Please update your checkpointer to implement it."
+        )
+
    def get_next_version(self, current: Optional[V], channel: BaseChannel) -> V:
        """Get the next version of a channel. Default is to use int versions, incrementing by 1. If you override, you can use str/int/float versions,
        as long as they are monotonically increasing."""
@@ -1,7 +1,7 @@
 import asyncio
 from collections import defaultdict
 from functools import partial
-from typing import Any, AsyncIterator, Dict, Iterator, Optional
+from typing import Any, AsyncIterator, Dict, Iterator, List, Optional, Tuple

 from langchain_core.runnables import RunnableConfig

@@ -53,6 +53,7 @@ class MemorySaver(BaseCheckpointSaver):
    ) -> None:
        super().__init__(serde=serde)
        self.storage = defaultdict(dict)
+        self.writes = defaultdict(list)

    def get_tuple(self, config: RunnableConfig) -> Optional[CheckpointTuple]:
        """Get a checkpoint tuple from the in-memory storage.
@@ -72,19 +73,27 @@ class MemorySaver(BaseCheckpointSaver):
        if ts := config["configurable"].get("thread_ts"):
            if saved := self.storage[thread_id].get(ts):
                checkpoint, metadata = saved
+                writes = self.writes[(thread_id, ts)]
                return CheckpointTuple(
                    config=config,
                    checkpoint=self.serde.loads(checkpoint),
                    metadata=self.serde.loads(metadata),
+                    pending_writes=[
+                        (id, c, self.serde.loads(v)) for id, c, v in writes
+                    ],
                )
        else:
            if checkpoints := self.storage[thread_id]:
                ts = max(checkpoints.keys())
                checkpoint, metadata = checkpoints[ts]
+                writes = self.writes[(thread_id, ts)]
                return CheckpointTuple(
                    config={"configurable": {"thread_id": thread_id, "thread_ts": ts}},
                    checkpoint=self.serde.loads(checkpoint),
                    metadata=self.serde.loads(metadata),
+                    pending_writes=[
+                        (id, c, self.serde.loads(v)) for id, c, v in writes
+                    ],
                )

    def list(
@@ -168,6 +177,30 @@ class MemorySaver(BaseCheckpointSaver):
            }
        }

+    def put_writes(
+        self,
+        config: RunnableConfig,
+        writes: List[Tuple[str, Any]],
+        task_id: str,
+    ) -> RunnableConfig:
+        """Save a list of writes to the in-memory storage.
+
+        This method saves a list of writes to the in-memory storage. The writes are associated
+        with the provided config.
+
+        Args:
+            config (RunnableConfig): The config to associate with the writes.
+            writes (list[tuple[str, Any]]): The writes to save.
+
+        Returns:
+            RunnableConfig: The updated config containing the saved writes' timestamp.
+        """
+        thread_id = config["configurable"]["thread_id"]
+        ts = config["configurable"]["thread_ts"]
+        self.writes[(thread_id, ts)].extend(
+            [(task_id, c, self.serde.dumps(v)) for c, v in writes]
+        )
+
    async def aget_tuple(self, config: RunnableConfig) -> Optional[CheckpointTuple]:
        """Asynchronous version of get_tuple.

@@ -224,3 +257,13 @@ class MemorySaver(BaseCheckpointSaver):
        return await asyncio.get_running_loop().run_in_executor(
            None, self.put, config, checkpoint, metadata
        )
+
+    async def aput_writes(
+        self,
+        config: RunnableConfig,
+        writes: List[Tuple[str, Any]],
+        task_id: str,
+    ) -> RunnableConfig:
+        return await asyncio.get_running_loop().run_in_executor(
+            None, self.put_writes, config, writes, task_id
+        )
@@ -171,6 +171,15 @@ class SqliteSaver(BaseCheckpointSaver, AbstractContextManager):
                metadata BLOB,
                PRIMARY KEY (thread_id, thread_ts)
            );
+            CREATE TABLE IF NOT EXISTS writes (
+                thread_id TEXT NOT NULL,
+                thread_ts TEXT NOT NULL,
+                task_id TEXT NOT NULL,
+                idx INTEGER NOT NULL,
+                channel TEXT NOT NULL,
+                value BLOB,
+                PRIMARY KEY (thread_id, thread_ts, task_id, idx)
+            );
            """
        )

@@ -233,56 +242,57 @@ class SqliteSaver(BaseCheckpointSaver, AbstractContextManager):
            CheckpointTuple(...)
        """  # noqa
        with self.cursor(transaction=False) as cur:
+            # find the latest checkpoint for the thread_id
            if config["configurable"].get("thread_ts"):
                cur.execute(
-                    "SELECT checkpoint, parent_ts, metadata FROM checkpoints WHERE thread_id = ? AND thread_ts = ?",
+                    "SELECT thread_id, thread_ts, parent_ts, checkpoint, metadata FROM checkpoints WHERE thread_id = ? AND thread_ts = ?",
                    (
                        str(config["configurable"]["thread_id"]),
                        str(config["configurable"]["thread_ts"]),
                    ),
                )
-                if value := cur.fetchone():
-                    return CheckpointTuple(
-                        config,
-                        self.serde.loads(value[0]),
-                        self.serde.loads(value[2]) if value[2] is not None else {},
-                        (
-                            {
-                                "configurable": {
-                                    "thread_id": config["configurable"]["thread_id"],
-                                    "thread_ts": value[1],
-                                }
-                            }
-                            if value[1]
-                            else None
-                        ),
-                    )
            else:
                cur.execute(
                    "SELECT thread_id, thread_ts, parent_ts, checkpoint, metadata FROM checkpoints WHERE thread_id = ? ORDER BY thread_ts DESC LIMIT 1",
                    (str(config["configurable"]["thread_id"]),),
                )
-                if value := cur.fetchone():
-                    return CheckpointTuple(
+            # if a checkpoint is found, return it
+            if value := cur.fetchone():
+                if not config["configurable"].get("thread_ts"):
+                    config = {
+                        "configurable": {
+                            "thread_id": value[0],
+                            "thread_ts": value[1],
+                        }
+                    }
+                # find any pending writes
+                cur.execute(
+                    "SELECT task_id, channel, value FROM writes WHERE thread_id = ? AND thread_ts = ?",
+                    (
+                        str(config["configurable"]["thread_id"]),
+                        str(config["configurable"]["thread_ts"]),
+                    ),
+                )
+                # deserialize the checkpoint and metadata
+                return CheckpointTuple(
+                    config,
+                    self.serde.loads(value[3]),
+                    self.serde.loads(value[4]) if value[4] is not None else {},
+                    (
                        {
                            "configurable": {
                                "thread_id": value[0],
-                                "thread_ts": value[1],
+                                "thread_ts": value[2],
                            }
-                        },
-                        self.serde.loads(value[3]),
-                        self.serde.loads(value[4]) if value[4] is not None else {},
-                        (
-                            {
-                                "configurable": {
-                                    "thread_id": value[0],
-                                    "thread_ts": value[2],
-                                }
-                            }
-                            if value[2]
-                            else None
-                        ),
-                    )
+                        }
+                        if value[2]
+                        else None
+                    ),
+                    [
+                        (task_id, channel, self.serde.loads(value))
+                        for task_id, channel, value in cur
+                    ],
+                )

    def list(
        self,
@@ -394,6 +404,28 @@ class SqliteSaver(BaseCheckpointSaver, AbstractContextManager):
            }
        }

+    def put_writes(
+        self,
+        config: RunnableConfig,
+        writes: Sequence[Tuple[str, Any]],
+        task_id: str,
+    ) -> None:
+        with self.lock, self.cursor() as cur:
+            cur.executemany(
+                "INSERT OR REPLACE INTO writes (thread_id, thread_ts, task_id, idx, channel, value) VALUES (?, ?, ?, ?, ?, ?)",
+                [
+                    (
+                        str(config["configurable"]["thread_id"]),
+                        str(config["configurable"]["thread_ts"]),
+                        task_id,
+                        idx,
+                        channel,
+                        self.serde.dumps(value),
+                    )
+                    for idx, (channel, value) in enumerate(writes)
+                ],
+            )
+
    async def aget_tuple(self, config: RunnableConfig) -> Optional[CheckpointTuple]:
        """Get a checkpoint tuple from the database asynchronously.

@@ -2,6 +2,7 @@ from __future__ import annotations

 import asyncio
 import concurrent.futures
+import json
 import time
 from collections import defaultdict, deque
 from functools import partial
@@ -23,6 +24,7 @@ from typing import (
    get_type_hints,
    overload,
 )
+from uuid import UUID, uuid5

 from langchain_core.callbacks.manager import AsyncParentRunManager, ParentRunManager
 from langchain_core.globals import get_debug
@@ -442,7 +444,7 @@ class Pregel(
            and signature(self.checkpointer.list).parameters.get("filter") is None
        ):
            raise ValueError("Checkpointer does not support filtering")
-        for config, checkpoint, metadata, parent_config in self.checkpointer.list(
+        for config, checkpoint, metadata, parent_config, _ in self.checkpointer.list(
            config, before=before, limit=limit, filter=filter
        ):
            with ChannelsManager(
@@ -489,6 +491,7 @@ class Pregel(
            checkpoint,
            metadata,
            parent_config,
+            _,
        ) in self.checkpointer.alist(config, before=before, limit=limit, filter=filter):
            async with AsyncChannelsManager(
                self.channels, checkpoint, config
@@ -565,6 +568,7 @@ class Pregel(
                deque(),
                None,
                [INTERRUPT],
+                str(uuid5(UUID(checkpoint["id"]), INTERRUPT)),
            )
            # execute task
            task.proc.invoke(
@@ -653,6 +657,7 @@ class Pregel(
                deque(),
                None,
                [INTERRUPT],
+                str(uuid5(UUID(checkpoint["id"]), INTERRUPT)),
            )
            # execute task
            await task.proc.ainvoke(
@@ -756,7 +761,7 @@ class Pregel(
        Args:
            input: The input to the graph.
            config: The configuration to use for the run.
-            stream_mode: The mode to stream output, defaults to 'values'.
+            stream_mode: The mode to stream output, defaults to self.stream_mode.
                Options are 'values', 'updates', and 'debug'.
                values: Emit the current values of the state for each step.
                updates: Emit only the updates to the state for each step.
@@ -777,7 +782,7 @@ class Pregel(
            >>> import operator
            >>> from typing_extensions import Annotated, TypedDict
            >>> from langgraph.graph import StateGraph
-            >>> from libs.langgraph.langgraph.constants import START
+            >>> from langgraph.constants import START
            ...
            >>> class State(TypedDict):
            ...     alist: Annotated[list, operator.add]
@@ -879,6 +884,23 @@ class Pregel(
                self.managed_values_dict, config, self
            ) as managed:

+                def put_writes(task_id: str, writes: Sequence[tuple[str, Any]]) -> None:
+                    if self.checkpointer is not None:
+                        bg.append(
+                            executor.submit(
+                                self.checkpointer.put_writes,
+                                {
+                                    **checkpoint_config,
+                                    "configurable": {
+                                        **checkpoint_config["configurable"],
+                                        "thread_ts": checkpoint["id"],
+                                    },
+                                },
+                                writes,
+                                task_id,
+                            )
+                        )
+
                def put_checkpoint(metadata: CheckpointMetadata) -> Iterator[Any]:
                    nonlocal checkpoint, checkpoint_config, channels

@@ -963,8 +985,7 @@ class Pregel(
                    # increment start to 0
                    start += 1
                else:
-                    # if received no input, take that as signal to proceed
-                    # past previous interrupt, if any
+                    # no input is taken as signal to proceed past previous interrupt
                    checkpoint = copy_checkpoint(checkpoint)
                    for k in self.stream_channels_list:
                        if k in checkpoint["channel_versions"]:
@@ -994,6 +1015,15 @@ class Pregel(
                        ),
                    )

+                    # assign pending writes to tasks
+                    if saved and saved.pending_writes:
+                        for task in next_tasks:
+                            task.writes.extend(
+                                (c, v)
+                                for tid, c, v in saved.pending_writes
+                                if tid == task.id
+                            )
+
                    # if no more tasks, we're done
                    if not next_tasks:
                        if step == start:
@@ -1027,12 +1057,15 @@ class Pregel(
                    futures = {
                        executor.submit(run_with_retry, task, self.retry_policy): task
                        for task in next_tasks
+                        if not task.writes
                    }
                    end_time = (
                        self.step_timeout + time.monotonic()
                        if self.step_timeout
                        else None
                    )
+                    if not futures:
+                        done, inflight = set(), set()
                    while futures:
                        done, inflight = concurrent.futures.wait(
                            futures,
@@ -1050,6 +1083,10 @@ class Pregel(
                                # exception will be handled in panic_or_proceed
                                futures.clear()
                            else:
+                                # save task writes to checkpointer, unless this
+                                # is the single or last task in this step
+                                if futures:
+                                    put_writes(task.id, task.writes)
                                # yield updates output for the finished task
                                if "updates" in stream_modes:
                                    yield from _with_mode(
@@ -1076,7 +1113,7 @@ class Pregel(

                    # combine pending writes from all tasks
                    pending_writes = deque[tuple[str, Any]]()
-                    for _, _, _, writes, _, _ in next_tasks:
+                    for _, _, _, writes, _, _, _ in next_tasks:
                        pending_writes.extend(writes)

                    if debug:
@@ -1167,6 +1204,73 @@ class Pregel(
        interrupt_after: Optional[Union[All, Sequence[str]]] = None,
        debug: Optional[bool] = None,
    ) -> AsyncIterator[Union[dict[str, Any], Any]]:
+        """Stream graph steps for a single input.
+
+        Args:
+            input: The input to the graph.
+            config: The configuration to use for the run.
+            stream_mode: The mode to stream output, defaults to self.stream_mode.
+                Options are 'values', 'updates', and 'debug'.
+                values: Emit the current values of the state for each step.
+                updates: Emit only the updates to the state for each step.
+                    Output is a dict with the node name as key and the updated values as value.
+                debug: Emit debug events for each step.
+            output_keys: The keys to stream, defaults to all non-context channels.
+            input_keys: The keys to use from the input, defaults to all input channels.
+            interrupt_before: Nodes to interrupt before, defaults to all nodes in the graph.
+            interrupt_after: Nodes to interrupt after, defaults to all nodes in the graph.
+            debug: Whether to print debug information during execution, defaults to False.
+
+        Yields:
+            The output of each step in the graph. The output shape depends on the stream_mode.
+
+        Examples:
+            Using different stream modes with a graph:
+            ```pycon
+            >>> import operator
+            >>> from typing_extensions import Annotated, TypedDict
+            >>> from langgraph.graph import StateGraph
+            >>> from langgraph.constants import START
+            ...
+            >>> class State(TypedDict):
+            ...     alist: Annotated[list, operator.add]
+            ...     another_list: Annotated[list, operator.add]
+            ...
+            >>> builder = StateGraph(State)
+            >>> builder.add_node("a", lambda _state: {"another_list": ["hi"]})
+            >>> builder.add_node("b", lambda _state: {"alist": ["there"]})
+            >>> builder.add_edge("a", "b")
+            >>> builder.add_edge(START, "a")
+            >>> graph = builder.compile()
+            ```
+            With stream_mode="values":
+
+            ```pycon
+            >>> async for event in graph.astream({"alist": ['Ex for stream_mode="values"']}, stream_mode="values"):
+            ...     print(event)
+            {'alist': ['Ex for stream_mode="values"'], 'another_list': []}
+            {'alist': ['Ex for stream_mode="values"'], 'another_list': ['hi']}
+            {'alist': ['Ex for stream_mode="values"', 'there'], 'another_list': ['hi']}
+            ```
+            With stream_mode="updates":
+
+            ```pycon
+            >>> async for event in graph.astream({"alist": ['Ex for stream_mode="updates"']}, stream_mode="updates"):
+            ...     print(event)
+            {'a': {'another_list': ['hi']}}
+            {'b': {'alist': ['there']}}
+            ```
+            With stream_mode="debug":
+
+            ```pycon
+            >>> async for event in graph.astream({"alist": ['Ex for stream_mode="debug"']}, stream_mode="debug"):
+            ...     print(event)
+            {'type': 'task', 'timestamp': '2024-06-23T...+00:00', 'step': 1, 'payload': {'id': '...', 'name': 'a', 'input': {'alist': ['Ex for stream_mode="debug"'], 'another_list': []}, 'triggers': ['start:a']}}
+            {'type': 'task_result', 'timestamp': '2024-06-23T...+00:00', 'step': 1, 'payload': {'id': '...', 'name': 'a', 'result': [('another_list', ['hi'])]}}
+            {'type': 'task', 'timestamp': '2024-06-23T...+00:00', 'step': 2, 'payload': {'id': '...', 'name': 'b', 'input': {'alist': ['Ex for stream_mode="debug"'], 'another_list': ['hi']}, 'triggers': ['a']}}
+            {'type': 'task_result', 'timestamp': '2024-06-23T...+00:00', 'step': 2, 'payload': {'id': '...', 'name': 'b', 'result': [('alist', ['there'])]}}
+            ```
+        """
        config = ensure_config(config)
        callback_manager = get_async_callback_manager_for_config(config)
        run_manager = await callback_manager.on_chain_start(
@@ -1240,6 +1344,24 @@ class Pregel(
                self.managed_values_dict, config, self
            ) as managed:

+                def put_writes(task_id: str, writes: Sequence[tuple[str, Any]]) -> None:
+                    if self.checkpointer is not None:
+                        bg.append(
+                            asyncio.create_task(
+                                self.checkpointer.aput_writes(
+                                    {
+                                        **checkpoint_config,
+                                        "configurable": {
+                                            **checkpoint_config["configurable"],
+                                            "thread_ts": checkpoint["id"],
+                                        },
+                                    },
+                                    writes,
+                                    task_id,
+                                )
+                            )
+                        )
+
                def put_checkpoint(metadata: CheckpointMetadata) -> Iterator[Any]:
                    nonlocal checkpoint, checkpoint_config, channels

@@ -1320,8 +1442,7 @@ class Pregel(
                    # increment start to 0
                    start += 1
                else:
-                    # if received no input, take that as signal to proceed
-                    # past previous interrupt, if any
+                    # no input is taken as signal to proceed past previous interrupt
                    checkpoint = copy_checkpoint(checkpoint)
                    for k in self.stream_channels_list:
                        if k in checkpoint["channel_versions"]:
@@ -1351,6 +1472,15 @@ class Pregel(
                        ),
                    )

+                    # assign pending writes to tasks
+                    if saved and saved.pending_writes:
+                        for task in next_tasks:
+                            task.writes.extend(
+                                (c, v)
+                                for tid, c, v in saved.pending_writes
+                                if tid == task.id
+                            )
+
                    # if no more tasks, we're done
                    if not next_tasks:
                        if step == start:
@@ -1387,10 +1517,13 @@ class Pregel(
                            arun_with_retry(task, self.retry_policy, do_stream)
                        ): task
                        for task in next_tasks
+                        if not task.writes
                    }
                    end_time = (
                        self.step_timeout + loop.time() if self.step_timeout else None
                    )
+                    if not futures:
+                        done, inflight = set(), set()
                    while futures:
                        done, inflight = await asyncio.wait(
                            futures,
@@ -1406,6 +1539,10 @@ class Pregel(
                                # exception will be handle in panic_or_proceed
                                futures.clear()
                            else:
+                                # save task writes to checkpointer, unless this
+                                # is the single or last task in this step
+                                if futures:
+                                    put_writes(task.id, task.writes)
                                # yield updates output for the finished task
                                if "updates" in stream_modes:
                                    for chunk in _with_mode(
@@ -1434,7 +1571,7 @@ class Pregel(

                    # combine pending writes from all tasks
                    pending_writes = deque[tuple[str, Any]]()
-                    for _, _, _, writes, _, _ in next_tasks:
+                    for _, _, _, writes, _, _, _ in next_tasks:
                        pending_writes.extend(writes)

                    if debug:
@@ -1671,7 +1808,7 @@ def _should_interrupt(
        # and any triggered node is in interrupt_nodes list
        and any(
            node
-            for node, _, _, _, config, _ in tasks
+            for node, _, _, _, config, _, _ in tasks
            if (
                (not config or TAG_HIDDEN not in config.get("tags"))
                if interrupt_nodes == "*"
@@ -1825,6 +1962,14 @@ def _prepare_next_tasks(
            continue
        if for_execution:
            if node := processes[packet.node].get_node():
+                triggers = [TASKS]
+                metadata = {
+                    "langgraph_step": step,
+                    "langgraph_node": packet.node,
+                    "langgraph_triggers": triggers,
+                    "langgraph_task_idx": len(tasks),
+                }
+                task_id = str(uuid5(UUID(checkpoint["id"]), json.dumps(metadata)))
                writes = deque()
                tasks.append(
                    PregelExecutableTask(
@@ -1836,14 +1981,7 @@ def _prepare_next_tasks(
                            merge_configs(
                                config,
                                processes[packet.node].config,
-                                {
-                                    "metadata": {
-                                        "langgraph_step": step,
-                                        "langgraph_node": packet.node,
-                                        "langgraph_triggers": [TASKS],
-                                        "langgraph_task_idx": len(tasks),
-                                    }
-                                },
+                                {"metadata": metadata},
                            ),
                            run_name=packet.node,
                            callbacks=(
@@ -1857,11 +1995,12 @@ def _prepare_next_tasks(
                                    _local_write, writes.extend, processes, channels
                                ),
                                CONFIG_KEY_READ: partial(
-                                    _local_read, checkpoint, channels, tasks, config
+                                    _local_read, checkpoint, channels, writes, config
                                ),
                            },
                        ),
-                        [TASKS],
+                        triggers,
+                        task_id,
                    )
                )
        else:
@@ -1879,7 +2018,7 @@ def _prepare_next_tasks(
    for name, proc in processes.items():
        seen = checkpoint["versions_seen"][name]
        # If any of the channels read by this process were updated
-        if triggers := [
+        if triggers := sorted(
            chan
            for chan in proc.triggers
            if not isinstance(
@@ -1887,7 +2026,7 @@ def _prepare_next_tasks(
            )
            and checkpoint["channel_versions"].get(chan, null_version)
            > seen.get(chan, null_version)
-        ]:
+        ):
            channels_to_consume.update(triggers)
            try:
                val = next(_proc_input(step, name, proc, managed, channels))
@@ -1906,8 +2045,14 @@ def _prepare_next_tasks(

            if for_execution:
                if node := proc.get_node():
+                    metadata = {
+                        "langgraph_step": step,
+                        "langgraph_node": name,
+                        "langgraph_triggers": triggers,
+                        "langgraph_task_idx": len(tasks),
+                    }
+                    task_id = str(uuid5(UUID(checkpoint["id"]), json.dumps(metadata)))
                    writes = deque()
-                    triggers = sorted(triggers)
                    tasks.append(
                        PregelExecutableTask(
                            name,
@@ -1918,14 +2063,7 @@ def _prepare_next_tasks(
                                merge_configs(
                                    config,
                                    proc.config,
-                                    {
-                                        "metadata": {
-                                            "langgraph_step": step,
-                                            "langgraph_node": name,
-                                            "langgraph_triggers": triggers,
-                                            "langgraph_task_idx": len(tasks),
-                                        }
-                                    },
+                                    {"metadata": metadata},
                                ),
                                run_name=name,
                                callbacks=(
@@ -1948,6 +2086,7 @@ def _prepare_next_tasks(
                                },
                            ),
                            triggers,
+                            task_id,
                        )
                    )
            else:
@@ -66,7 +66,7 @@ def map_debug_tasks(
    step: int, tasks: list[PregelExecutableTask]
 ) -> Iterator[DebugOutputTask]:
    ts = datetime.now(timezone.utc).isoformat()
-    for name, input, _, _, config, triggers in tasks:
+    for name, input, _, _, config, triggers, _ in tasks:
        if config is not None and TAG_HIDDEN in config.get("tags", []):
            continue

@@ -91,7 +91,7 @@ def map_debug_task_results(
    stream_channels_list: Sequence[str],
 ) -> Iterator[DebugOutputTaskResult]:
    ts = datetime.now(timezone.utc).isoformat()
-    for name, _, _, writes, config, _ in tasks:
+    for name, _, _, writes, config, _, _ in tasks:
        if config is not None and TAG_HIDDEN in config.get("tags", []):
            continue

@@ -138,7 +138,7 @@ def print_step_tasks(step: int, next_tasks: list[PregelExecutableTask]) -> None:
        )
        + "\n".join(
            f"- {get_colored_text(name, 'green')} -> {pformat(val)}"
-            for name, val, _, _, _, _ in next_tasks
+            for name, val, _, _, _, _, _ in next_tasks
        )
    )

@@ -105,7 +105,7 @@ def map_output_updates(
    if isinstance(output_channels, str):
        if updated := [
            (node, value)
-            for node, _, _, writes, _, _ in output_tasks
+            for node, _, _, writes, _, _, _ in output_tasks
            for chan, value in writes
            if chan == output_channels
        ]:
@@ -122,7 +122,7 @@ def map_output_updates(
                node,
                {chan: value for chan, value in writes if chan in output_channels},
            )
-            for node, _, _, writes, _, _ in output_tasks
+            for node, _, _, writes, _, _, _ in output_tasks
            if any(chan in output_channels for chan, _ in writes)
        ]:
            grouped = defaultdict(list)
@@ -18,6 +18,7 @@ class PregelExecutableTask(NamedTuple):
    writes: deque[tuple[str, Any]]
    config: RunnableConfig
    triggers: list[str]
+    id: str


 class StateSnapshot(NamedTuple):
@@ -22,6 +22,14 @@ from langchain_core.runnables.graph import Edge, Graph, Node, is_uuid
 from langchain_core.runnables.utils import accepts_config
 from typing_extensions import TypeGuard

+try:
+    from langchain_core.runnables.config import _set_config_context
+except ImportError:
+    # For forwards compatibility
+    def _set_config_context(context: RunnableConfig) -> None:  # type: ignore
+        """Set the context for the current thread."""
+        var_child_runnable_config.set(context)
+

 # Before Python 3.11 native StrEnum is not available
 class StrEnum(str, enum.Enum):
@@ -89,7 +97,7 @@ class RunnableCallable(Runnable):
        else:
            config = merge_configs(self.config, config)
            context = copy_context()
-            context.run(var_child_runnable_config.set, config)
+            context.run(_set_config_context, config)
            if accepts_config(self.func):
                kwargs["config"] = config
            ret = context.run(self.func, input, **kwargs)
@@ -110,7 +118,7 @@ class RunnableCallable(Runnable):
        else:
            config = merge_configs(self.config, config)
            context = copy_context()
-            context.run(var_child_runnable_config.set, config)
+            context.run(_set_config_context, config)
            if accepts_config(self.afunc):
                kwargs["config"] = config
            if sys.version_info >= (3, 11):
@@ -387,13 +387,13 @@ css = ["tinycss2 (>=1.1.0,<1.3)"]

 [[package]]
 name = "certifi"
-version = "2024.6.2"
+version = "2024.7.4"
 description = "Python package for providing Mozilla's CA Bundle."
 optional = false
 python-versions = ">=3.6"
 files = [
-    {file = "certifi-2024.6.2-py3-none-any.whl", hash = "sha256:ddc6c8ce995e6987e7faf5e3f1b02b302836a0e5d98ece18392cb1a36c72ad56"},
-    {file = "certifi-2024.6.2.tar.gz", hash = "sha256:3cd43f1c6fa7dedc5899d69d3ad0398fd018ad1a17fba83ddaf78aa46c747516"},
+    {file = "certifi-2024.7.4-py3-none-any.whl", hash = "sha256:c198e21b1289c2ab85ee4e67bb4b4ef3ead0892059901a8d5b622f24a1101e90"},
+    {file = "certifi-2024.7.4.tar.gz", hash = "sha256:5a1e7645bc0ec61a09e26c36f6106dd4cf40c6db3a1fb6352b0244e7fb057c7b"},
 ]

 [[package]]
@@ -1,6 +1,6 @@
 [tool.poetry]
 name = "langgraph"
-version = "0.1.6"
+version = "0.1.7"
 description = "Building stateful, multi-actor applications with LLMs"
 authors = []
 license = "MIT"
@@ -14,6 +14,7 @@ from typing import (
    Literal,
    Optional,
    Sequence,
+    Tuple,
    TypedDict,
    Union,
 )
@@ -37,6 +38,7 @@ from langgraph.channels.context import Context
 from langgraph.channels.last_value import LastValue
 from langgraph.channels.topic import Topic
 from langgraph.checkpoint.base import (
+    BaseCheckpointSaver,
    Checkpoint,
    CheckpointMetadata,
    CheckpointTuple,
@@ -193,6 +195,12 @@ def test_checkpoint_errors() -> None:
        ) -> RunnableConfig:
            raise ValueError("Faulty put")

+    class FaultyPutWritesCheckpointer(MemorySaver):
+        def put_writes(
+            self, config: RunnableConfig, writes: List[Tuple[str, Any]], task_id: str
+        ) -> RunnableConfig:
+            raise ValueError("Faulty put_writes")
+
    class FaultyVersionCheckpointer(MemorySaver):
        def get_next_version(self, current: Optional[int], channel: BaseChannel) -> int:
            raise ValueError("Faulty get_next_version")
@@ -200,10 +208,9 @@ def test_checkpoint_errors() -> None:
    def logic(inp: str) -> str:
        return ""

-    builder = Graph()
+    builder = StateGraph(Annotated[str, operator.add])
    builder.add_node("agent", logic)
-    builder.set_entry_point("agent")
-    builder.set_finish_point("agent")
+    builder.add_edge(START, "agent")

    graph = builder.compile(checkpointer=FaultyGetCheckpointer())
    with pytest.raises(ValueError, match="Faulty get_tuple"):
@@ -217,6 +224,13 @@ def test_checkpoint_errors() -> None:
    with pytest.raises(ValueError, match="Faulty get_next_version"):
        graph.invoke("", {"configurable": {"thread_id": "thread-1"}})

+    # add parallel node
+    builder.add_node("parallel", logic)
+    builder.add_edge(START, "parallel")
+    graph = builder.compile(checkpointer=FaultyPutWritesCheckpointer())
+    with pytest.raises(ValueError, match="Faulty put_writes"):
+        graph.invoke("", {"configurable": {"thread_id": "thread-1"}})
+

 def test_reducer_before_first_node() -> None:
    from langchain_core.messages import HumanMessage
@@ -944,6 +958,122 @@ def test_invoke_checkpoint(mocker: MockerFixture) -> None:
    assert checkpoint["channel_values"].get("total") == 5


+@pytest.mark.parametrize(
+    "checkpointer",
+    [
+        MemorySaverAssertImmutable(),
+        SqliteSaver.from_conn_string(":memory:"),
+    ],
+    ids=[
+        "memory",
+        "sqlite",
+    ],
+)
+def test_pending_writes_resume(checkpointer: BaseCheckpointSaver) -> None:
+    try:
+
+        class State(TypedDict):
+            value: Annotated[int, operator.add]
+
+        class AwhileMaker:
+            def __init__(self, sleep: float, rtn: Union[Dict, Exception]) -> None:
+                self.sleep = sleep
+                self.rtn = rtn
+                self.reset()
+
+            def __call__(self, input: State) -> Any:
+                self.calls += 1
+                time.sleep(self.sleep)
+                if isinstance(self.rtn, Exception):
+                    raise self.rtn
+                else:
+                    return self.rtn
+
+            def reset(self):
+                self.calls = 0
+
+        one = AwhileMaker(0.2, {"value": 2})
+        two = AwhileMaker(0.6, ValueError("I'm not good"))
+        builder = StateGraph(State)
+        builder.add_node("one", one)
+        builder.add_node("two", two)
+        builder.add_edge(START, "one")
+        builder.add_edge(START, "two")
+        graph = builder.compile(checkpointer=checkpointer)
+
+        thread1: RunnableConfig = {"configurable": {"thread_id": 1}}
+        with pytest.raises(ValueError, match="I'm not good"):
+            graph.invoke({"value": 1}, thread1)
+
+        # both nodes should have been called once
+        assert one.calls == 1
+        assert two.calls == 1
+
+        # latest checkpoint should be before nodes "one", "two"
+        state = graph.get_state(thread1)
+        assert state is not None
+        assert state.values == {"value": 1}
+        assert state.next == ("one", "two")
+        assert state.metadata == {"source": "loop", "step": 0, "writes": None}
+        # should contain pending write of "one"
+        checkpoint = checkpointer.get_tuple(thread1)
+        assert checkpoint is not None
+        assert checkpoint.pending_writes == [
+            (AnyStr(), "one", "one"),
+            (AnyStr(), "value", 2),
+        ]
+        # both pending writes come from same task
+        assert checkpoint.pending_writes[0][0] == checkpoint.pending_writes[1][0]
+
+        # resume execution
+        with pytest.raises(ValueError, match="I'm not good"):
+            graph.invoke(None, thread1)
+
+        # node "one" succeeded previously, so shouldn't be called again
+        assert one.calls == 1
+        # node "two" should have been called once again
+        assert two.calls == 2
+
+        # confirm no new checkpoints saved
+        state_two = graph.get_state(thread1)
+        assert state_two == state
+
+        # resume execution, without exception
+        two.rtn = {"value": 3}
+        # both the pending write and the new write were applied, 1 + 2 + 3 = 6
+        assert graph.invoke(None, thread1) == {"value": 6}
+    finally:
+        if getattr(checkpointer, "__exit__", None):
+            checkpointer.__exit__(None, None, None)
+
+
+def test_cond_edge_after_send() -> None:
+    class Node:
+        def __init__(self, name: str):
+            self.name = name
+            setattr(self, "__name__", name)
+
+        def __call__(self, state):
+            return state + [self.name]
+
+    def send_for_fun(state):
+        return [Send("2", state)]
+
+    def route_to_three(state) -> Literal["3"]:
+        return "3"
+
+    builder = StateGraph(list)
+    builder.add_node(Node("1"))
+    builder.add_node(Node("2"))
+    builder.add_node(Node("3"))
+    builder.add_edge(START, "1")
+    builder.add_conditional_edges("1", send_for_fun)
+    builder.add_conditional_edges("2", route_to_three)
+    graph = builder.compile()
+
+    assert graph.invoke(["0"]) == ["0", "1", "2", "3"]
+
+
 def test_invoke_checkpoint_sqlite(mocker: MockerFixture) -> None:
    adder = mocker.Mock(side_effect=lambda x: x["total"] + x["input"])

@@ -1,7 +1,6 @@
 import asyncio
 import json
 import operator
-import time
 from collections import Counter
 from contextlib import asynccontextmanager, contextmanager
 from typing import (
@@ -11,8 +10,11 @@ from typing import (
    AsyncIterator,
    Dict,
    Generator,
+    List,
+    Literal,
    Optional,
    Sequence,
+    Tuple,
    TypedDict,
    Union,
 )
@@ -75,6 +77,12 @@ async def test_checkpoint_errors() -> None:
        ) -> RunnableConfig:
            raise ValueError("Faulty put")

+    class FaultyPutWritesCheckpointer(MemorySaver):
+        async def aput_writes(
+            self, config: RunnableConfig, writes: List[Tuple[str, Any]], task_id: str
+        ) -> RunnableConfig:
+            raise ValueError("Faulty put_writes")
+
    class FaultyVersionCheckpointer(MemorySaver):
        def get_next_version(self, current: Optional[int], channel: BaseChannel) -> int:
            raise ValueError("Faulty get_next_version")
@@ -82,10 +90,9 @@ async def test_checkpoint_errors() -> None:
    def logic(inp: str) -> str:
        return ""

-    builder = Graph()
+    builder = StateGraph(Annotated[str, operator.add])
    builder.add_node("agent", logic)
-    builder.set_entry_point("agent")
-    builder.set_finish_point("agent")
+    builder.add_edge(START, "agent")

    graph = builder.compile(checkpointer=FaultyGetCheckpointer())
    with pytest.raises(ValueError, match="Faulty get_tuple"):
@@ -123,6 +130,21 @@ async def test_checkpoint_errors() -> None:
        ):
            pass

+    # add a parallel node
+    builder.add_node("parallel", logic)
+    builder.add_edge(START, "parallel")
+    graph = builder.compile(checkpointer=FaultyPutWritesCheckpointer())
+    with pytest.raises(ValueError, match="Faulty put_writes"):
+        await graph.ainvoke("", {"configurable": {"thread_id": "thread-1"}})
+    with pytest.raises(ValueError, match="Faulty put_writes"):
+        async for _ in graph.astream("", {"configurable": {"thread_id": "thread-2"}}):
+            pass
+    with pytest.raises(ValueError, match="Faulty put_writes"):
+        async for _ in graph.astream_events(
+            "", {"configurable": {"thread_id": "thread-3"}}, version="v2"
+        ):
+            pass
+

 async def test_node_cancellation_on_external_cancel() -> None:
    inner_task_cancelled = False
@@ -213,6 +235,11 @@ async def test_step_timeout_on_stream_hang() -> None:
        AsyncSqliteSaver.from_conn_string(":memory:"),
        None,
    ],
+    ids=[
+        "memory",
+        "aiosqlite",
+        "none",
+    ],
 )
 async def test_cancel_graph_astream(
    checkpointer: Optional[BaseCheckpointSaver],
@@ -279,6 +306,11 @@ async def test_cancel_graph_astream(
        AsyncSqliteSaver.from_conn_string(":memory:"),
        None,
    ],
+    ids=[
+        "memory",
+        "aiosqlite",
+        "none",
+    ],
 )
 async def test_cancel_graph_astream_events_v2(
    checkpointer: Optional[BaseCheckpointSaver],
@@ -327,7 +359,6 @@ async def test_cancel_graph_astream_events_v2(
        ) as stream:
            async for chunk in stream:
                if chunk["event"] == "on_chain_stream" and not chunk["parent_ids"]:
-                    print(time.perf_counter(), "got event out here", chunk)
                    got_event = True
                    assert chunk["data"]["chunk"] == {"alittlewhile": {"value": 2}}
                    break
@@ -1036,6 +1067,122 @@ async def test_invoke_checkpoint(mocker: MockerFixture) -> None:
    assert checkpoint["channel_values"].get("total") == 5


+@pytest.mark.parametrize(
+    "checkpointer",
+    [
+        MemorySaverAssertImmutable(),
+        AsyncSqliteSaver.from_conn_string(":memory:"),
+    ],
+    ids=[
+        "memory",
+        "sqlite",
+    ],
+)
+async def test_pending_writes_resume(checkpointer: BaseCheckpointSaver) -> None:
+    try:
+
+        class State(TypedDict):
+            value: Annotated[int, operator.add]
+
+        class AwhileMaker:
+            def __init__(self, sleep: float, rtn: Union[Dict, Exception]) -> None:
+                self.sleep = sleep
+                self.rtn = rtn
+                self.reset()
+
+            async def __call__(self, input: State) -> Any:
+                self.calls += 1
+                await asyncio.sleep(self.sleep)
+                if isinstance(self.rtn, Exception):
+                    raise self.rtn
+                else:
+                    return self.rtn
+
+            def reset(self):
+                self.calls = 0
+
+        one = AwhileMaker(0.2, {"value": 2})
+        two = AwhileMaker(0.6, ValueError("I'm not good"))
+        builder = StateGraph(State)
+        builder.add_node("one", one)
+        builder.add_node("two", two)
+        builder.add_edge(START, "one")
+        builder.add_edge(START, "two")
+        graph = builder.compile(checkpointer=checkpointer)
+
+        thread1: RunnableConfig = {"configurable": {"thread_id": 1}}
+        with pytest.raises(ValueError, match="I'm not good"):
+            await graph.ainvoke({"value": 1}, thread1)
+
+        # both nodes should have been called once
+        assert one.calls == 1
+        assert two.calls == 1
+
+        # latest checkpoint should be before nodes "one", "two"
+        state = await graph.aget_state(thread1)
+        assert state is not None
+        assert state.values == {"value": 1}
+        assert state.next == ("one", "two")
+        assert state.metadata == {"source": "loop", "step": 0, "writes": None}
+        # should contain pending write of "one"
+        checkpoint = await checkpointer.aget_tuple(thread1)
+        assert checkpoint is not None
+        assert checkpoint.pending_writes == [
+            (AnyStr(), "one", "one"),
+            (AnyStr(), "value", 2),
+        ]
+        # both pending writes come from same task
+        assert checkpoint.pending_writes[0][0] == checkpoint.pending_writes[1][0]
+
+        # resume execution
+        with pytest.raises(ValueError, match="I'm not good"):
+            await graph.ainvoke(None, thread1)
+
+        # node "one" succeeded previously, so shouldn't be called again
+        assert one.calls == 1
+        # node "two" should have been called once again
+        assert two.calls == 2
+
+        # confirm no new checkpoints saved
+        state_two = await graph.aget_state(thread1)
+        assert state_two == state
+
+        # resume execution, without exception
+        two.rtn = {"value": 3}
+        # both the pending write and the new write were applied, 1 + 2 + 3 = 6
+        assert await graph.ainvoke(None, thread1) == {"value": 6}
+    finally:
+        if getattr(checkpointer, "__aexit__", None):
+            await checkpointer.__aexit__(None, None, None)
+
+
+async def test_cond_edge_after_send() -> None:
+    class Node:
+        def __init__(self, name: str):
+            self.name = name
+            setattr(self, "__name__", name)
+
+        async def __call__(self, state):
+            return state + [self.name]
+
+    async def send_for_fun(state):
+        return [Send("2", state)]
+
+    async def route_to_three(state) -> Literal["3"]:
+        return "3"
+
+    builder = StateGraph(list)
+    builder.add_node(Node("1"))
+    builder.add_node(Node("2"))
+    builder.add_node(Node("3"))
+    builder.add_edge(START, "1")
+    builder.add_conditional_edges("1", send_for_fun)
+    builder.add_conditional_edges("2", route_to_three)
+    graph = builder.compile()
+
+    assert await graph.ainvoke(["0"]) == ["0", "1", "2", "3"]
+
+
 async def test_invoke_checkpoint_aiosqlite(mocker: MockerFixture) -> None:
    add_one = mocker.Mock(side_effect=lambda x: x["total"] + x["input"])

@@ -1,5 +1,14 @@
 import functools
+import sys
+import uuid
+from typing import TypedDict
+from unittest.mock import patch

+import langsmith
+import pytest
+
+from langgraph.graph import END, StateGraph
+from langgraph.graph.graph import CompiledGraph
 from langgraph.utils import is_async_callable, is_async_generator


@@ -70,3 +79,43 @@ def test_is_generator() -> None:
    assert not is_async_generator(sync_runnable)
    wrapped_sync_runnable = functools.wraps(sync_runnable)(sync_runnable)
    assert not is_async_generator(wrapped_sync_runnable)
+
+
+@pytest.fixture
+def rt_graph() -> CompiledGraph:
+    class State(TypedDict):
+        foo: int
+        node_run_id: int
+
+    def node(_: State):
+        from langsmith import get_current_run_tree  # type: ignore
+
+        return {"node_run_id": get_current_run_tree().id}  # type: ignore
+
+    graph = StateGraph(State)
+    graph.add_node(node)
+    graph.set_entry_point("node")
+    graph.add_edge("node", END)
+    return graph.compile()
+
+
+def test_runnable_callable_tracing_nested(rt_graph: CompiledGraph) -> None:
+    with patch("langsmith.client.Client", spec=langsmith.Client) as mock_client:
+        with patch("langchain_core.tracers.langchain.get_client") as mock_get_client:
+            mock_get_client.return_value = mock_client
+            with langsmith.tracing_context(enabled=True):
+                res = rt_graph.invoke({"foo": 1})
+    assert isinstance(res["node_run_id"], uuid.UUID)
+
+
+@pytest.mark.skipif(
+    sys.version_info < (3, 11),
+    reason="Python 3.11+ is required for async contextvars support",
+)
+async def test_runnable_callable_tracing_nested_async(rt_graph: CompiledGraph) -> None:
+    with patch("langsmith.client.Client", spec=langsmith.Client) as mock_client:
+        with patch("langchain_core.tracers.langchain.get_client") as mock_get_client:
+            mock_get_client.return_value = mock_client
+            with langsmith.tracing_context(enabled=True):
+                res = await rt_graph.ainvoke({"foo": 1})
+    assert isinstance(res["node_run_id"], uuid.UUID)
@@ -4,7 +4,17 @@ import asyncio
 import logging
 import os
 import sys
-from typing import Any, AsyncIterator, Dict, List, NamedTuple, Optional, Union, overload
+from typing import (
+    Any,
+    AsyncIterator,
+    Dict,
+    List,
+    NamedTuple,
+    Optional,
+    TypedDict,
+    Union,
+    overload,
+)

 import httpx
 import httpx_sse
@@ -29,6 +39,21 @@ from langgraph_sdk.schema import (
 logger = logging.getLogger(__name__)


+class RunCreate(TypedDict):
+    """Payload for creating a background run."""
+
+    thread_id: Optional[str]
+    assistant_id: str
+    input: Optional[dict]
+    metadata: Optional[dict]
+    config: Optional[Config]
+    checkpoint_id: Optional[str]
+    interrupt_before: Optional[list[str]]
+    interrupt_after: Optional[list[str]]
+    webhook: Optional[str]
+    multitask_strategy: Optional[MultitaskStrategy]
+
+
 def get_client(
    *, url: str = "http://localhost:8123", api_key: Optional[str] = None
 ) -> LangGraphClient:
@@ -561,6 +586,15 @@ class RunsClient:
        else:
            return await self.http.post("/runs", json=payload)

+    async def create_batch(self, payloads: list[RunCreate]) -> list[Run]:
+        """Create a batch of background runs."""
+
+        def filter_payload(payload: RunCreate):
+            return {k: v for k, v in payload.items() if v is not None}
+
+        payloads = [filter_payload(payload) for payload in payloads]
+        return await self.http.post("/runs/batch", json=payloads)
+
    @overload
    async def wait(
        self,
Author	SHA1	Message	Date
Jacob LeeandGitHub	b6c46d1a45	Update README.md	2024-07-11 15:42:32 -07:00
Jacob LeeandGitHub	9c7dafb75a	Update README.md	2024-07-11 15:42:16 -07:00
jacoblee93	b67c3d1edd	Sync readmes	2024-07-11 15:16:58 -07:00
Jacob LeeandGitHub	634fd082c5	Fix example CC @vbarda	2024-07-11 15:14:12 -07:00
Andrew NguonlyandGitHub	e9f7cd3e52	docs: Add Cloud Concepts docs, update cloud deployment how-to guide (#996 ) * Update Cloud deployment docs. Add Cloud Concepts docs. * Update quick start. Update how-to for setting up repo. * Delete unused image base_url.png. * Fix spelling error. Fix broken anchor link. * Fix broken links.	2024-07-11 14:33:25 -07:00
Vadym BardaandGitHub	39321ba042	docs: reduce number of cells in streaming how-tos (#997 )	2024-07-11 17:19:58 -04:00
Vadym BardaandGitHub	85ec1934de	langgraph: update stream_mode docstring for .stream() (#994 ) * langgraph: update stream_mode docstring for .stream() * add astream docstring * fix import	2024-07-11 09:08:19 -07:00
Vadym BardaandGitHub	45dbf8c656	sdk-py: remove defaults from typedef (#989 )	2024-07-10 18:08:05 -04:00
Nuno Campos	60763d4ddf	lib 0.1.7	2024-07-10 14:44:07 -07:00
Vadym BardaandGitHub	8c4da2c41c	sdk-py: add create_batch method (#988 )	2024-07-10 17:37:56 -04:00
Nuno CamposandGitHub	dfb2ac321f	lib: Checkpoint pending writes whenever a node finishes (#976 ) * lib: Checkpoint pending writes whenever a node finishes - Whenever a node finishes, checkpoint pending writes * Rename arg * Add tests, resume from pending writes * Add comment * Add descriptive error * Fix bug found by will * Fix comments * Lint * Don't save pending write if executing only one node in step	2024-07-10 14:28:06 -07:00
Lance MartinandGitHub	4d2456be40	Update HIL update / edit state documentation (#966 ) * Update user input example * Update editing graph state * Minor clarifications	2024-07-10 13:24:16 -07:00
Vadym BardaandGitHub	166aaa1bac	docs: update another langsmith key ref (#986 )	2024-07-10 12:33:34 -04:00
Vadym BardaandGitHub	7cad1658f6	docs: update key in local cloud quickstart (#985 )	2024-07-10 12:18:10 -04:00
Vadym BardaandGitHub	a3a9c6ceed	docs: add how-to on how to stream tokens from within a tool (#984 )	2024-07-10 12:16:18 -04:00
William FHandGitHub	4849501ccf	Copy core context in RunnableCallable (#973 )	2024-07-09 20:26:45 -07:00
Vadym BardaandGitHub	a6dddd322a	docs: add how-to for streaming tokens w/o langchain chat models (#972 )	2024-07-09 21:56:18 -04:00
Isaac FranciscoandGitHub	17785375b8	[docs]: adding LangGraph up documentation (#975 ) * first draft * harrison comments	2024-07-09 18:22:11 -07:00
William FHandGitHub	b632fd8c54	Add cleaned up csbot chatbot example (#971 )	2024-07-09 14:06:05 -07:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	f32daee7d5	chore(deps): bump certifi from 2024.6.2 to 2024.7.4 in /libs/langgraph (#969 ) Bumps [certifi](https://github.com/certifi/python-certifi) from 2024.6.2 to 2024.7.4. - [Commits](https://github.com/certifi/python-certifi/compare/2024.06.02...2024.07.04) --- updated-dependencies: - dependency-name: certifi dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2024-07-09 13:52:41 -07:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	df52115de0	chore(deps-dev): bump certifi in /libs/cli/examples (#967 ) Bumps [certifi](https://github.com/certifi/python-certifi) from 2024.6.2 to 2024.7.4. - [Commits](https://github.com/certifi/python-certifi/compare/2024.06.02...2024.07.04) --- updated-dependencies: - dependency-name: certifi dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2024-07-09 12:54:47 -07:00
Dhaval Thakkar GitHub Vadym Barda Vadym Barda	2eb7d14acd	docs: add Redis Checkpointer example (#961 ) --------- Co-authored-by: Vadym Barda <vadim.barda@gmail.com> Co-authored-by: Vadym Barda <vadym@langchain.dev>	2024-07-09 14:32:01 -04:00
Lance MartinandGitHub	2842dae598	Minor edit to low-level docs to clarify Super Steps (#951 )	2024-07-09 10:02:08 -07:00
Nuno Campos	2354066c6d	Format	2024-07-09 09:40:29 -07:00
Nuno Campos	4b564f69a8	Fix tests	2024-07-09 09:35:50 -07:00